A method for detecting prohibited article risk based on association relationship

Abstract

The embodiment of the application discloses a kind of forbidden article risk detection methods based on association, comprising: obtaining the multiple images of luggage or package to be detected;Each image is subjected to target detection, and the category and position of each entity in each image are obtained, wherein the entity includes article and article component;According to the detection result of single image, the instance subgraph of each image is constructed;And according to the similarity between each instance subgraph and the pre-constructed forbidden article knowledge graph, the potential combination forbidden article risk is identified;According to the detection result of multiple images, the user subgraph of each recipient is constructed;And according to the similarity between each user subgraph and each node pair in the knowledge graph, the potential continuity forbidden article risk is identified.The embodiment can identify forbidden article combination risk.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image detection and understanding, and particularly relate to a risk detection method for contraband based on association relationship. BACKGROUND

[0002] With the increasing demand for safety detection in the fields of logistics express, airport security and import and export supervision, the detection technology for contraband based on X-ray images has been widely studied and applied. However, the traditional risk detection methods for contraband based on image processing and deep learning still have certain limitations: they can only identify single objects or single batches of contraband, and are difficult to deal with the strategies commonly seen in actual transportation, such as splitting, disguising, mixed loading or batch transportation.

[0003] For example, a certain contraband can be disassembled into multiple parts for dispersed transportation, and these parts can form a complete contraband when combined, but traditional target detection often only regards them as ordinary parts, thereby ignoring the potential threat. For another example, if batteries, fireworks and compressed gas cylinders are separately packed in different packages or different areas, although each of them corresponds to a certain risk, the combination of them will form a greater hidden danger, and the traditional method lacks comprehensive judgment of such superimposed risks. Relying solely on the detection of single target features often cannot deal with such disguising means, and how to conduct in-depth identification of contraband risks based on the association relationship of the goods is a problem to be solved.

[0004] In the prior art, patent application CN118941565A discloses a contraband detection model training method and a contraband detection method, and patent application CN118279883A discloses a contraband detection method and device, computer equipment and storage medium, which cannot solve the above problems. SUMMARY

[0005] Embodiments of the present application provide a risk detection method for contraband based on association relationship to solve the above technical problems.

[0006] In a first aspect, embodiments of the present application provide a risk detection method for contraband based on association relationship, comprising:

[0007] obtaining multiple images of luggage or packages to be detected;

[0008] performing target detection on each image to obtain the category and position of each entity in each image, wherein the entity includes an article and an article component;

[0009] constructing an instance subgraph of each image according to the detection result of a single image, and identifying potential combined contraband risks according to the similarity between each instance subgraph and a pre-constructed contraband knowledge graph;

[0010] According to the detection results of multiple images, a user subgraph of each recipient is constructed; and according to the similarity between each user subgraph and each node pair in the knowledge graph, a potential continuity prohibited article risk is identified.

[0011] In a second aspect, an electronic device is provided, and the electronic device includes:

[0012] one or more processors;

[0013] a memory for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the prohibited article risk detection method based on the association relationship according to any embodiment.

[0015] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the program is executed by a processor to implement the prohibited article risk detection method based on the association relationship according to any embodiment.

[0016] In summary, the embodiments of the present application provide a prohibited article risk detection method based on the association relationship, which fuses visual perception and knowledge reasoning, analyzes the potential prohibited article risk in the package by constructing the association relationship among the articles, article components and their attributes, to cope with the current common prohibited article transportation concealment means, such as split transportation, disguised package, mixed loading of foreign objects and batch transfer, etc., proposes a collaborative detection strategy that fuses image target detection and knowledge graph reasoning, and makes up for the deficiencies of traditional methods in multi-object association analysis and context understanding.

[0017] Specifically, the embodiments first extract the entity category and position in the X-ray image by using a target detection algorithm, and then perform semantic mapping and logical reasoning on the detection results based on the predefined symbiotic, containing and attribute relationships in the knowledge graph, to identify the high-risk combination or suspected prohibited article information hidden in the image. Not only can the detection of single target features be realized, but also the association relationship between the articles can be used for joint reasoning, thereby effectively making up for the problem that the traditional prohibited article detection method relies on single article features and has insufficient hidden risk identification capability. In addition, the embodiments introduce a user subgraph mechanism to track and analyze the article receiving conditions of a user at different times and in different packages, to identify potential continuity illegal behavior, and further improve the breadth and depth of risk early warning of the system. The entire method improves the intelligent identification and early warning capability of prohibited article risks in complex security inspection environments, and can be applied to complex security inspection scenes such as express delivery logistics, airport security inspection, import and export supervision, etc. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a partial schematic diagram of the prohibited items knowledge graph provided by an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of a method for detecting prohibited items based on association relationships provided by an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a subgraph construction and matching reasoning process provided by an embodiment of the present invention;

[0022] Figure 4 is a flow chart of another method for detecting prohibited items risk based on association relationships provided by an embodiment of the present invention;

[0023] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0025] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0026] In the description of the present application, it is also necessary to point out that, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "linking" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium, or internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0027] The embodiment provides a risk detection method for prohibited articles based on association relationship, which deeply analyzes the association relationship of attributes and spatial positions of articles or components based on the detection results of articles or components of X-ray images and the prohibited article knowledge graph, so as to improve the accuracy of detecting the risk of prohibited articles. In order to illustrate the method, the target detection model and the prohibited article knowledge graph supporting the implementation of the method are introduced first.

[0028] In a specific embodiment, the target detection model can be constructed in advance by the following method for detecting articles or article components in images:

[0029] Step one, obtain X-ray images in various scenes, and perform preprocessing such as denoising, contrast enhancement, and size normalization on the images, and mark them by professional personnel. Optionally, first, obtain X-ray images of luggage or packages from actual scenes such as airport security, logistics center, import and export supervision, etc. The original data set is denoted as , where represents the X-ray image, and is the total number of images.

[0030] Then, the following operations are performed on each image to improve the effect of subsequent detection and analysis:

[0031] S1, in order to eliminate random noise in the image, Gaussian filtering is performed on each image . Denote the new image obtained after filtering as , where is a Gaussian kernel with a standard deviation of , and represents convolution operation.

[0032] S2, in order to enhance the local contrast and make the object edge clearer, CLAHE (Contrast Limited Adaptive Histogram Equalization) is performed on the Gaussian filtered image to obtain the equalized image , where . represent the specific operations in CLAHE.

[0033] S3, uniformly scaling the equalized image to a size matching the subsequent detection model, such as . Optionally, let the input image size be , and the output image size be , then for the output coordinate , the floating point coordinate in the input image is given by equation (1) (here, center alignment is performed using "+0.5"):

[0034]

[0035] Let , and the pixel can be output by bilinear interpolation calculation using equation (2):

[0036]

[0037] Let , and the image of size can be obtained.

[0038] Finally, the pre-processed image is marked by professional annotators for prohibited items, and the corresponding annotation information is obtained, where is the bounding box coordinates and size of the prohibited item or prohibited item component i in the image, , , and represent the horizontal and vertical coordinates of the bounding box and the width and height, respectively; is the category of the prohibited item or prohibited item component i (such as battery, fireworks, compressed gas cylinder, etc.), and the final constructed dataset is , which is used for training and testing of the subsequent target detection model.

[0039] Step two, build a target detection model suitable for X-ray images, train using the data annotated in step one, and obtain model parameters with high detection accuracy. The trained model is used to identify the category and location of each item in the image. Optionally, use the DETR (Detection Transformer) target detection model based on transformer to predict the X-ray image, and the parameterization function of the model is as shown in equation (3):

[0040]

[0041] where, input image output several prediction results, including bounding box , class and confidence .

[0042] To optimize the detection accuracy and positioning effect at the same time, a comprehensive loss function as shown in formula (4) is set in the model training :

[0043]

[0044] wherein, the classification loss Focal Loss is used to distinguish different classes; the bounding box regression loss IoU Loss is used to improve the accuracy of target positioning; and are balance coefficients, used to adjust the weights of classification loss and regression loss in the total loss, and N1 represents the number of samples in the same training batch.

[0045] The labeled training set is used to iteratively train the detection model, and the model parameters are constantly updated through back propagation and optimization algorithm Adam , to obtain the trained model . In the inference stage, formula (5) is used for target detection:

[0046]

[0047] wherein, represents the detection result.

[0048] In a specific embodiment, the prohibited item knowledge graph can be pre-constructed in the following way: given the association information of various common prohibited items and their components, and the association relationship between the involved items, components and key attributes is represented in a structured form in the knowledge graph, a prohibited item association relationship knowledge graph (hereinafter referred to as prohibited item knowledge graph) is constructed.

[0049] Optionally, the association relationship here can be refined into three categories: symbiotic relationship, containing relationship and attribute relationship, each of which is important for different security scenarios and risk combinations. The symbiotic relationship refers to the case where multiple objects form potential prohibited items after combination, i.e. these objects have the possibility of coexistence in space or transportation logic; the containing relationship describes the structural pattern of a certain type of object being nested or hidden in another object; the attribute relationship focuses on the semantic association of the features of the object, including color, shape, material, composition, etc.

[0050] For example,Figure 1 The built part of the knowledge graph is demonstrated, and the associated information of various common prohibited articles is sorted based on the knowledge of professionals. The symbiotic relationship, the containing relationship and the attribute relationship are structurally described in the knowledge graph to obtain the knowledge graph , wherein, is a node set, and each node represents an article or an attribute or component (such as "battery", "fireworks", "deep red", "powder", etc.) thereof; is a relationship set, representing the association relationship between nodes, which is divided into three categories: containing (such as "package", "mixing"), symbiotic ("composition", "coexistence"), and attribute (such as "color", "shape"), and each relationship is assigned a risk coefficient , indicating the danger degree of the article combination, and the larger the value is, the higher the potential risk is, for example: .

[0051] Based on the above target detection model and the prohibited article knowledge graph, Figure 2 is a flowchart of a prohibited article risk detection method based on an association relationship provided by an embodiment of the present application. The method uses a target detection model to extract entity information in an image, and combines a knowledge graph to construct the association relationship between entities, thereby deeply mining the potential information of prohibited articles or high-risk combinations in the image, and significantly improving the recognition efficiency and risk warning level of prohibited articles in a complex security environment. The method is executed by an electronic device, as shown in Figure 2 , specifically comprising:

[0052] S110, obtaining multiple images of luggage or packages to be detected.

[0053] This step obtains multiple X-ray images newly input as the data source of the entire method. These images can cover different times and different places, and each X-ray image includes at least one luggage or package.

[0054] S120, performing target detection on each image to obtain the category and position of each entity in each image, wherein the entity includes an article and an article component.

[0055] This step uses the trained detection model to infer each X-ray image , and each image obtains a set of detection results , wherein, represents the number of detected entities, represents the detailed detection result of the i-th entity.

[0056] S130: construct instance subgraphs of each image based on the detection results of the single image; and identify potential combined prohibited item risks based on the similarity between each instance subgraph and a pre-constructed prohibited item knowledge graph.

[0057] This step firstly uses the detection results of a single image Constructing instance subgraph ,in, is the node set of all entities and their attributes detected in the p-th image; It represents a collection of various association relationships (such as coexistence, intermingling, inclusion, and attributes) between nodes. In a specific embodiment, the process may include the following steps:

[0058] Step 1: Generate entity nodes of the instance subgraph of any image based on the categories of each entity in the image. That is, take the category of each entity as an entity node in the instance subgraph.

[0059] Step 2: According to the position of each entity in any image, determine the inclusion relationship, mixed relationship and coexistence relationship between each entity node, and add the relationship edge corresponding to each relationship. (the big box surrounds the small box), then the entity is considered Contains entities , and add an inclusion edge to the instance subgraph ;like , and the overlapping area reaches the threshold (intersection over union ratio ), then the entity is considered and For a mixed relationship, add a mixed relationship edge in the instance subgraph ;like But if they are adjacent in space, then the entities and For coexistence relationships, add coexistence relationship edges in the instance subgraph .

[0060] Step 3: Identify entity attribute information from the entity names of any image, construct corresponding attribute nodes, and add attribute edges. For example, if the item node If the item name contains information such as color and shape (such as "red powder" and "cylindrical can"), the attribute node is disassembled from the item name. Add to , and add attribute edges .

[0061] After obtaining the instance subgraph of a single image, rule matching and logical reasoning are performed in combination with the contraband knowledge graph. If there is a contraband combination in the single image that triggers the graph rule, it is determined that the image has a potential combined contraband behavior. In a specific embodiment, the process can include the following steps:

[0062] Step one, convert each node in any instance subgraph into an embedding representation. Take the instance subgraph as an example, first, use the BERT model to perform semantic understanding on each node to obtain an embedding representation , where represents the specific operation of converting to an embedding representation using the BERT model.

[0063] Step two, use the attention pooling mechanism to weight and fuse the embedding representations of each node to obtain the embedding representation of the instance subgraph. Optionally, first calculate the attention weight score of each node's embedding representation using formula (6) :

[0064]

[0065] where , , are preset attention parameters, , , , represents the output dimension of the BERT model, represents the dimension of the intermediate attention space, and R represents the real number space corresponding to the dimension; is a nonlinear activation function.

[0066] Then, use formula (7) to weight and sum the node embeddings according to the attention weights to obtain the overall representation vector of the instance subgraph :

[0067]

[0068] This vector can comprehensively consider the importance of each node in the graph, so that nodes with stronger contraband risk relevance occupy a larger weight in aggregation, thereby enhancing the discriminability of the subgraph representation.

[0069] Step three, using the same method, the high-risk sub-structure in the pre-constructed prohibited item knowledge graph is processed to obtain the embedding representation of the high-risk sub-structure. First, the semantic understanding of each node in the high-risk sub-structure is obtained by using the BERT model to obtain the embedding representation; then the attention pooling mechanism is used to weight and fuse the embedding representation of each node to obtain the embedding representation of the high-risk sub-structure .

[0070] Step four, calculating the similarity between the embedding representation of any instance subgraph and the embedding representation of the high-risk sub-structure . If the similarity is higher than the set threshold, it is identified that there is a prohibited item risk in the any instance subgraph. Optionally, the similarity between and is calculated by formula (8) , and whether there is a potential prohibited item combination risk in the current image is judged by formula (9):

[0071]

[0072]

[0073] , wherein is the risk triggering threshold, and if the threshold is exceeded, it is considered that there is a risk in the instance subgraph. Since each node in the instance subgraph belongs to the same time slice, the influence of the time factor is not considered in this stage.

[0074] S140, constructing a user subgraph for each recipient according to the detection results of multiple images; and identifying a potential continuous prohibited item risk according to the similarity between each user subgraph and each node pair in the knowledge graph.

[0075] The so-called continuous prohibited item risk refers to a combination of prohibited item risks across time. In this step, a user subgraph is created according to the detection results in S120, and all entity information carried or received by the user at different times or batches is recorded in the subgraph as the basis for cross-time and cross-image risk identification.

[0076] Optionally, the user subgraph can be recorded as , , wherein represents a user node, represents a set of all entity nodes carried or received by the user, represents a set of edges between the user and each entity node. is the time length for the first entity to join the user subgraph, that is, the time length from the time when the user receives the entity to the current time, , The number of received entities is represented. The user subgraph is used to represent the user's behavior of receiving the combination of prohibited items at different times, and the structure is star-shaped scattering.

[0077] In a specific embodiment, the construction of the user subgraph can include the following steps:

[0078] Step one, for the user subgraph of any user, the any user is taken as the center node. Each user subgraph takes a recipient as the center node.

[0079] Step two, from the detection results of multiple images, extract the entities in the luggage or package received by the any user.

[0080] Step three, according to the extracted entities, construct the entity nodes of the user subgraph, and establish edges with the center node respectively.

[0081] Step four, record the duration of receiving the corresponding entity in each entity node, and update dynamically.

[0082] After each detection task, update the user subgraph: if the user receives a new item or item component , add the relationship edge , and update the "retention" time of all entity nodes in real time .

[0083] After obtaining each user subgraph, cross-image reasoning is performed in combination with the knowledge graph. If multiple items or item components received by the same user at different time periods constitute an illegal combination relationship, it is identified as a potential continuous prohibited item risk. In a specific embodiment, the process can include the following steps:

[0084] Step one, respectively calculate the similarity of two target entities in any user subgraph and two entities in the pre-constructed prohibited item knowledge graph. Optionally, use formula (10) to calculate the similarity between each entity node in the user subgraph and each entity node in the association relationship knowledge graph , and match to obtain possible prohibited items or prohibited item components in the user subgraph:

[0085]

[0086] Wherein, and represent the embedding representation of entity nodes and , respectively, represents the similarity of two nodes. If the similarity is greater than a set threshold, it is judged that the entity node for possible contraband or contraband components.

[0087] Step two, identify potential continuity contraband risk according to the similarity of each target entity and the time length of each target entity received by the user. Assuming that two specific entity nodes and in the user subgraph match two entity nodes and in the knowledge graph respectively, then the risk coefficient of the two target entities and is determined according to the following formula:

[0088]

[0089] wherein, represents the similarity of and , represents the similarity of and , and respectively represent the time when the user last carried or received and ; and respectively represent the time length of the user receiving and ; is a pre-constructed time length decay function, the longer the time length of the user receiving the target entity from the current time, the smaller the value of and respectively represent the matching sparsity coefficient at the node level, and respectively represent the time factor weight coefficient; and Optionally, wherein, is an adjustable factor, is a time interval between the current time and the time when the user receives the target entity.

[0090] If , it is considered that and correspond to potential continuity contraband risk; otherwise, it is considered safe. Figure 3 is a schematic diagram of the entire subgraph construction and matching reasoning process described above, which can be understood in combination with Figure 3 , wherein q in the diagram represents the number of risks identified.

[0091] ​Finally, the risk identification results obtained by S120-S140 are fused, and the analysis conclusion is visualized in the original image in the form of image superposition marking, and the alarm mechanism is triggered if necessary to prompt the staff to manually review or take follow-up processing measures.

[0092] Optionally, after completing target detection and knowledge graph reasoning, the information fusion stage is entered, and the detection results obtained by the target detection model and the high-risk combination information obtained by the graph reasoning are fused. In a specific embodiment, the fusion process includes the following steps:

[0093] Step 1: Perform entity alignment, based on the component or attribute information in the graph, trace back each entity component or attribute in each risk combination to the complete item to which it belongs, and update the complete item and the entity component or attribute information to the together.

[0094] Step 2: If the entities of the combined risk relationship (including the combined prohibited article risk and the continuous prohibited article risk) correspond to different target boxes in the image, then highlight the association of these entities.

[0095] Step 3: Re-correct the confidence of each target box according to formula (12):

[0096]

[0097] wherein, represents the confidence of the target box after correction, represents the set of other entities that have a combined risk relationship with the entity in the target box , represents the risk coefficient of and , represents the weight coefficient.

[0098] Since the same entity may have an association relationship with multiple entities, to highlight the highest potential risk, this embodiment selects the highest risk score in all association relationships for fusion as the final risk possibility output. After information fusion is completed, the system will perform visual marking on the original X image and provide an alarm prompt to assist staff in quickly identifying potential risk targets.

[0099] Figure 4 is a flowchart of another prohibited article risk detection method based on association relationships provided by the embodiments of the present application, including the complete process from target detection model construction, knowledge graph construction to risk identification. The above method details can also be combinedFigure 4 Understand.

[0100] In summary, this embodiment provides a method for detecting the risk of prohibited items based on association relationships, which integrates visual perception and knowledge reasoning. By constructing association relationships between items, item components and their attributes, it analyzes the potential risks of prohibited items in packages to address the current common concealed means of transporting prohibited items, such as split transportation, disguised packages, mixed loading of foreign objects and batch transshipment. A collaborative detection strategy that integrates image target detection and knowledge graph reasoning is proposed to make up for the shortcomings of traditional methods in multi-object association analysis and context understanding.

[0101] Specifically, compared with the prior art, the advantages of this embodiment include:

[0102] 1. This embodiment enhances the ability to identify covert transportation. By introducing a modeling mechanism for the symbiosis, inclusion, and attribute relationships between items, it can effectively identify complex covert methods such as split transportation, disguised cover-up, mixed loading of foreign objects, and batch transfers, significantly improving the comprehensiveness and practicality of prohibited item detection.

[0103] 2. This embodiment not only leverages the single-object detection capabilities of traditional deep learning models but also integrates knowledge graph reasoning to enable logical judgment of relationships between multiple objects, overcoming the technical bottleneck of existing methods that prevent them from identifying high-risk combinations.

[0104] 3. This embodiment introduces a user subgraph mechanism to track and analyze the receipt of items by users at different times and in different packages, enabling the identification of potential continuous illegal activities and further enhancing the breadth and depth of the system's risk warning capabilities.

[0105] 4. After completing graph inference, this embodiment can overlay the inference results on the original image in a visual form, supplemented by an alarm mechanism, to achieve intuitive presentation and rapid response of detection results, facilitating manual review and security inspection and disposal decision-making.

[0106] It should be noted that the user data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0107] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 5The processor 60 in the device is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected through a bus or other means, Figure 5 The connection through the bus is taken as an example.

[0108] The memory 61 can be used for storing software programs, computer executable programs and modules, such as program instructions / modules corresponding to the risk detection method of prohibited articles based on association relationship in the embodiments of the present application. The processor 60 executes the software programs, instructions and modules stored in the memory 61, thereby performing various function applications and data processing of the device, that is, implementing the risk detection method of prohibited articles based on association relationship.

[0109] The memory 61 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal and the like. In addition, the memory 61 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device or other non-volatile solid-state memory device. In some examples, the memory 61 can further include a memory remotely arranged relative to the processor 60, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0110] The input device 62 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device. The output device 63 can include a display device such as a display screen.

[0111] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the risk detection method of prohibited articles based on association relationship in any of the embodiments.

[0112] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0113] The computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave, in which the computer-readable program code is contained. Such propagated data signal can take a variety of forms, including but not limited to electro-magnetic, optical or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can be used to carry or propagate program code that is used by or in connection with an instruction execution system, apparatus or device.

[0114] The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0115] The computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting prohibited items based on association relationships, characterized in that: include: Acquire multiple images of the luggage or parcel to be inspected; Performing object detection on each image to obtain the category and location of each entity in each image, where entities include objects and object components; Based on the detection results of a single image, an instance subgraph of each image is constructed; specifically, based on the category of each entity in any image, entity nodes of the instance subgraph of the image are generated; based on the position of each entity in the image, inclusion relationships, mixed relationships, and coexistence relationships between the entity nodes are determined, and relationship edges corresponding to each relationship are added; entity attribute information is identified from the entity names of each entity in the image, and corresponding attribute nodes are constructed and attribute edges are added; Identify potential combinatorial prohibited items risks based on the similarity between each instance subgraph and the pre-built prohibited items knowledge graph; Based on the detection results of multiple images, a user subgraph for each recipient is constructed. Specifically, for any user's user subgraph, the user is used as the central node. Entities in the luggage or parcel with the user as the recipient are extracted from the detection results of multiple images. Based on each extracted entity, an entity node of the user subgraph is constructed, and edges are established between each entity node and the central node. The duration of receipt of the corresponding entity is recorded in each entity node and dynamically updated. Potential continuous prohibited item risks are identified based on the similarity between each user subgraph and each node pair in the knowledge graph.

2. The method according to claim 1, characterized in that The method of identifying potential combined prohibited items risks based on the similarity between each instance subgraph and the pre-built prohibited items knowledge graph includes: Convert each node in any instance subgraph into an embedded representation; Using the attention pooling mechanism, the embedded representations of each node are weightedly fused to obtain the embedded representation of any instance subgraph; Using the same method, the high-risk substructure in the pre-built prohibited items knowledge graph is processed to obtain an embedded representation of the high-risk substructure; The similarity between the embedding representation of any one of the instance subgraphs and the embedding representation of the high-risk substructure is calculated; if the similarity is higher than a set threshold, it is identified that there is a risk of prohibited items in any one of the instance subgraphs.

3. The method according to claim 1, characterized in that The nodes of the user subgraph include user nodes and entity nodes with the user nodes as recipients; The identifying of potential continuous prohibited item risks based on the similarity between each user subgraph and each node pair in the knowledge graph includes: Calculate the similarity between the two target entities in any user subgraph and the two entities in the pre-built prohibited items knowledge graph; Potential continuous prohibited item risks are identified based on the similarities and the time intervals when any one of the users receives each target entity.

4. The method according to claim 3, characterized in that The identifying of potential continuous prohibited item risks based on the similarities and the time duration of each target entity received by any user includes: According to the following formula, the two target entities are determined and The risk factor : ， in, and Represent two entities in the pre-built prohibited items knowledge graph, express and The similarity of express and The similarity of and Respectively indicate that any user receives and duration; It is a pre-built time decay function. The longer the time from the user receives the target entity to the current time, The smaller the value of ; 、 、 and Respectively represent weight coefficients; according to the risk coefficients, identify whether the two target entities correspond to potential continuous prohibited items risks.

5. The method according to claim 1, wherein The performing target detection on each image to obtain the category and position of each entity in each image includes: performing target detection on each image to obtain the category and target frame of each entity in each image; Correspondingly, after identifying the potential continuous prohibited item risk based on the similarity between each user subgraph and each node pair in the knowledge graph, it also includes: if each entity with potential combined prohibited item risk or continuous prohibited item risk corresponds to a different target box in the image, the different target boxes are associated and highlighted.

6. The method according to claim 5, characterized in that The target detection is performed on each image to obtain the category and position of each entity in each image, including: performing target detection on each image to obtain the target frame of each entity in each image and confidence ; Correspondingly, after the different target frames are associated and highlighted, the method further includes: correcting the confidence of each target frame according to the following formula: ， in, Represents the target box The corrected confidence level, Representation and target box Other entities that have a combined prohibited items risk or a continuous prohibited items risk A collection of express and The risk factor, Represents the weight coefficient.

7. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the association-based prohibited item risk detection method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that A computer program is stored thereon, which, when executed by a processor, implements the risk detection method for prohibited items based on association relationships as described in any one of claims 1-6.

Images