Methods, apparatus and devices for identifying target objects in images
By using a feature enhancement model to improve the feature extraction and recognition capabilities of target objects in X-ray images by leveraging global category prototype features, the problem of decreased X-ray image detection performance is solved, and more efficient target recognition is achieved.
Patent Information
- Application Number
- CN202210758316.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-29
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2042-06-29
AI Technical Summary
Existing machine learning models struggle to effectively identify target objects in X-ray images because X-ray images are heavily occluded and lack color, making feature extraction difficult and lacking in distinguishability, thus leading to decreased detection performance.
A feature enhancement model is adopted to enhance the features of the target object by using global category prototype features obtained through training. This includes object detection, prototype aggregation, and feature fusion networks to improve feature extraction and recognition capabilities.
It effectively enhances the feature representation of the target object, improves the target detection performance of the machine model, and can more accurately identify the target object in the image.
Smart Images
Figure CN115240135B_ABST
Abstract
Description
Technical Field
[0001] This invention relates generally to the field of image processing, and more specifically to methods, apparatus and devices for identifying target objects in images. Background Technology
[0002] As society continues to develop, more and more scenarios require the acquisition and analysis of scene images to solve problems. However, due to various reasons, the acquired images may have various problems, making it difficult to extract features or the extracted features may not meet the requirements. For example, public transportation security checks require X-ray machines to capture images of passengers' luggage to determine if there are any prohibited items. Current security checks are usually performed manually to inspect the captured images for prohibited items, but manual security checks suffer from high labor costs, many missed or false detections, and low detection efficiency. Therefore, machines can be used to replace manual security checks.
[0003] However, unlike natural light images which contain rich texture information, X-ray images generated by X-ray machines suffer from severe occlusion and lack of color, resulting in existing machine models being able to extract very few features of the target object and lacking discriminability. This makes it impossible for them to correctly distinguish the category of the target object, thus producing incorrect predictions and causing a decline in detection performance.
[0004] Therefore, there is a need for a new method, apparatus, and device for identifying target objects in images to solve the above problems. Summary of the Invention
[0005] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. The summary section of this invention is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.
[0006] According to one aspect of the present invention, a method for identifying a target object in an image is provided, the method comprising: acquiring an image to be detected, the image containing the target object to be identified; inputting the image into a trained feature enhancement model to enhance the features of the target object based on stored global category prototype features associated with the target object, wherein the global category prototype features are obtained and stored during training of the feature enhancement model; and identifying the target object in the image based on the enhanced features of the target object.
[0007] In one embodiment, inputting the image into a trained feature enhancement model to enhance the features of the target object based on stored global category prototype features associated with the target object includes: performing feature extraction on the image to obtain an image feature map; performing object detection on the image feature map to obtain candidate boxes surrounding the target object; mapping the candidate boxes onto the image feature map to obtain candidate box features; and fusing the candidate box features and the global category prototype features corresponding to the candidate boxes to enhance the features of the target object.
[0008] In one embodiment, fusing the candidate box features and the global category prototype features corresponding to the candidate boxes to enhance the features of the target object includes: calculating the similarity between the candidate box features and each global category prototype feature to determine the global category prototype features corresponding to the candidate boxes.
[0009] In one embodiment, the image to be detected is an X-ray image, and the target object is a contraband.
[0010] In one embodiment, the trained feature enhancement model includes an object detection network, a prototype aggregation network, and a feature fusion network, which is trained through the following steps: acquiring a training image set containing target objects, and labeling each object in the training image set; extracting features from each training image in the training image set via the object detection network in the feature enhancement model to obtain an image feature map of the training image, performing object detection and filtering on the image feature map to obtain candidate boxes surrounding the target objects in the training image; processing the image feature map and the candidate boxes via the prototype aggregation network in the feature enhancement model to obtain candidate box features corresponding to the candidate boxes and global category prototype features for the category to which the target object belongs, wherein the global category prototype features are stored and used to enhance the features of the target object to be identified in the image to be detected based on the global category prototype features during the testing phase; fusing the candidate box features and the global category prototype features corresponding to the candidate boxes via the feature fusion network in the feature enhancement model to enhance the features of the target object; and calculating a loss based on the global category prototype features and updating the parameters of the feature enhancement model based on the loss.
[0011] In one embodiment, the process of processing the image feature map and the candidate boxes via the prototype aggregation network in the feature enhancement model to obtain candidate box features corresponding to the candidate boxes and global category prototype features for the category to which the target object belongs includes: mapping the candidate boxes onto the image feature map to obtain the candidate box features; dividing the candidate box features according to different target objects to obtain a target candidate box feature set; and establishing target prototype features for each target object based on the target candidate box feature set; obtaining a category probability vector for the category to which the target object belongs based on the target prototype features; calculating category prototype features for the category to which the target object belongs based on the target prototype features and the category probability vector; and establishing global category prototype features based on the category prototype features, and updating the global category prototype features in each training iteration.
[0012] In one embodiment, the process of fusing the candidate box features and the global category prototype features corresponding to the candidate box via a feature fusion network in the feature enhancement model to enhance the features of the target object includes: determining the global category prototype features corresponding to the candidate box based on the category label of the target object corresponding to the candidate box features.
[0013] According to another aspect of the present invention, an apparatus for identifying a target object in an image is provided, the apparatus comprising a memory and a processor, the memory storing a computer program executed by the processor, the computer program, when executed by the processor, causing the processor to perform the method for identifying a target object in an image as described above.
[0014] According to another aspect of the present invention, an apparatus is provided, the apparatus comprising an image acquisition device and a device for identifying a target object in an image as described above, wherein the image acquisition device is used to acquire an image of an object to be detected to obtain an image to be detected, and the device for identifying a target object in an image is used to process the image to be detected to identify the target object in the image based on enhanced features of the target object.
[0015] According to another aspect of the present invention, a computer-readable medium is provided, wherein computer-executable instructions are stored thereon, which, when executed, perform the method for identifying a target object in an image as described above.
[0016] The method, apparatus, and device for identifying target objects in an image according to embodiments of the present invention can enhance the representation features of the target and identify the target based on the enhanced representation features. This effectively solves the problem of performance degradation in target detection tasks caused by weak target representation features, makes it easier to distinguish target categories, and improves the target detection performance of machine models. Attached Figure Description
[0017] The following drawings, which are incorporated herein by reference as part of this invention, are provided for understanding the invention. The drawings illustrate embodiments of the invention and their descriptions, serving to explain the principles of the invention.
[0018] In the attached image:
[0019] Figure 1 An exemplary flowchart of a method for identifying a target object in an image according to an embodiment of the present invention is shown;
[0020] Figure 2 A schematic structural block diagram of a feature enhancement model according to an embodiment of the present invention is shown;
[0021] Figure 3 A schematic structural block diagram of an apparatus for identifying a target object in an image according to an embodiment of the present invention is shown; and
[0022] Figure 4 A schematic structural block diagram of a device according to an embodiment of the present invention is shown. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.
[0024] As mentioned above, the captured images may have a variety of problems, making it difficult to solve these problems by extracting image features.
[0025] Therefore, in order to improve the ability of machine models to solve problems using image features, the present invention provides a method for identifying target objects in an image, the method comprising: acquiring an image to be detected, the image containing a target object to be identified; inputting the image into a trained feature enhancement model to enhance the features of the target object based on stored global category prototype features associated with the target object, wherein the global category prototype features are obtained and stored during training of the feature enhancement model; and identifying the target object in the image based on the enhanced features of the target object.
[0026] The method for identifying target objects in an image according to the present invention can enhance the representation features of the target and identify the target based on the enhanced representation features. It effectively solves the problem of performance degradation of target detection tasks caused by weak representation features of the target, makes it easier to distinguish target categories, and improves the target detection performance of machine models.
[0027] The method, apparatus, and device for identifying target objects in an image according to the present invention are described in detail below with reference to specific embodiments.
[0028] First, refer to Figure 1 An exemplary flowchart illustrating the steps of a method 100 for identifying a target object in an image according to an embodiment of the present invention is provided. Figure 1 As shown, the method 100 for enhancing the features of a target object in an image may include the following steps:
[0029] In step S110, an image to be detected is acquired, which contains the target object to be identified.
[0030] In step S120, the image is input into a trained feature enhancement model to enhance the features of the target object based on stored global category prototype features associated with the target object, wherein the global category prototype features are obtained and stored during the training of the feature enhancement model.
[0031] In step S130, the target object in the image is identified based on the enhanced features of the target object.
[0032] In one embodiment, the method 100 for identifying target objects in an image can be used in various scenarios, such as security checks in public places, organ identification in medicine, and road target detection in autonomous driving, etc., and the present invention does not limit it.
[0033] In one embodiment, the image to be detected can be an X-ray image, or an image obtained in any of the above scenarios; the present invention does not limit this.
[0034] In one embodiment, the image to be detected may include a target object to be identified and non-target objects. For example, in a security check scenario, the target object may include various prohibited items, such as alcohol, controlled knives, firearms, explosives, etc.; the non-target object may include any other items besides the target object to be identified, such as cosmetics, water cups, computers, mobile phones, etc., and the present invention does not limit this.
[0035] refer to Figure 2 , Figure 2 A schematic structural block diagram of a feature enhancement model 200 according to an embodiment of the present invention is shown. Figure 2 As shown, the feature enhancement model 200 may include an object detection network 210, a prototype aggregation network 220, and a feature fusion network 230.
[0036] In one embodiment, step S120, which involves enhancing the features of the target object based on the stored global category prototype features associated with the target object, may include the following steps:
[0037] Step S1220: Extract features from the image to be detected to obtain the image feature map;
[0038] Step S1240: Process the image feature map to obtain the candidate box features corresponding to the candidate boxes surrounding the target object to be identified;
[0039] Step S1260: Fuse the candidate box features and the global category prototype features corresponding to the candidate boxes to enhance the features of the target object.
[0040] In one embodiment, the feature extraction in step S1220 can be implemented using a feature extraction layer (e.g., a convolutional layer) in the object detection network 210.
[0041] In one embodiment, processing the image feature map in step S1240 may include the following steps:
[0042] Step S1242: Perform target detection on the image feature map to obtain candidate boxes surrounding the target object;
[0043] Step S1244: Map the candidate boxes onto the image feature map to obtain the candidate box features.
[0044] In one embodiment, the object detection of the image feature map in step S1242 can be implemented using the object detection layer in the object detection network 210. In one embodiment, the object detection layer can employ any object detection algorithm known in the art to implement object detection, such as R-CNN, Fast R-CNN, AttentionNet, reinforcement learning, CornerNet, CenterNet, FCOS, etc., and the present invention does not limit it.
[0045] In one embodiment, the resulting candidate box can be represented in any manner known in the art, such as using the center point coordinates (x, y) and four parameters: width w and height h. This invention does not limit this representation.
[0046] In one embodiment, when performing object detection on the image feature map, multiple initial candidate boxes surrounding the target object can be obtained. These multiple initial candidate boxes can then be filtered to remove redundant initial candidate boxes, resulting in the initial candidate box with the highest overlap rate with the target object. In one embodiment, any algorithm known in the art can be used to filter the initial candidate boxes, such as non-maximum suppression, mean shift, etc., and this invention does not limit this.
[0047] In step S1244, the candidate box is mapped onto the image feature map, which means determining the region corresponding to the candidate box on the image feature map, and then extracting the image features of that region as the candidate box features. This step can be implemented using the mapping layer in the prototype aggregation network 220.
[0048] In one embodiment, when fusing the candidate box feature and the global category prototype feature corresponding to the candidate box in step S1260, it is first necessary to determine which stored global category prototype feature the candidate box feature is to be fused with.
[0049] In one embodiment, the global category prototype feature corresponding to the candidate box can be determined by calculating the similarity (e.g., cosine similarity, etc.) between the candidate box feature and each global category prototype feature.
[0050] In one embodiment, the fusion of candidate box features and global category prototype features corresponding to the candidate boxes in step S1260 can be implemented by feature fusion network 230.
[0051] Because the degree of information loss varies across different categories in different scenarios, the demand for reliable features contained in the category prototype features also differs among categories. Therefore, a convolutional network with an activation layer can be designed for each category to adaptively adjust the demand for reliable features in the category prototype features. Thus, in one embodiment, the feature fusion operation between all candidate box features and the global category prototype features that need to be fused with each candidate box feature can be calculated using the following formula:
[0052] f e =ReLU(Conv (k) (Ω (k) ))+f R
[0053] Where f R Ω represents the extracted candidate bounding box features, i.e., the features to be enhanced. (k) Conv represents the global category prototype feature participating in the fusion. (k) Let f represent a convolutional network for class k, ReLU represent the activation layer, and f e This represents the enhanced features obtained after feature fusion.
[0054] In one embodiment, global category prototype features are obtained and stored during the training of feature enhancement model 200.
[0055] In one embodiment, the trained feature enhancement model 200 can be obtained by training through the following steps:
[0056] Obtain a training image set containing the target object, and label each object in the training image set;
[0057] Each training image in the training image set is processed by the object detection network 210 in the feature enhancement model to obtain the image feature map of the training image and the candidate box surrounding the target object in the training image;
[0058] The image feature map and candidate boxes are processed by the prototype aggregation network 220 in the feature enhancement model to obtain candidate box features corresponding to the candidate boxes and global category prototype features for the category to which the target object belongs.
[0059] The feature fusion network 230 in the feature enhancement model fuses the candidate box features and the global category prototype features corresponding to the candidate box to enhance the features of the target object.
[0060] The loss is calculated based on the global category prototype features, and the parameters of the feature enhancement model are updated based on this loss.
[0061] In one embodiment, when applying the feature enhancement model 200 to different scenarios, images obtained in the corresponding scenarios can be used as training images. For example, in a security check scenario, images of luggage requiring security checks can be obtained as training images. In this invention, luggage is a general term for luggage, parcels, bags, boxes, backpacks, handbags, etc. The training images include target objects to be identified and non-target objects. For example, in a security check scenario, target objects may include various prohibited items, such as alcohol, controlled knives, firearms, explosives, etc.; non-target objects may include any other items besides the target object to be identified, such as cosmetics, water cups, computers, mobile phones, etc. This invention does not limit this. As needed, the training image set may include any number of training images (e.g., 100 images), and these training images may be divided into several training batches (e.g., 10 training batches, 10 images per batch), with each training iteration using one batch of training images for training. This invention does not limit this.
[0062] In one embodiment, when labeling each object in the training image set, target objects and non-target objects can be labeled with different labels, and different categories of target objects can also be labeled with different labels. For example, the label of non-target objects is 0, and the labels of different categories of target objects can be 1, 2, 3, 4, etc. The present invention does not limit this.
[0063] In one embodiment, when the object detection network 210 processes each training image in the training image set, it first extracts features from the training image to obtain the image feature map of the training image, and then performs object detection and filtering on the image feature map to obtain candidate boxes surrounding the target object.
[0064] In one embodiment, for each training image, image feature maps at multiple scales can be extracted to improve the accuracy of model training.
[0065] In one embodiment, object detection of the image feature map can be performed using any object detection algorithm known in the art, such as R-CNN, Fast R-CNN, AttentionNet, reinforcement learning, CornerNet, CenterNet, FCOS, etc., and the present invention does not limit it.
[0066] In one embodiment, the resulting candidate box can be represented in any manner known in the art, such as using the center point coordinates (x, y) and four parameters: width w and height h. This invention does not limit this representation.
[0067] In one embodiment, when the prototype aggregation network processes the image feature map and candidate boxes, it first maps the candidate boxes onto the image feature map, i.e., determines the region corresponding to the candidate box on the image feature map, and then extracts the image features of that region to obtain the candidate box features. After obtaining the candidate box features, the candidate box features of all target objects can be divided according to different target objects, i.e., the candidate box features of each target object are divided into a set, thus obtaining the target candidate box feature set for that target object. Where i is the i-th target object. Then, based on the feature set of the target candidate boxes, the global category prototype feature for the category to which the target object belongs is calculated.
[0068] In one embodiment, all candidate bounding box features are divided into candidate bounding box feature sets V for different target objects. (i) At this time, it is necessary to filter out low-quality candidate boxes, that is, to filter out candidate boxes whose Intersection over Union (IoU) value between the candidate box and the ground truth bounding box of the target object is less than a threshold, so that the feature set of candidate boxes for each target object is optimized. Each candidate box feature Truth annotation box corresponding to target object i The intersection-union ratio (IoU) between them must be greater than the threshold.
[0069] In one embodiment, when calculating the global category prototype feature for the category to which the target object belongs based on the target candidate box feature set, in order to aggregate all candidate box features corresponding to the same target object to generate the most accurate feature map for that target object, a target prototype feature can first be established for each target object based on the target candidate box feature set to represent that target object. For example, the target prototype feature Ω for each target object... (i) It can be established using the following formula:
[0070]
[0071] in, V represents the feature set of candidate boxes (i) The feature of the nth candidate box. The truth box representing the target object i, N k IoU(·) represents the number of candidate box features belonging to target object i, IoU(·) represents the intersection-union ratio between the two, and Ω represents the intersection-union ratio between them. (i) The target prototype feature represents the target object i.
[0072] After obtaining the target prototype feature Ω (i) Then, based on the target prototype feature Ω (i) Calculate the category probability vector P of the target object. (i) , where i is the i-th target object. The category probability vector P(i) Let P be a one-dimensional vector of length equal to the number of all categories. (i,k) Represents the target prototype feature Ω (i) The probability of belonging to the k-th class. For the training phase, the class probability vector P... (i) It is a vector containing only 0 and 1, that is, the probability of the target object corresponding to the ground truth class label is 1, and the probability of the other classes is 0.
[0073] After obtaining the category probability vector P (i) Then, based on the target prototype feature Ω (i) and category probability vector P (i) Calculate the category prototype feature Ω for the category to which the target object belongs. (k) Ω, where k is the k-th category. The category prototype feature Ω of the k-th category. (k) It can be calculated using the following formula:
[0074]
[0075] Among them, P (i,k) Represents the target prototype feature Ω (i) The probability of belonging to the k-th category, N (i) Ω represents the number of target objects belonging to the k-th category. (k) This represents the category prototype feature of the k-th category.
[0076] After obtaining the category prototype feature Ω (k) Then, based on the category prototype feature Ω (k) Establish a global class prototype feature L, and update this global class prototype feature L in each training iteration. The update of the global class prototype feature L can be achieved using the following formula:
[0077]
[0078] Where α represents the cosine similarity between the two variables, i.e. This represents the prototype of the class feature of the k-th class during the l-th training iteration. This represents the global category prototype feature of the k-th category after the (l-1)-th training iteration. After training, the updated final global category prototype feature is stored so that, during the testing phase, the features of the target object to be identified in the image to be detected can be enhanced based on this global category prototype feature.
[0079] To further enhance the extractable and reliable features between different categories, thereby better distinguishing objects of different categories, the obtained category prototype features can be mapped into a linear feature vector, and a loss function can be designed on these feature vectors to calculate the prototype discrimination loss. In one embodiment, any type of loss function known in the art can be set as needed, such as absolute value loss function, logarithmic loss function, exponential loss function, cross-entropy loss function, perceptual loss function, etc., and the present invention does not limit this.
[0080] In one embodiment, the loss function can be set as the average of the cosine similarities between each two categories, calculated according to the following formula when calculating the prototype discrimination loss:
[0081]
[0082] in The prototype differentiation loss is represented by cos(·), and the cosine similarity is represented by Ω. (i) N represents the global category prototype feature of the i-th category. k This indicates the quantity of all categories.
[0083] During the training phase, when fusing candidate box features and global category prototype features corresponding to the candidate boxes, the global category prototype features corresponding to the candidate boxes can be determined based on the category label of the target object corresponding to the candidate box features.
[0084] Because the degree of information loss varies across different categories in different scenarios, the demand for reliable features contained in the category prototype features also differs among categories. Therefore, a convolutional network with an activation layer can be designed for each category to adaptively adjust the demand for reliable features in the category prototype features. When performing feature fusion on all candidate box features and the global category prototype features that need to be fused with each candidate box feature, the following formula can be used for calculation:
[0085] f e =ReLU(Conv (k) (Ω (k) ))+f R
[0086] Where f R Ω represents the extracted candidate bounding box features, i.e., the features to be enhanced. (k) Conv represents the global category prototype feature participating in the fusion. (k) Let f represent a convolutional network for class k, ReLU represent the activation layer, and f e This represents the enhanced features obtained after feature fusion.
[0087] The method for identifying target objects in an image according to the present invention can enhance the representation features of the target and identify the target based on the enhanced representation features. It effectively solves the problem of performance degradation of target detection tasks caused by weak target representation features, makes it easier to distinguish target categories, and improves the target detection performance of machine models.
[0088] The present invention also provides an apparatus for identifying target objects in an image. (Reference) Figure 3 , Figure 3 A schematic structural block diagram of a device 300 for identifying a target object in an image according to an embodiment of the present invention is shown. Figure 3 As shown, the apparatus 300 for identifying target objects in an image may include a memory 310 and a processor 320. The memory 310 stores a computer program executed by the processor 320. When the computer program is executed by the processor 320, the processor 320 performs the method 100 for identifying target objects in an image according to the embodiments of the present invention described above. Those skilled in the art can understand the specific operation of the apparatus 300 for identifying target objects in an image according to the embodiments of the present invention in conjunction with the foregoing description; for the sake of brevity, it will not be described in detail here.
[0089] The present invention also provides a device. (See reference) Figure 4 , Figure 4 A schematic structural block diagram of a device 400 according to an embodiment of the present invention is shown. Figure 4 As shown, the device 400 may include an image acquisition device 410 and a device 300 for identifying a target object in an image as described above, wherein the image acquisition device 410 is used to acquire an image of an item to be detected to obtain an image to be detected, and the device 300 for identifying a target object in an image is used to process the image to be detected to identify the target object in the image based on the enhanced features of the target object.
[0090] In one embodiment, the image acquisition device 410 can be any image acquisition device known in the art, such as an X-ray machine, camera, video camera, scanner, etc.
[0091] The apparatus and device for identifying target objects in an image according to the present invention can enhance the representation features of the target and identify the target based on the enhanced representation features. This effectively solves the problem of performance degradation in target detection tasks caused by weak representation features of the target, makes it easier to distinguish target categories, and improves the target detection performance of machine models.
[0092] The present invention also provides a computer-readable medium storing computer-executable instructions that, when executed, perform corresponding steps of the method 100 for identifying a target object in an image as described above. Any tangible, non-transitory computer-readable medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer-executable instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that the instructions, which execute on the computer or other programmable data processing apparatus, can generate means for implementing a specified function. These computer-executable instructions may also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable storage medium can form an article of manufacture including means for implementing the specified function. The computer-executable instructions may also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, can provide steps for implementing the specified function.
[0093] Furthermore, according to embodiments of the present invention, a computer program is also provided, which, when run by a computer or processor, is used to perform corresponding steps of the method 100 for identifying target objects in an image according to embodiments of the present invention.
[0094] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.
[0095] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0096] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.
[0097] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.
[0098] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.
[0099] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0100] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for identifying target objects in an image, characterized in that, The method includes: Acquire an image to be detected, the image containing the target object to be identified; The image is input into a trained feature enhancement model to enhance the features of the target object based on stored global category prototype features associated with the target object, wherein the global category prototype features are obtained and stored during training of the feature enhancement model; and Identifying a target object in an image based on enhanced target object features, wherein inputting the image into a trained feature enhancement model to enhance the features of the target object based on stored global category prototype features associated with the target object includes: Feature extraction is performed on the image to obtain the image feature map; Target detection is performed on the image feature map to obtain candidate bounding boxes surrounding the target object; and Mapping the candidate bounding boxes onto the image feature map yields the candidate bounding box features; and The candidate box features and the global category prototype features corresponding to the candidate boxes are fused to enhance the features of the target object.
2. The method as described in claim 1, characterized in that, The method of fusing the candidate box features and the global category prototype features corresponding to the candidate boxes to enhance the features of the target object includes: Calculate the similarity between the candidate box features and each global category prototype feature to determine the global category prototype feature corresponding to the candidate box.
3. The method as described in claim 1, characterized in that, The image to be detected is an X-ray image, and the target object is a contraband.
4. The method as described in claim 1, characterized in that, The trained feature enhancement model includes an object detection network, a prototype aggregation network, and a feature fusion network, which are trained through the following steps: Obtain a training image set containing the target object, and label each object in the training image set; Feature extraction is performed on each training image in the training image set through the target detection network in the feature enhancement model to obtain the image feature map of the training image. Target detection and filtering are then performed on the image feature map to obtain candidate boxes surrounding the target objects in the training image. The image feature map and the candidate boxes are processed by the prototype aggregation network in the feature enhancement model to obtain candidate box features corresponding to the candidate boxes and global category prototype features for the category to which the target object belongs. The global category prototype features are stored and used to enhance the features of the target object to be identified in the image to be detected based on the global category prototype features during the testing phase. The feature enhancement model fuses the candidate box features and the global category prototype features corresponding to the candidate boxes through a feature fusion network to enhance the features of the target object; and The loss is calculated based on the global category prototype features, and the parameters of the feature enhancement model are updated based on the loss.
5. The method as described in claim 4, characterized in that, The process of processing the image feature map and the candidate bounding boxes via the prototype aggregation network in the feature enhancement model to obtain candidate bounding box features corresponding to the candidate bounding boxes and global category prototype features for the category to which the target object belongs includes: The candidate bounding boxes are mapped onto the image feature map to obtain the candidate bounding box features; The candidate box features are divided according to different target objects to obtain a target candidate box feature set; and Based on the target candidate box feature set, establish target prototype features for each target object; Based on the target prototype features, the category probability vector of the target object's category is obtained; Based on the target prototype features and the category probability vector, calculate the category prototype features for the category to which the target object belongs; and A global category prototype feature is established based on the category prototype feature, and the global category prototype feature is updated in each training iteration.
6. The method as described in claim 4, characterized in that, The feature enhancement model fuses the candidate box features and the global category prototype features corresponding to the candidate box through a feature fusion network to enhance the features of the target object, including: The global category prototype feature corresponding to the candidate box is determined based on the category label of the target object corresponding to the candidate box feature.
7. An apparatus for identifying a target object in an image, characterized in that, The device includes a memory and a processor, the memory storing a computer program executed by the processor, the computer program, when executed by the processor, causing the processor to perform the method for identifying a target object in an image as described in any one of claims 1-6.
8. A device, characterized in that, The device includes an image acquisition device and the device for identifying a target object in an image as described in claim 7, wherein the image acquisition device is used to acquire an image of an item to be detected to obtain an image to be detected, and the device for identifying a target object in an image is used to process the image to be detected to identify the target object in the image based on the enhanced features of the target object.
9. A computer-readable medium, characterized in that, The computer-readable medium stores computer-executable instructions that, when executed, perform the method for identifying a target object in an image as described in any one of claims 1-6.