Target detection method and device based on registered instance segmentation, equipment and medium

Abstract

The application discloses a target detection method and device based on registered instance segmentation, equipment and medium. The target detection method based on registered instance segmentation comprises the following steps: obtaining a registered image and a to-be-detected image, wherein the registered image comprises a pre-labeled registered object; determining a target region from the to-be-detected image according to cross features between the registered object and the registered image, wherein a feature similarity between a region feature of the target region and the cross features is greater than a preset threshold; and performing image segmentation processing on the to-be-detected image according to the target region to obtain a target detection result. The above scheme can improve the efficiency of target detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method, apparatus, device, and medium based on registered instance segmentation. Background Technology

[0002] Object detection has always been one of the key technologies in the field of computer vision. In object detection methods, traditional template matching algorithms are typically used when detecting regularly shaped and fixed objects.

[0003] However, traditional methods not only rely on templates to comprehensively cover the target object type, but also severely test the algorithm's robustness to slight deformations and feature transformations of the target object. Especially given the wide variety of target objects and subtle changes in their features, traditional template matching methods struggle to reliably and accurately detect them. Therefore, object detection often employs deep learning-based algorithms, learning from a large number of image features of target objects to regress their location information. However, this supervised learning requires significant training time and resources each time.

[0004] Currently, there is an urgent need for an accurate and efficient target detection method that can not only detect various types of target objects, but also reduce the resources required for training. Summary of the Invention

[0005] This application provides at least one object detection method, apparatus, device, and computer-readable storage medium based on registered instance segmentation.

[0006] The first aspect of this application provides a target detection method based on registered instance segmentation, comprising: acquiring a registered image and a target image, wherein the registered image includes a pre-annotated registered object; determining a target region from the target image based on the cross features between the registered object and the registered image, wherein the feature similarity between the region features of the target region and the cross features is greater than a preset threshold; and performing image segmentation processing on the target image based on the target region to obtain a target detection result.

[0007] In one embodiment, determining the target region from the image to be inspected based on the cross features between the registered object and the registered image includes: acquiring the features to be inspected from the image to be inspected; determining a similarity distribution map based on the feature similarity between the cross features and the features to be inspected; and determining the target region of the image to be inspected based on the similarity distribution map and the preset threshold.

[0008] In one embodiment, determining the target region of the image to be inspected based on the similarity distribution map and the preset threshold includes: comparing the similarity corresponding to each position in the similarity distribution map with the preset threshold to obtain a high similarity region; and determining the target region based on the region center of the high similarity region and the size information of the registered object.

[0009] In one embodiment, before determining the target region from the image to be inspected based on the cross features between the registered object and the registered image, the method further includes: acquiring the registration features of the registered image and the object features of the registered object; and performing cross-fusion processing on the registration features and the object features according to a preset attention mechanism to obtain the cross features.

[0010] In one embodiment, the registration features include shallow registration features and deep registration features, the object features include shallow object features and deep object features, and the cross features include shallow cross features and deep cross features. The step of cross-fusing the registration features and the object features according to a preset attention mechanism to obtain the cross features includes: cross-fusing the shallow registration features and the shallow object features according to the attention mechanism to obtain the shallow cross features; and cross-fusing the deep registration features and the deep object features according to the attention mechanism to obtain the deep cross features.

[0011] In one embodiment, determining the target region from the image to be inspected based on the cross features between the registered object and the registered image includes: acquiring the features to be inspected of the image to be inspected, the features to be inspected including shallow features and deep features to be inspected; fusing the shallow cross features and the shallow features to be inspected with four-dimensional convolution, and fusing the deep cross features and the deep features to be inspected with four-dimensional convolution to obtain shallow fused features and deep fused features; fusing the shallow fused features and the deep fused features with upsampling to obtain a similarity distribution map; and determining the target region in the image to be inspected based on the similarity distribution map.

[0012] In one embodiment, before determining the target region from the image to be inspected based on the cross features between the registered object and the registered image, the method further includes: obtaining object size information of each registered object in the registered image; determining target size information based on the object size information of each registered object; and adjusting the input parameter size of a preset feature extraction network based on the target size information, the image size information of the registered image, and a preset network input standard.

[0013] A second aspect of this application provides a target detection apparatus based on registered instance segmentation, comprising: an acquisition module for acquiring a registered image and a target image, wherein the registered image includes pre-annotated registered objects; a region determination module for determining a target region from the target image based on the cross features between the registered objects and the registered image, wherein the feature similarity between the region features of the target region and the cross features is greater than a preset threshold; and an image segmentation module for performing image segmentation processing on the target image based on the target region to obtain a target detection result.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the above-described target detection method based on registered instance segmentation.

[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described target detection method based on registered instance segmentation.

[0016] The above scheme acquires a registration image containing pre-annotated registered objects and an image to be detected. After extracting features from the registered objects, the registration image, and the image to be detected, a pre-trained similarity matching model is used to determine highly similar target regions from the image to be detected based on the cross features between the registered objects and the registration image. Then, a segmentation model is used to segment the image to be detected based on the target regions to obtain the target detection result, thus achieving the task of instance segmentation and detection of targets. Compared to traditional target detection methods where the detection effect deteriorates when the features of the target object change or requires a large amount of model training resources, the method of this application can annotate the registration image to provide example features and achieve target detection through adaptive similarity matching and instance segmentation without retraining the model, thereby improving the efficiency and accuracy of target detection.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a flowchart illustrating an exemplary embodiment of the target detection method based on registered instance segmentation of this application;

[0020] Figure 2This is an exemplary registered image diagram in the target detection method based on registered instance segmentation of this application;

[0021] Figure 3 This is an exemplary feature cross-fusion processing diagram in the target detection method based on registered instance segmentation of this application;

[0022] Figure 4 This is another exemplary feature cross-fusion processing diagram in the target detection method based on registered instance segmentation of this application;

[0023] Figure 5 This is an exemplary feature similarity matching diagram in the target detection method based on registered instance segmentation of this application;

[0024] Figure 6 This is an exemplary overall flowchart of the object detection method based on registered instance segmentation in this application;

[0025] Figure 7 This is a block diagram illustrating a target detection device based on registered instance segmentation, as shown in an exemplary embodiment of this application.

[0026] Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0027] Figure 9 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0028] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0030] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0031] To facilitate understanding, one of the applicable scenarios of this application is illustrated below. In existing object detection technologies, different application scenarios may be implemented based on different detection principles. For example, in industrial scenarios, when detecting relatively regular, template-based target objects, traditional template matching algorithms are typically used. This method relies on the template's comprehensive coverage of target object types and heavily tests the algorithm's robustness to object deformation and feature transformation. If target features change, traditional template matching methods struggle to reliably and accurately detect the target. Therefore, on the other hand, there are deep learning-based object detection algorithms that train neural network models to learn a large number of relevant features of target objects, and then use the trained neural network model for object detection. However, supervised training of neural network models requires significant training time and resources each time, and the model needs to be retrained if more types of target objects are to be detected.

[0032] Therefore, the method of this application can pre-annotate the registered objects (target objects) to be detected in the registered image, adopt an unsupervised target detection method, learn the object features of the registered objects, and then calculate the similarity between the image to be detected and the object features to locate the target position in the image to be detected. Then, the detected target objects in the image to be detected can be segmented through image segmentation processing to obtain the target detection result.

[0033] Please see Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the target detection method based on registered instance segmentation according to this application. Specifically, it may include the following steps:

[0034] Step S110: Obtain the registration image and the image to be inspected. The registration image includes pre-annotated registration objects.

[0035] In this context, a registered image refers to an image used as a reference in the object detection process, providing example features (i.e., object features of registered objects in the registered image) in the subsequent similarity matching stage. Registered objects are pre-annotated on the registered image during the registration stage. The goal of the object detection task is to detect target objects similar to registered objects from the image to be detected. Registered images and images to be detected can be the same or different (usually different). This application may have one or more images to be detected, one or more registered images, and each registered image may contain one or more classes of registered objects, with one or more objects per class, which is not limited here.

[0036] For example, reference can be made to, such as Figure 2 As shown, Figure 2This is an exemplary registration image diagram from the object detection method based on registration instance segmentation of this application. The registered objects in the registration image can be labeled using methods such as bounding boxes (or other feasible methods).

[0037] Step S120: Based on the cross features between the registered object and the registered image, determine the target region from the image to be inspected. The feature similarity between the region features and the cross features of the target region is greater than a preset threshold.

[0038] Here, cross-features refer to features obtained by performing feature cross-processing on the features of the registered object and the features of the registered image. The feature cross-processing in this application can fuse two or more features in a certain way to generate new features.

[0039] This application employs feature cross-processing between the features of the registered object and the features of the registered image to capture potential interactions between features, thereby enhancing the expressive power of the features and improving object detection capabilities. Various specific feature cross-processing methods exist, which will not be elaborated upon here.

[0040] For example, after obtaining the cross features between the registered object and the registered image, the target region can be determined from the image to be inspected based on the feature similarity between the cross features and the features to be inspected in the image to be inspected. The target region can be a region in the image to be inspected where the feature similarity is greater than or equal to a preset threshold (feature similarity threshold). This means that the feature similarity between the region features and the cross features of the target region is greater than the preset threshold; in other words, the target region is a potential region in the image to be inspected where the target object to be detected may be located.

[0041] Step S130: Perform image segmentation processing on the image to be inspected based on the target region to obtain the target detection result.

[0042] Based on the steps described above, after obtaining the target region (which is equivalent to obtaining the location information of the target object in the image to be inspected), image segmentation processing can be performed on the image to be inspected based on the target region to obtain the segmentation result, which is also the target detection result.

[0043] As can be seen, this application obtains a registration image including pre-annotated registration objects and an image to be detected. After extracting features from the registration objects, registration images, and images to be detected, a pre-trained similarity matching model can be used to determine target regions with high similarity from the images to be detected based on the cross features between the registration objects and registration images. Then, a segmentation model can be used to segment the images to be detected based on the target regions to obtain the target detection results, thus achieving the task of instance segmentation and detection of targets. Compared with traditional target detection methods, where the detection effect deteriorates when the features of the target object change or requires a large amount of model training resources, the method of this application can annotate the registration image to provide example features and achieve target detection through adaptive similarity matching and instance segmentation without retraining the model, thereby improving the efficiency and accuracy of target detection.

[0044] Based on the above embodiments, this application embodiment describes the steps of determining a target region from an image to be inspected based on the cross features between the registered object and the registered image. Specifically, the method of this embodiment includes the following steps:

[0045] Obtain the features to be inspected from the image to be inspected; determine the similarity distribution map based on the feature similarity between the cross features and the features to be inspected; determine the target region of the image to be inspected based on the similarity distribution map and a preset threshold.

[0046] Referring to the foregoing embodiments, obtaining the cross-features between the registered object and the registered image is equivalent to enhancing the feature representation capability of the registered object. Therefore, the feature similarity between the cross-features and the features to be detected can be analyzed to obtain a similarity distribution map.

[0047] The similarity distribution map represents the distribution of feature similarity. Each position in the similarity distribution map has a corresponding feature similarity, which is equivalent to representing the similarity between each position in the image to be inspected and the registered object. Therefore, the target region (e.g., the region with high similarity) in the image to be inspected can be determined by comparing the feature similarity at each position in the similarity distribution map with a preset threshold.

[0048] For example, the methods for obtaining the detection features of the image to be detected and the registration features of the registered image can be the same. For instance, the basic feature maps of the two types of images can be extracted separately using a ResNet50 backbone, thus obtaining the detection features and registration features respectively.

[0049] One method to obtain the object features of a registered object is to determine the corresponding object features within the registered object's location information (such as image coordinates) after extracting the registration features. For example, the location of the object features can be determined using the ROI Align algorithm based on the registered object's location information and the registration features. Alternatively, one can pre-extract the image region corresponding to the registered object from the registered image based on the registered object's location information, thus obtaining the registered object image. Then, feature extraction, including but not limited to the examples above, can be performed on the registered object image to obtain the object features.

[0050] Furthermore, after obtaining the cross features and the features to be detected, the feature similarity between the cross features and the features to be detected can be calculated. For example, the cross features and the features to be detected can be input into a similarity matching model. The similarity matching model will perform processing on the cross features and the features to be detected, including but not limited to fusion, convolution, upsampling, etc., to obtain a similarity distribution map of the cross features and the features to be detected corresponding to the image to be detected. Then, the target region in the image to be detected can be determined based on the similarity distribution map and a preset threshold.

[0051] Based on the above embodiments, this application embodiment describes the steps for determining the target region of an image to be inspected based on a similarity distribution map and a preset threshold. Specifically, the method of this embodiment includes the following steps:

[0052] The similarity at each location in the similarity distribution map is compared with a preset threshold to obtain high similarity regions; the target region is determined based on the region center of the high similarity region and the size information of the registered object.

[0053] As illustrated in the foregoing embodiments, after obtaining the similarity distribution map, the target region where the target object is located can be determined based on the similarity distribution map.

[0054] One approach is to compare the similarity at each location in the similarity distribution map with a preset threshold, identifying locations with similarities greater than the threshold and thus determining high-similarity regions. Alternatively, a sliding window algorithm can be used to unify the similarity within a certain region. For example, the preset sliding window size could be 3x3 pixels or other sizes (i.e., a total of 9 pixels in each sliding window), and the window is slid across the similarity distribution map. For each of the 9 pixels in the sliding window, the similarity at the corresponding positions can be averaged, weighted summed, replaced by the maximum value (selecting the maximum similarity among the 9 pixels as the corresponding similarity), or replaced by the minimum value (selecting the minimum similarity among the 9 pixels as the corresponding similarity), thus clustering the 9 pixels in the same window into the same region and assigning them the same similarity. Therefore, in the subsequent determination of the target region, comparing the similarity at the region level with the preset threshold significantly improves the efficiency of target region determination compared to comparing the similarity at the pixel level.

[0055] For pixels that are adjacent in position and have the same similarity, or for regions that are adjacent in position and have the same similarity, they can be merged to obtain merged regions, and then the similarity of the merged regions is compared with a preset threshold.

[0056] Alternatively, after identifying a high-similarity region, it can be directly designated as the target region, or the target region can be determined based on the region center of the high-similarity region and the size information of the registered object.

[0057] For example, a high similarity region represents a region with a high degree of similarity. However, if there are other similar objects in the two images, the high similarity region may not accurately represent the region where the target object is located. Therefore, in order to improve the accuracy of target detection, after determining the high similarity region, this application can also choose to determine the region center (e.g., geometric center) of the high similarity region, and then determine the target region based on the size information of the registered object marked in the registered image.

[0058] Especially in the aforementioned example, to accelerate processing efficiency, the similarity within a certain region is unified. After obtaining the high-similarity region, it is suitable to determine the target region based on the size information of the registered object and the region center to compensate for the potential decrease in accuracy due to increased efficiency. If the high-similarity region is determined based on pixel-level similarity, then the high-similarity region can be directly determined as the target region.

[0059] Of course, this application is not limited to determining the target region based on the size information of the registered object and the region center after determining the high similarity region based on pixel-level similarity; or determining the high similarity region determined based on region-level similarity as the target region, etc. There are no restrictions here, and it can be set as needed according to the actual application scenario.

[0060] Based on the above embodiments, this application embodiment describes the steps prior to determining the target region from the image to be inspected based on the cross features between the registered object and the registered image. Specifically, the method of this embodiment includes the following steps:

[0061] Obtain the registration features of the registered image and the object features of the registered object; perform cross-fusion processing on the registration features and object features according to the preset attention mechanism to obtain cross features.

[0062] In conjunction with the foregoing embodiments, the method for obtaining the cross features between the registered object and the registered image in this application may be obtained by cross-fusion processing of the registration features of the registered image and the object features of the registered object.

[0063] For example, the feature cross-fusion process can be implemented based on a pre-set attention mechanism. For instance, see... Figure 3 As shown, Figure 3 This is an exemplary feature cross-fusion processing diagram in the object detection method based on registered instance segmentation of this application. After obtaining the cross features, the cross features and the detection features of the image to be detected can be learned on the enhanced features through four-dimensional convolution (Conv4D). Finally, the similarity distribution map corresponding to the image to be detected is obtained through the decoding structure composed of convolution and upsampling.

[0064] Based on the above embodiments, this application embodiment describes the steps of cross-fusion processing of registration features and object features according to a preset attention mechanism to obtain cross features. The registration features include shallow registration features and deep registration features, the object features include shallow object features and deep object features, and the cross features include shallow cross features and deep cross features. Specifically, the method of this embodiment includes the following steps:

[0065] Based on the attention mechanism, shallow registration features and shallow object features are cross-fused to obtain shallow cross features; based on the attention mechanism, deep registration features and deep object features are cross-fused to obtain deep cross features.

[0066] Referring to the foregoing embodiments, in order to comprehensively analyze each image and improve the accuracy of object detection, this application extracts at least two different dimensional features for each image during the feature extraction process (equivalent to outputting feature maps at two different network depths), resulting in shallow and deep feature maps. For example, the two different dimensional feature sizes can be [N, 1024, input_H / 16, input_W / 16] and [N, 512, input_H / 8, input_W / 8]. Here, N refers to the number of images, 1024 and 512 refer to the depth, and input_H and input_W are the network input sizes.

[0067] For example, the registered image can have its basic features extracted using a ResNet50-based backbone, and two sets of basic feature maps—shallow and deep—can be output at two different network depths, thus obtaining shallow registration features and deep registration features. Similarly, the image to be inspected can have its basic features extracted using a ResNet50-based backbone, and two sets of basic feature maps—shallow and deep—can be output at two different network depths, thus obtaining shallow and deep inspection features. Furthermore, based on the location information of the registered objects in the registered image, algorithms such as ROI Align can be used to determine the shallow and deep object features of the registered objects in the shallow and deep registration features, respectively. For example, the feature sizes can be [targetNum, 1024, input_H / 64, input_W / 64] and [N, 512, input_H / 64, input_W / 64]. `targetNum` refers to the number of registered objects.

[0068] For reference, please see below. Figure 4 As shown, Figure 4 This is another exemplary feature cross-fusion processing diagram in the object detection method based on registered instance segmentation of this application. During the registration phase, to learn the cross-features between the registered image and the registered object, an attention-based cross-feature learning module can be used. For both shallow and deep features, shallow registration features and shallow object features are cross-fused according to the attention mechanism to obtain shallow cross-features; similarly, deep registration features and deep object features are cross-fused according to the attention mechanism to obtain deep cross-features.

[0069] Based on the above embodiments, this application embodiment describes the steps of determining a target region from an image to be inspected based on the cross features between the registered object and the registered image. Specifically, the method of this embodiment includes the following steps:

[0070] The process involves acquiring the features to be detected in the image, including shallow and deep features; fusing the shallow cross features and shallow features to be detected with four-dimensional convolution, and fusing the deep cross features and deep features to be detected with four-dimensional convolution to obtain shallow and deep fused features; fusing and upsampling the shallow and deep fused features to obtain a similarity distribution map; and determining the target region in the image to be detected based on the similarity distribution map.

[0071] In conjunction with the foregoing embodiments, features at different depths can also be extracted from the image to be inspected to obtain shallow and deep inspection features.

[0072] After obtaining shallow and deep cross features, during the similarity matching learning process, cross features of the same depth and the features to be detected are input into the similarity matching module. Feature learning on the features is enhanced by 4D convolution. Finally, the similarity distribution map of the image to be detected is obtained through the decoding structure composed of convolution and upsampling.

[0073] For example, reference can be made to, such as Figure 5 As shown, Figure 5 This is an exemplary feature similarity matching diagram in the target detection method based on registered instance segmentation of this application. Shallow cross features and shallow detectable features are input into the similarity matching module, where they undergo fusion processing and four-dimensional convolution (Conv4D) to obtain shallow fused features. Similarly, deep cross features and deep detectable features are fused and subjected to four-dimensional convolution to obtain deep fused features. Then, the shallow and deep fused features are fused, convolved, and upsampled to obtain a similarity distribution map. Therefore, the target region in the image to be detected can be determined based on the similarity at each position in the similarity distribution map and a preset threshold.

[0074] Based on the above embodiments, this application embodiment describes the steps prior to determining the target region from the image to be inspected based on the cross features between the registered object and the registered image. Specifically, the method of this embodiment includes the following steps:

[0075] Obtain the object size information of each registered object in the registered image; determine the target size information based on the object size information of each registered object; adjust the input parameter size of the preset feature extraction network based on the target size information, the image size information of the registered image, and the preset network input standard.

[0076] Referring to the foregoing embodiments, after labeling each registered object in the registered image, the object size information of each registered object can be obtained. However, in actual target detection scenarios, small-sized targets may be difficult to detect.

[0077] To address this, this application proposes a dynamic-size object registration method during object detection. After the registered objects are labeled, the target size information can be determined based on the object size information of each registered object. For example, the average size information (target size information) is calculated based on the object size information of each registered object as [target_h, target_w]. If we take the standard that its mapping to the neural network input size is 16, then the dynamic input size of the object detection model of this application can be set to... Here, image_w and image_h are the dimensions of the original image (equivalent to the image size information of the registered image), ensuring optimal detection results in scenarios with different target sizes. Therefore, this standard can also be used to maintain dynamic input training during the pre-training process of the similarity matching model.

[0078] Referring to the foregoing embodiments, a registered image in this application can simultaneously contain one or more types of registered objects. If multiple types of registered objects exist, the object detection method of this application can be implemented as described in the foregoing embodiments; alternatively, clustering can be performed on each registered object first (for example, various feasible clustering methods in the art can be referred to, which will not be elaborated here), so that registered objects of the same type have the same type label. Then, in the processes of feature extraction, similarity matching, and image segmentation, the processing is performed separately for each type of registered object to avoid confusion between different types of objects. Finally, when outputting the object detection result, the object type of the object detection result can also be output accordingly.

[0079] In summary, the overall process of this application can be referenced as follows: Figure 6 As shown, Figure 6 This is an exemplary overall flowchart of the object detection method based on registered instance segmentation in this application. The image to be detected and the registered image are input into the backbone network for feature extraction (image feature extraction) to obtain the detection features and cross features. Then, the detection features and cross features are input into a similarity matching model for processing, obtaining similarity distribution information and projecting it as a similarity distribution density map, where regions with higher values ​​represent higher similarity. A pre-set similarity threshold can determine the contours and center points of highly similar regions. Combining the object's size information during registration with the center points of similar regions, a rough target region bounding box can be constructed. This bounding box, along with the initial image to be detected, is used to segment the target region using the SAM (Segment Anything Model) semantic segmentation model. The final segmentation results are the object detection results based on registered instance segmentation. The SAM model can also be fine-tuned and optimized for specific application scenarios, such as fine-tuning it with industrial samples in industrial inspection scenarios.

[0080] It should be further noted that the execution entity of the target detection method based on registered instance segmentation can be a target detection device based on registered instance segmentation. For example, the target detection method based on registered instance segmentation can be executed by a terminal device, a server, or other processing devices. The terminal device can be a user equipment (UE), computer, mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this target detection method based on registered instance segmentation can be implemented by a processor calling computer-readable instructions stored in memory.

[0081] Figure 7 This is a block diagram illustrating a target detection device based on registered instance segmentation, as shown in an exemplary embodiment of this application. Figure 7 As shown, the exemplary target detection device 700 based on registered instance segmentation includes: an acquisition module 710, a region determination module 720, and an image segmentation module 730.

[0082] Specifically:

[0083] The acquisition module 710 is used to acquire the registered image and the image to be inspected. The registered image includes pre-annotated registered objects.

[0084] The region determination module 720 is used to determine the target region from the image to be inspected based on the cross features between the registered object and the registered image, wherein the feature similarity between the region features and the cross features of the target region is greater than a preset threshold.

[0085] The image segmentation module 730 is used to perform image segmentation processing on the image to be inspected based on the target region to obtain the target detection result.

[0086] In this exemplary object detection device based on registered instance segmentation, a registered image including pre-annotated registered objects and an image to be detected are acquired. After extracting features from the registered objects, the registered image, and the image to be detected, a pre-trained similarity matching model can be used to determine target regions with high similarity from the image to be detected based on the cross features between the registered objects and the registered image. Then, a segmentation model can be used to perform image segmentation processing on the image to be detected based on the target regions to obtain the object detection result, thus achieving the task of instance segmentation and detection of the target. Compared with traditional object detection methods, where the detection effect deteriorates when the features of the object to be detected change or requires a large amount of model training resources, the method of this application can annotate the registered image to provide example features and achieve object detection through adaptive similarity matching and instance segmentation without retraining the model, thereby improving the efficiency and accuracy of object detection.

[0087] It should be noted that the apparatus and method provided in the above embodiments belong to the same concept, and the specific manner in which each module and unit performs its operation has been described in detail in the method embodiments, and will not be repeated here. In practical applications, the apparatus provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation. The functions of each module can be found in the target detection method embodiment based on registered instance segmentation, and will not be repeated here.

[0088] Please see Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the electronic device of this application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is used to execute program instructions stored in the memory 101 to implement the steps in any of the above embodiments of the target detection method based on registered instance segmentation. In a specific implementation scenario, the electronic device 100 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 100 may also include mobile devices such as laptops and tablets, which are not limited here.

[0089] Specifically, processor 102 controls itself and memory 101 to implement the steps in any of the above-described embodiments of the target detection method based on registered instance segmentation. Processor 102 may also be referred to as a CPU (Central Processing Unit). Processor 102 may be an integrated circuit chip with signal processing capabilities. Processor 102 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 102 may be implemented using integrated circuit chips.

[0090] In this exemplary electronic device, a registered image including a pre-annotated registered object and an image to be detected are acquired. After extracting features from the registered object, the registered image, and the image to be detected, a pre-trained similarity matching model can be used to determine a target region with high similarity from the image to be detected based on the cross features between the registered object and the registered image. Then, a segmentation model can be used to perform image segmentation processing on the image to be detected based on the target region to obtain the target detection result, thus achieving the task of instance segmentation and detection of the target. Compared with traditional target detection methods, where the detection effect deteriorates when the features of the target object change or require a large amount of model training resources, the method of this application can annotate the registered image to provide example features and achieve target detection through adaptive similarity matching and instance segmentation without retraining the model, thereby improving the efficiency and accuracy of target detection.

[0091] Please see Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor. The program instructions 111 are used to implement the steps in any of the above embodiments of the target detection method based on registered instance segmentation.

[0092] In this exemplary storage medium, by running program instructions within the storage medium, a registered image including pre-annotated registered objects and an image to be detected are obtained. After extracting features from the registered objects, the registered image, and the image to be detected, a pre-trained similarity matching model can be used to determine target regions with high similarity from the image to be detected based on the cross features between the registered objects and the registered image. Then, a segmentation model can be used to perform image segmentation processing on the image to be detected based on the target regions to obtain the target detection result, thus achieving the task of instance segmentation and detection of the target. Compared to traditional target detection methods where the detection effect deteriorates when the features of the target object change or require a large amount of model training resources, the method of this application can annotate the registered image to provide example features and achieve target detection through adaptive similarity matching and instance segmentation without retraining the model, thereby improving the efficiency and accuracy of target detection.

[0093] In some embodiments, the apparatus provided in this disclosure may have functions or include modules that can be used to perform the methods described in the above method embodiments. Specific implementations can be referred to the descriptions of the above method embodiments, and for brevity, will not be repeated here. The descriptions of the various embodiments above tend to emphasize the differences between them; their similarities or commonalities can be referred to each other, and for brevity, will not be repeated here.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A target detection method based on registered instance segmentation, characterized in that, The method includes: Acquire a registered image and an image to be inspected, wherein the registered image includes pre-labeled registered objects; Obtain the registration features of the registered image and the object features of the registered object; The registration features and the object features are cross-fused according to a preset attention mechanism to obtain cross features; Based on the cross features between the registered object and the registered image, a target region is determined from the image to be inspected, wherein the feature similarity between the region features of the target region and the cross features is greater than a preset threshold; The step of determining the target region from the image to be inspected based on the cross features between the registered object and the registered image includes: obtaining the features to be inspected from the image to be inspected; and determining the target region from the image to be inspected based on the feature similarity between the cross features and the features to be inspected. The image to be detected is segmented based on the target region to obtain the target detection result.

2. The method according to claim 1, characterized in that, The step of determining the target region from the image to be inspected based on the intersection features between the registered object and the registered image includes: Obtain the features to be detected from the image to be inspected; A similarity distribution map is determined based on the feature similarity between the cross features and the features to be detected; The target region of the image to be inspected is determined based on the similarity distribution map and the preset threshold.

3. The method according to claim 2, characterized in that, Determining the target region of the image to be inspected based on the similarity distribution map and the preset threshold includes: The similarity corresponding to each position in the similarity distribution map is compared with the preset threshold to obtain high similarity regions; The target region is determined based on the region center of the high similarity region and the size information of the registered object.

4. The method according to claim 1, characterized in that, The registration features include shallow registration features and deep registration features; the object features include shallow object features and deep object features; the cross features include shallow cross features and deep cross features; and the cross features are obtained by performing cross-fusion processing on the registration features and the object features according to a preset attention mechanism, including: The shallow registration features and the shallow object features are cross-fused according to the attention mechanism to obtain the shallow cross features; The deep registration features and the deep object features are cross-fused according to the attention mechanism to obtain the deep cross features.

5. The method according to claim 4, characterized in that, The step of determining the target region from the image to be inspected based on the intersection features between the registered object and the registered image includes: Obtain the features to be inspected from the image to be inspected, including shallow features and deep features to be inspected; The shallow cross features and the shallow detectable features are fused and subjected to four-dimensional convolution, and the deep cross features and the deep detectable features are fused and subjected to four-dimensional convolution to obtain shallow fused features and deep fused features. The shallow fusion features and the deep fusion features are fused and upsampled to obtain a similarity distribution map. The target region in the image to be inspected is determined based on the similarity distribution map.

6. The method according to claim 1, characterized in that, Before determining the target region from the image to be inspected based on the cross features between the registered object and the registered image, the method further includes: Obtain the object size information of each registered object in the registered image; Determine the target size information based on the object size information of each registered object; The input parameters of the preset feature extraction network are adjusted based on the target size information, the image size information of the registered image, and the preset network input standard.

7. A target detection device based on registered instance segmentation, characterized in that, The device includes: The acquisition module is used to acquire a registered image and an image to be inspected, wherein the registered image includes pre-annotated registered objects; acquire the registration features of the registered image and the object features of the registered objects; and perform cross-fusion processing on the registration features and the object features according to a preset attention mechanism to obtain cross features; A region determination module is used to determine a target region from an image to be inspected based on the intersection features between the registered object and the registered image, wherein the feature similarity between the region features of the target region and the intersection features is greater than a preset threshold; the step of determining the target region from the image to be inspected based on the intersection features between the registered object and the registered image includes: acquiring the features to be inspected from the image to be inspected; and determining the target region from the image to be inspected based on the feature similarity between the intersection features and the features to be inspected. The image segmentation module is used to perform image segmentation processing on the image to be detected based on the target region to obtain the target detection result.

8. An electronic device, characterized in that, The method includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images