A target detection method and device
By generating a prototype of the support map and using masking, the problem of low detection accuracy in machine vision under different lighting and environments was solved, achieving higher accuracy target recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU HUAXING YUANCHUANG TECH CO LTD
- Filing Date
- 2023-02-03
- Publication Date
- 2026-05-01
AI Technical Summary
Existing machine vision inspection technology has low detection accuracy for the same object under different lighting and environments, and is easily affected by the imaging of light sources, resulting in inaccurate detection results.
By generating a prototype of the support map, the similarity score between the image to be detected and the prototype is calculated, and the target region is filtered out using the first and second masks. By combining the neural network model and mask processing, the target in the image to be detected can be accurately identified.
It improves the target detection accuracy under different lighting and environments, and can more accurately identify targets in the image to be detected, adapting to the detection needs under different conditions.
Smart Images
Figure CN116051959B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of machine vision, and in particular to a method and apparatus for object detection. Background Technology
[0002] In the field of industrial inspection, in order to detect whether a machine is malfunctioning, defect samples are usually collected and deep learning is performed to detect possible anomalies during machine operation.
[0003] In existing technologies, machine vision suffers from low accuracy, is easily affected by light source imaging, and relies on high-quality datasets. Changes in lighting conditions, object density, and environment can cause detection models to produce different detection results for the same object in images under different environments, resulting in low accuracy. Summary of the Invention
[0004] Based on this, it is necessary to provide a target detection method to address the above-mentioned technical problems. This method can generate a prototype of the support map, compare the detected image with the prototype, and determine the prototype that matches the image to be detected by the prototype similarity score. The obtained prototype can be further filtered according to the first mask and the second mask, so as to obtain the target in the region to be detected more accurately.
[0005] Firstly, this application provides a target detection method. The method includes:
[0006] Acquire the image to be detected;
[0007] The image to be detected is input into a neural network model to obtain a feature map of the image to be detected. The similarity between the feature map and the prototype is calculated to obtain a similarity score of the prototype. Based on the similarity score of the prototype, a prototype that matches the feature map is determined. The prototype includes a background prototype and a target prototype.
[0008] The prototype is processed according to the first mask to obtain the target prototype, wherein the first mask is used to identify the region in the image that does not include the background.
[0009] The target prototype is processed according to the second mask to obtain the target in the image to be detected. The second mask is used to identify the target region in the image.
[0010] In one embodiment, the prototype includes:
[0011] Acquire an image containing the target;
[0012] The image is input into a neural network model to obtain the feature map of the image;
[0013] The feature map is processed according to the third mask to obtain a background feature map and a target feature map. The third mask is used to distinguish the background and the target in the image. A background prototype and a target prototype are generated according to the background feature map and the target feature map. The prototype is obtained by clustering prototype and superpixel prototype.
[0014] In one embodiment, the method further includes:
[0015] The prototypes are classified according to the logistic regression function, and the maximum value independent variable point set function is used on the obtained prototype categories to obtain the prototype label map, which corresponds to the category information of the prototype.
[0016] Based on the prototype category information, the prototype label map is mapped to a category label map to obtain the category of the target in the image to be detected.
[0017] In one embodiment, if the target prototype does not include the target in the image to be detected, the target is obtained according to the anomaly detection algorithm, and the category of the target is determined by the rule engine.
[0018] In one embodiment, supervised learning is performed on an image containing a target, at least one of which is included, to obtain a second mask, wherein the second mask is different for different targets.
[0019] In one embodiment, if the number of images to be detected that include the target is less than a threshold, the target of the image to be detected is obtained by unsupervised learning.
[0020] If the number of images to be detected that include the target is greater than the threshold, the mask is obtained according to supervised learning to obtain the target of the image to be detected;
[0021] If the number of images to be detected that include the new target is less than a threshold, the new target in the image to be detected is obtained according to unsupervised learning. When the number of images to be detected that include the new target is greater than the threshold, the mask is obtained according to supervised learning and used to identify the new target in the image to be detected.
[0022] Secondly, this application also provides a target detection device, the device comprising:
[0023] The image acquisition module is used to acquire the image to be detected;
[0024] The similarity calculation module is used to input the image to be detected into a neural network model to obtain a feature map of the image to be detected, calculate the similarity between the feature map and the prototype to obtain a similarity score of the prototype, and determine the prototype that matches the feature map based on the similarity score of the prototype. The prototype includes a background prototype and a target prototype.
[0025] The first matching module is used to process the prototype according to the first mask to obtain the target prototype, wherein the first mask is used to identify the region in the image that does not include the background.
[0026] The second matching module is used to process the target prototype according to the second mask to obtain the target in the image to be detected. The second mask is used to identify the target region in the image.
[0027] In one embodiment, the prototype includes:
[0028] Acquire an image containing the target;
[0029] The image is input into a neural network model to obtain the feature map of the image;
[0030] The feature map is processed according to the third mask to obtain a background feature map and a target feature map. The third mask is used to distinguish the background and the target in the image. A background prototype and a target prototype are generated according to the background feature map and the target feature map. The prototype is obtained by clustering prototype and superpixel prototype.
[0031] In one embodiment, the device further includes:
[0032] The first classification module is used to classify prototypes according to a logistic regression function, and to obtain a prototype label map by using a maximum independent variable point set function on the obtained prototype categories. The prototype label map corresponds to the category information of the prototype.
[0033] The second classification module is used to map the prototype label map to a category label map based on the prototype category information, so as to obtain the category of the target in the image to be detected.
[0034] In one embodiment, if the target prototype does not include the target in the image to be detected, the target is obtained according to the anomaly detection algorithm, and the category of the target is determined by the rule engine.
[0035] In one embodiment, supervised learning is performed on an image containing a target, at least one of which is included, to obtain a second mask, wherein the second mask is different for different targets.
[0036] In one embodiment, if the number of images to be detected that include the target is less than a threshold, the target of the image to be detected is obtained by unsupervised learning.
[0037] If the number of images to be detected that include the target is greater than the threshold, the mask is obtained according to supervised learning to obtain the target of the image to be detected;
[0038] If the number of images to be detected that include the new target is less than a threshold, the new target in the image to be detected is obtained according to unsupervised learning. When the number of images to be detected that include the new target is greater than the threshold, the mask is obtained according to supervised learning and used to identify the new target in the image to be detected.
[0039] Thirdly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the target detection method.
[0040] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the target detection method.
[0041] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of a target detection method.
[0042] The above-mentioned target detection method has at least the following beneficial effects: in the early stage of detection, the number of images containing targets is small, and they can be classified by unsupervised detection; after a period of detection, the number of images containing targets may increase, and targets in the images to be detected can be obtained through supervised learning; if new targets appear, new targets can be identified by anomaly detection algorithms.
[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments or conventional technologies of this disclosure, the accompanying drawings used in the description of the embodiments or conventional technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 This is a diagram illustrating the application environment of a target detection method in one embodiment;
[0046] Figure 2 This is a flowchart illustrating a target detection method in one embodiment;
[0047] Figure 3 This is a schematic diagram of prototype generation in one embodiment;
[0048] Figure 4 This is a schematic diagram illustrating the generation of the target category in one embodiment;
[0049] Figure 5 This is a schematic diagram of a target detection method in one embodiment;
[0050] Figure 6 This is a structural block diagram of a target detection device in one embodiment;
[0051] Figure 7 This is a structural block diagram of a target detection device in one embodiment;
[0052] Figure 8 This is an internal structural diagram of a computer device in one embodiment;
[0053] Figure 9 This is an internal structure diagram of a server in one embodiment. Detailed Implementation
[0054] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0055] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any specific order.
[0056] This disclosure provides a target detection method that can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0057] In some embodiments of this disclosure, such as Figure 2 As shown, a target detection method is provided, which is applied to... Figure 1 The method is illustrated using the example of a server processing a target. It is understood that this method can be applied to a server, and also to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In a specific embodiment, the method may include the following steps:
[0058] S202: Acquire the image to be detected.
[0059] The image to be detected in this embodiment typically includes an image of a target. The target usually refers to an image of a specific object to be identified, such as a defect on a manufactured product in the application scenario of this embodiment. It can also be used to detect defects such as spots, pits, scratches, color differences, and defects on the surface of a workpiece. The image to be detected can include an image obtained by capturing the target object using a camera mounted on an accessory to the target object, an image transmitted from the camera via hardware or a communication network, or an image containing the target object directly input into the processing device.
[0060] S204: Input the image to be detected into a neural network model to obtain a feature map of the image to be detected, calculate the similarity between the feature map and the prototype to obtain a similarity score of the prototype, and determine the prototype that matches the feature map based on the similarity score of the prototype. The prototype includes a background prototype and a target prototype.
[0061] First, necessary feature information can be extracted from the image, such as Histogram of Oriented Gradients (HOG) features and Haar features. These extracted features can then be used to locate and classify the target in the image to be detected. Next, a neural network model can be used to extract features from the image to be detected, resulting in a feature map. This feature map represents the main information of the target object in the image, and based on this, similar or identical target objects can be detected from other images. Neural network models can include Residual Neural Networks (ResNet), VggNet, etc.
[0062] A prototype can refer to a representative point, category, or feature in the sample space. A prototype can be represented as a representative template, and prototype clustering can be understood as clustering based on a prototype, completing the clustering process by referencing a template vector or template distribution. Prototype clustering algorithms can include K-means, Gaussian clustering, and binary K-means. Clustering algorithms can first initialize the prototypes and then iteratively update them to obtain prototypes with smaller errors.
[0063] In machine learning, feature similarity and category can be determined by judging the magnitude of differences between different features. The similarity score is obtained by calculating the similarity between a feature map and its prototype. This similarity score is typically obtained by comparing the similarity between two features, usually by calculating the distance between them. A smaller distance generally indicates higher similarity, and vice versa. Similarity calculation methods can include Euclidean distance, Manhattan distance, cosine similarity, Minkowski distance, etc., and can calculate image similarity from multiple aspects such as brightness, contrast, and structure.
[0064] Based on the similarity score of the prototypes, prototypes that match the feature maps are determined. During the matching process, one or more prototypes may be obtained. If multiple prototypes are obtained, the top five or top ten similarity scores can be selected for subsequent filtering. Prototypes are obtained through clustering of support maps. The prototype map can be an image containing the target, and it can include either the background or the target. During prototype clustering, the background prototype and the target prototype are relatively far apart. Random initialization can generate background prototypes and target prototypes. Multiple support maps may contain multiple categories. During the update iteration process, the background prototype and the target prototype can be clustered again. One category may contain multiple prototypes.
[0065] S206: Process the prototype according to the first mask to obtain the target prototype, wherein the first mask is used to identify the region in the image that does not include the background.
[0066] Feature maps of good-quality images can be extracted using neural network models. A good-quality image can be represented as an image that does not contain the target object, while a defective image can be represented as an image that contains the target object. Anomaly detection algorithms can be used to model the feature maps of good-quality images to identify outliers that do not belong to the feature points. The core of anomaly detection lies in finding a probability model. Based on this model, the probability of a sample falling into the normal sample group can be determined, distinguishing between normal and abnormal samples. Anomaly detection algorithms can include Gaussian distribution, isolated forest algorithms, etc. In some embodiments of this disclosure, a Gaussian distribution model can be established for the feature maps. Generally, the features of a Gaussian distribution are dense in the middle and sparse at the edges. Good-quality images can be mapped to the vicinity of the center of the Gaussian distribution, while defective images can be mapped to the edges. Based on the Gaussian distribution model, the distribution of each feature can be intuitively obtained.
[0067] The mask value can be either 0 or non-zero. When the mask value is non-zero, the non-zero region is selected; when the mask value is 0, it remains unchanged. The mask can be used to extract regions of interest, mask regions of no interest, and extract structural features. In some embodiments of this disclosure, a first mask can be obtained based on a Gaussian distribution model. The first mask can be represented as a background region where the value is 0, and the remaining regions are non-zero. The first mask can be used in conjunction with the prototype to obtain the target prototype.
[0068] S208: The target prototype is processed according to the second mask to obtain the target in the image to be detected. The second mask is used to identify the target region in the image.
[0069] During image acquisition, as the number of detections increases, the number of images containing targets may also increase. Supervised learning can be used to build a model to extract targets from the images. An image may contain multiple targets; a second mask can be used to mark the target locations. The second mask can be represented as a non-zero value for the target portion and a zero value for the remaining regions. The second mask is then used in conjunction with the target prototype to obtain the target in the image.
[0070] In the above target detection method, the feature map of the image to be detected can be extracted first, and the similarity score between the feature map and the prototype can be calculated. The prototype with the higher similarity score can be selected and operated again with the first mask and the second mask to obtain the target in the image to be detected.
[0071] In some embodiments of this disclosure, the prototype includes:
[0072] Acquire an image containing the target;
[0073] The image is input into a neural network model to obtain the feature map of the image;
[0074] The feature map is processed according to the third mask to obtain a background feature map and a target feature map. The third mask is used to distinguish the background and the target in the image. A background prototype and a target prototype are generated according to the background feature map and the target feature map. The prototype is obtained by clustering prototype and superpixel prototype.
[0075] like Figure 3 As shown, Figure 3 This is a schematic diagram of prototype generation in one embodiment. The third mask can be represented as a background region of 0 and a target region of non-0, used to distinguish between the background region and the target region. In some embodiments of this disclosure, feature maps containing the target image can be extracted using a neural network model. Superpixels are used to divide the background and target regions marked by the third mask into multiple sub-regions. Superpixel segmentation can divide the feature map into multiple superpixel blocks, with relatively clear features and boundaries between the blocks. During superpixel clustering, points with similar pixel values can be clustered to obtain superpixel clusters. Superpixel clustering uses the superpixel centroid as a prototype, and the superpixel-guided clustering method is adaptive, adjusting the number of prototypes generated according to the size of the target in the image to be detected. Prototype clustering can be performed again on prototypes of the same target; one target can correspond to multiple prototypes.
[0076] In some embodiments of this disclosure, the method further includes:
[0077] The prototypes are classified according to the logistic regression function, and the maximum value independent variable point set function is used on the obtained prototype categories to obtain the prototype label map, which corresponds to the category information of the prototype.
[0078] Based on the prototype category information, the prototype label map is mapped to a category label map to obtain the category of the target in the image to be detected.
[0079] like Figure 4 As shown, Figure 4 This is a schematic diagram illustrating target category generation in one embodiment. The distance between the feature vector of each spatial location in the image to be detected and each computed prototype is calculated. This distance is input into a logistic regression function (softmax) to obtain the probability that the feature vector of each spatial location in the image to be detected matches the prototype, forming a corresponding prototype label. The prototype with the highest probability corresponding to each spatial location can then be predicted using a maximum independent variable point set function (argmax). Based on the prototype category information, the prototype label map can be mapped to a category label map to obtain the category of the target in the image to be detected. The distance function typically uses cosine distance or squared Euclidean distance. In some embodiments of this disclosure, using cosine distance is more stable, has better performance, and is easier to optimize.
[0080] In some embodiments of this disclosure, if the target prototype does not include the target in the image to be detected, the target is obtained according to an anomaly detection algorithm, and the category of the target is determined by a rule engine.
[0081] The prototype is obtained through clustering prototypes and superpixel prototypes, and it is obtained from images containing the target. During the clustering process, new targets may appear, but there is no corresponding prototype, so it may be impossible to find the category of the target using the prototype. In the above process of modeling the good product image, the Gaussian distribution model obtained by the anomaly detection algorithm can identify new targets. The category of the target can be determined by a rule engine. The rule engine can be a manual judgment method. Since the probability and number of new targets appearing are small, using a rule engine to determine the category can meet the judgment requirements.
[0082] In some embodiments of this disclosure, supervised learning is performed on an image containing a target, wherein the target includes at least one, to obtain a second mask, and the second mask is different for different targets.
[0083] The image to be detected may contain one target or multiple targets. When performing supervised learning on an image containing targets, the recognition model can be trained by annotation. The number of generated second masks can correspond to the number of targets, and the second masks are different for different targets.
[0084] like Figure 5 As shown, Figure 5 This is a schematic diagram of a target detection method in one embodiment. The support map can represent an image containing the target, the good image can represent an image not containing the target, the supervision map can represent multiple images currently containing the target, used to generate a second mask, and the test map can be used to detect targets contained in the images. The prototype generator can include multiple clustering methods for prototype clustering.
[0085] In some embodiments of this disclosure, if the number of images to be detected that include the target is less than a threshold, the target of the image to be detected is obtained by unsupervised learning.
[0086] If the number of images to be detected that include the target is greater than the threshold, the mask is obtained according to supervised learning to obtain the target of the image to be detected;
[0087] If the number of images to be detected that include the new target is less than a threshold, the new target in the image to be detected is obtained according to unsupervised learning. When the number of images to be detected that include the new target is greater than the threshold, the mask is obtained according to supervised learning and used to identify the new target in the image to be detected.
[0088] The threshold can depend on the number of images to be detected required for training. In some embodiments of this disclosure, when the yield rate is high, it is difficult to obtain defect images higher than the threshold, while good product images are readily available. Defect images can be found using unsupervised learning. Unsupervised learning can cluster defect images into one class and good product images into another, which can be used to find defect images. After the production line has been running for a period of time, defect images higher than the threshold can be obtained. Defect images can be clustered according to the aforementioned prototypes to meet the detection requirements. Severe defects and specific defects have a low probability of occurrence, and their number is generally less than the threshold. Unsupervised learning can be used to first distinguish the images to be detected that include new targets. When the number exceeds the threshold, supervised learning can then be used for detection.
[0089] In the above-mentioned object detection method, feature maps of the image to be detected can be extracted through a neural network model, the similarity between the feature maps and the prototype can be calculated, and then the prototype can be processed using a first mask and a second mask to obtain the category of the target.
[0090] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0091] Based on the same inventive concept, this disclosure also provides a target detection apparatus for implementing the target detection method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in the target detection apparatus embodiments provided below can be found in the limitations of the target detection method described above, and will not be repeated here.
[0092] The apparatus may include a system (including a distributed system), software (application), module, component, server, client, etc., that uses the methods described in the embodiments of this specification, combined with necessary hardware implementation. Based on the same innovative concept, the apparatuses in one or more embodiments provided in this disclosure are as described in the following embodiments. Since the implementation schemes and methods for solving the problem by the apparatus are similar, the implementation of the specific apparatus in the embodiments of this specification can refer to the implementation of the foregoing methods, and repeated details will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatuses described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.
[0093] In one embodiment, such as Figure 6 As shown, a target detection device 600 is provided. This device can be the aforementioned server, or a module, component, device, or unit integrated into the server. The device 600 may include:
[0094] Image acquisition module 602 is used to acquire the image to be detected;
[0095] The similarity calculation module 604 is used to input the image to be detected into a neural network model to obtain a feature map of the image to be detected, calculate the similarity between the feature map and the prototype to obtain a similarity score of the prototype, and determine the prototype that matches the feature map based on the similarity score of the prototype. The prototype includes a background prototype and a target prototype.
[0096] The first matching module 606 is used to process the prototype according to the first mask to obtain the target prototype, wherein the first mask is used to identify the region in the image that does not include the background.
[0097] The second matching module 608 is used to process the target prototype according to the second mask to obtain the target in the image to be detected, wherein the second mask is used to identify the target region in the image.
[0098] In one embodiment, the prototype includes:
[0099] Acquire an image containing the target;
[0100] The image is input into a neural network model to obtain the feature map of the image;
[0101] The feature map is processed according to the third mask to obtain a background feature map and a target feature map. The third mask is used to distinguish the background and the target in the image. A background prototype and a target prototype are generated according to the background feature map and the target feature map. The prototype is obtained by clustering prototype and superpixel prototype.
[0102] In one embodiment, such as Figure 7 As shown, the device further includes:
[0103] The first classification module 702 is used to classify prototypes according to a logistic regression function, and to use a maximum value independent variable point set function on the obtained prototype categories to obtain a prototype label map, wherein the prototype label map corresponds to the category information of the prototype.
[0104] The second classification module 704 is used to map the prototype label map to a category label map based on the prototype category information to obtain the category of the target in the image to be detected.
[0105] In one embodiment, if the target prototype does not include the target in the image to be detected, the target is obtained according to the anomaly detection algorithm, and the category of the target is determined by the rule engine.
[0106] In one embodiment, supervised learning is performed on an image containing a target, which includes at least one target, to obtain a second mask, wherein the second mask is different for different targets.
[0107] In one embodiment, if the number of images to be detected that include the target is less than a threshold, the target of the image to be detected is obtained by unsupervised learning.
[0108] If the number of images to be detected that include the target is greater than the threshold, the mask is obtained according to supervised learning to obtain the target of the image to be detected;
[0109] If the number of images to be detected that include the new target is less than a threshold, the new target in the image to be detected is obtained according to unsupervised learning. When the number of images to be detected that include the new target is greater than the threshold, the mask is obtained according to supervised learning and used to identify the new target in the image to be detected.
[0110] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0111] The various modules in the aforementioned target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0112] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores a mask. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a target detection method.
[0113] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a target detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0114] Those skilled in the art will understand that Figure 8 , Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the computer device to which the present disclosure is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0115] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described in any embodiment of this disclosure.
[0116] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described in any embodiment of this disclosure.
[0117] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this disclosure may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this disclosure may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0119] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the appended claims.
Claims
1. A target detection method, characterized in that, The method includes: Acquire the image to be detected; The image to be detected is input into a neural network model to obtain a feature map of the image to be detected. The similarity between the feature map and the prototype is calculated to obtain a similarity score for the prototype. Based on the similarity score, a prototype matching the feature map is determined. The prototype includes a background prototype and a target prototype. The prototype includes: Acquire an image containing the target; The image is input into a neural network model to obtain the feature map of the image; The feature map is processed according to the third mask to obtain a background feature map and a target feature map. The third mask is used to distinguish between the background and the target in the image. A background prototype and a target prototype are generated according to the background feature map and the target feature map. The prototype is obtained by clustering prototype and superpixel prototype. The prototype is processed according to the first mask to obtain the target prototype, wherein the first mask is used to identify the region in the image that does not include the background. The target prototype is processed according to the second mask to obtain the target in the image to be detected. The second mask is used to identify the target region in the image.
2. The method according to claim 1, characterized in that, The method further includes: The prototypes are classified according to the logistic regression function, and the maximum value independent variable point set function is used on the obtained prototype categories to obtain the prototype label map, which corresponds to the category information of the prototype. Based on the prototype category information, the prototype label map is mapped to a category label map to obtain the category of the target in the image to be detected.
3. The method according to claim 1, characterized in that, If the target prototype does not include the target in the image to be detected, the target is obtained according to the anomaly detection algorithm, and the category of the target is determined by the rule engine.
4. The method according to claim 1, characterized in that, Supervised learning is performed on an image containing a target, wherein the target includes at least one, to obtain a second mask, and the second mask is different for different targets.
5. The method according to any one of claims 1 to 4, characterized in that, If the number of images to be detected that include the target is less than a threshold, the target of the image to be detected is obtained by unsupervised learning. If the number of images to be detected that include the target is greater than the threshold, the mask is obtained according to supervised learning to obtain the target of the image to be detected; If the number of images to be detected that include the new target is less than a threshold, the new target in the image to be detected is obtained according to unsupervised learning. When the number of images to be detected that include the new target is greater than the threshold, the mask is obtained according to supervised learning and used to identify the new target in the image to be detected.
6. A target detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be detected; A similarity calculation module is used to input the image to be detected into a neural network model to obtain a feature map of the image to be detected, calculate the similarity between the feature map and the prototype to obtain a similarity score of the prototype, and determine the prototype that matches the feature map based on the similarity score of the prototype. The prototype includes a background prototype and a target prototype. The prototype includes: Acquire an image containing the target; The image is input into a neural network model to obtain the feature map of the image; The feature map is processed according to the third mask to obtain a background feature map and a target feature map. The third mask is used to distinguish between the background and the target in the image. A background prototype and a target prototype are generated according to the background feature map and the target feature map. The prototype is obtained by clustering prototype and superpixel prototype. The matching module is used to process the prototype according to a first mask to obtain a target prototype, wherein the first mask is used to identify regions in the image that do not include the background; and to process the target prototype according to a second mask to obtain a target in the image to be detected, wherein the second mask is used to identify the target region in the image.
7. The apparatus according to claim 6, characterized in that, The device further includes: The first classification module is used to classify prototypes according to a logistic regression function, and to obtain a prototype label map by using a maximum independent variable point set function on the obtained prototype categories. The prototype label map corresponds to the category information of the prototype. The second classification module is used to map the prototype label map to a category label map based on the prototype category information, so as to obtain the category of the target in the image to be detected.
8. The apparatus according to claim 6, characterized in that, If the target prototype does not include the target in the image to be detected, the target is obtained according to the anomaly detection algorithm, and the category of the target is determined by the rule engine.
9. The apparatus according to claim 6, characterized in that, Supervised learning is performed on an image containing a target, wherein the target includes at least one, to obtain a second mask, and the second mask is different for different targets.
10. The apparatus according to any one of claims 6 to 9, characterized in that, If the number of images to be detected that include the target is less than a threshold, the target of the image to be detected is obtained by unsupervised learning. If the number of images to be detected that include the target is greater than the threshold, the mask is obtained according to supervised learning to obtain the target of the image to be detected; If the number of images to be detected that include the new target is less than a threshold, the new target in the image to be detected is obtained according to unsupervised learning. When the number of images to be detected that include the new target is greater than the threshold, the mask is obtained according to supervised learning and used to identify the new target in the image to be detected.
Citation Information
Patent Citations
Target detection method and device
CN110096960A
Panoramic segmentation method, system and device based on graph neural network and storage medium
CN111428726A