Image target detection method, device, equipment and storage medium

Abstract

This application discloses an image target detection method, apparatus, device, and storage medium. The method involves acquiring a target image to be detected and inputting it into an image segmentation model for segmentation processing to obtain a first mask image. The target positions in each first connected component of the first mask image are detected. Based on the target positions, image content corresponding to the first connected components is extracted from the target image and replaced in the first mask image to obtain a first sub-image. The first sub-image and the target image are then weighted and superimposed to obtain a second sub-image. Each second sub-image is input into a trained metric learning model, which determines the first feature data corresponding to the second sub-image. Based on the first feature data, the second sub-images are clustered. The first connected components are merged based on the clustering results, and the target detection result corresponding to the target image is determined based on the merging results. This method can improve the accuracy of target detection. This application can be widely applied in the field of artificial intelligence technology.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image target detection method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, various types of machine learning models have achieved good application results in fields such as image classification, facial recognition, and autonomous driving. In the production field, there are applications using computer vision technology to detect defects in target objects. For example, in tasks such as removing impurities from food, tea, and medicinal materials using automated equipment, image segmentation technology can be used to obtain a mask image of the impurities. Then, the coordinates of the impurities are calculated from the mask image and fed back to the robotic arm to detect the impurities.

[0003] In related technologies, the results obtained from image segmentation models are often used as the target detection results for the corresponding images. However, in applications, it has been found that in many scenarios, the target may be covered by other materials, resulting in fragmented segmentation results that cannot accurately reflect the actual situation, leading to low detection accuracy and poor practicality.

[0004] Therefore, the problems existing in the current technology still need to be solved and optimized. Summary of the Invention

[0005] The purpose of this application is to at least partially solve one of the technical problems existing in the related art.

[0006] Therefore, one objective of the embodiments of this application is to provide an image target detection method, apparatus, device, and storage medium.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include:

[0008] On one hand, embodiments of this application provide an image target detection method, the method comprising:

[0009] Acquire the target image to be detected;

[0010] The target image is input into a trained image segmentation model, and the target image is segmented by the image segmentation model to obtain a first mask image corresponding to the target image;

[0011] Detect the target location of each first connected component in the first mask image;

[0012] For each first connected component, the image content corresponding to the first connected component is extracted from the target image according to the target location and replaced in the first mask image to obtain the first sub-image corresponding to each first connected component;

[0013] The first subgraph and the target image are weighted and superimposed to obtain the second subgraph corresponding to each first connected component;

[0014] Each of the second subgraphs is input into the trained metric learning model, and the first feature data corresponding to the second subgraph is determined through the metric learning model;

[0015] Based on the first feature data, the second sub-graph is clustered, the first connected components are merged based on the clustering results, and the target detection result corresponding to the target image is determined based on the merging results.

[0016] In addition, the image target detection apparatus according to the above embodiments of this application may also have the following additional technical features:

[0017] Furthermore, in one embodiment of this application, the image segmentation model is trained through the following steps:

[0018] Obtain multiple first sample images and a first label corresponding to each first sample image; wherein, the first label is used to indicate the positional contour of the sample object in the first sample image;

[0019] The first sample image is input into the image segmentation model to be trained, and the first sample image is segmented by the image segmentation model to obtain the second mask image corresponding to the first sample image.

[0020] Based on the second mask image and the first label, determine the first loss value for training;

[0021] Based on the first loss value, the parameters of the image segmentation model are updated to obtain a trained image segmentation model.

[0022] Furthermore, in one embodiment of this application, after the step of acquiring multiple first sample images, the method further includes:

[0023] The first sample image is preprocessed, and the preprocessing operation includes at least one of denoising and normalization.

[0024] Furthermore, in one embodiment of this application, the metric learning model is trained through the following steps:

[0025] Obtain multiple second sample images and a second label corresponding to each second sample image; wherein, the second label is used to indicate the positional contour of the sample object in the second sample image;

[0026] The second sample image is input into the trained image segmentation model, and the image segmentation model is used to segment the second sample image to obtain the third mask image corresponding to the second sample image.

[0027] Detect the original position of each second connected component in the third mask image;

[0028] For each second connected component, the image content corresponding to the second connected component is extracted from the second sample image according to the original position and replaced in the second mask image to obtain a third sub-image corresponding to each second connected component;

[0029] The third subgraph and the second sample image are weighted and superimposed to obtain the fourth subgraph corresponding to each second connected component;

[0030] Based on the second label, determine the degree of association between any two of the fourth subgraphs;

[0031] Each of the fourth subgraphs is input into the metric learning model to be trained, and the second feature data corresponding to the fourth subgraph is determined by the metric learning model.

[0032] Calculate the Euclidean distance between any two fourth subgraphs based on the second feature data corresponding to the fourth subgraph.

[0033] The second loss value for training is determined based on the Euclidean distance and the correlation degree.

[0034] Based on the second loss value, the parameters of the metric learning model are updated to obtain the trained metric learning model.

[0035] Furthermore, in one embodiment of this application, determining the correlation degree between any two of the fourth subgraphs based on the second label includes:

[0036] If the second label determines that the two fourth subgraphs belong to the same sample object, the correlation between the two fourth subgraphs is determined to be 1; or, if the second label determines that the two fourth subgraphs do not belong to the same sample object, the correlation between the two fourth subgraphs is determined to be 0.

[0037] Further, in one embodiment of this application, determining the first loss value for training based on the second mask image and the first label includes:

[0038] Based on the second mask image and the first label, the first loss value for training is determined using the cross-entropy loss function.

[0039] Furthermore, in one embodiment of this application, the clustering of the second subgraph includes:

[0040] The second subgraph is clustered using the K-means clustering algorithm.

[0041] On the other hand, embodiments of this application provide an image target detection device, the device comprising:

[0042] The acquisition unit is used to acquire the target image to be detected;

[0043] The segmentation unit is used to input the target image into a trained image segmentation model, and to segment the target image through the image segmentation model to obtain a first mask image corresponding to the target image;

[0044] The detection unit is used to detect the target location of each first connected component in the first mask image;

[0045] The replacement unit is configured to, for each first connected component, extract the image content corresponding to the first connected component from the target image according to the target location and replace it in the first mask image to obtain a first sub-image corresponding to each first connected component;

[0046] The overlay unit is used to weight overlay the first subgraph and the target image to obtain a second subgraph corresponding to each first connected component.

[0047] An extraction unit is used to input each of the second subgraphs into a trained metric learning model, and to determine the first feature data corresponding to the second subgraph through the metric learning model;

[0048] The merging unit is used to cluster the second sub-graph according to the first feature data, merge the first connected components according to the clustering results, and determine the target detection result corresponding to the target image according to the merging results.

[0049] On the other hand, embodiments of this application provide a computer device, including:

[0050] At least one processor;

[0051] At least one memory for storing at least one program;

[0052] When the at least one program is executed by the at least one processor, the at least one processor implements the image target detection method described above.

[0053] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the above-described image target detection method.

[0054] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application:

[0055] This application discloses an image target detection method, which involves: acquiring a target image to be detected; inputting the target image into a trained image segmentation model to segment the target image, obtaining a first mask image corresponding to the target image; detecting the target position of each first connected component in the first mask image; for each first connected component, extracting image content corresponding to the first connected component from the target image according to the target position and replacing it in the first mask image, obtaining a first sub-image corresponding to each first connected component; weighted superposition of the first sub-image and the target image to obtain a second sub-image corresponding to each first connected component; inputting each second sub-image into a trained metric learning model to determine first feature data corresponding to the second sub-image; clustering the second sub-images according to the first feature data; merging the first connected components according to the clustering results; and determining the target detection result corresponding to the target image based on the merging results. This method can improve the accuracy and practicality of target detection by extracting deep features of connected components obtained from image segmentation, clustering connected components, and merging connected components belonging to related targets. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0057] Figure 1 This is a schematic diagram of the implementation environment of an image target detection method provided in this application embodiment;

[0058] Figure 2 This is a flowchart illustrating an image target detection method provided in an embodiment of this application;

[0059] Figure 3 This is a schematic diagram illustrating an image segmentation process to obtain a mask image, as provided in an embodiment of this application.

[0060] Figure 4 This is a schematic diagram of a mask sub-image provided in an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of a first sub-diagram provided in an embodiment of this application;

[0062] Figure 6 This is a schematic diagram of a second sub-diagram provided in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram illustrating the principle of a metric learning model provided in the embodiments of this application;

[0064] Figure 8 This is a flowchart illustrating the implementation of an image target detection method provided in this application embodiment;

[0065] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0067] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0068] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0070] First, let's analyze some of the terms used in this application:

[0071] 1) Artificial intelligence (AI) is a new branch of computer science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. AI attempts to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results—theories, methods, technologies, and application systems for this purpose.

[0072] 2) Machine learning, an interdisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0073] 3) Deep learning, a branch of machine learning, is a machine learning method based on artificial neural networks. Compared to traditional machine learning methods, deep learning learns feature representations of data through multi-layered neural network models, thereby enabling the learning and modeling of complex patterns and relationships. Deep learning has wide applications in various fields, including computer vision, natural language processing, speech recognition, and recommender systems. In computer vision, deep learning can be used for tasks such as image classification, object detection, and image generation; in natural language processing, it can be used for tasks such as text classification, machine translation, and sentiment analysis.

[0074] 4) Image segmentation refers to dividing an image into several non-overlapping regions based on features such as grayscale, color, spatial texture, and geometric shape. These features exhibit consistency or similarity within the same region, while showing significant differences between different regions. Simply put, it's about separating the target from the background in an image.

[0075] 5) Connected regions generally refer to image regions composed of foreground pixels with the same pixel value and adjacent positions.

[0076] 6) Metric learning, also known as similarity learning, is a spatial mapping method that can learn a feature space. It transforms all data into feature vectors in a common space. In these vectors, the feature vectors of more similar samples are closer together, while the feature vectors of dissimilar samples are farther apart, thus distinguishing the data.

[0077] In recent years, with the rapid development of artificial intelligence technology, various types of machine learning models have achieved good application results in fields such as image classification, facial recognition, and autonomous driving. In the production field, there are applications using computer vision technology to detect defects in target objects. For example, in tasks such as removing impurities from food, tea, and medicinal materials using automated equipment, image segmentation technology can be used to obtain a mask image of the impurities. Then, the coordinates of the impurities are calculated from the mask image and fed back to the robotic arm to detect the impurities.

[0078] In related technologies, the results obtained from image segmentation models are often used as the target detection results for the corresponding images. However, in applications, it has been found that in many scenarios, the target may be covered by other substances, resulting in fragmented segmentation results that cannot accurately reflect the actual situation, leading to low detection accuracy and poor practicality. For example, in the aforementioned automated equipment task of removing impurities from food, tea, and medicinal materials, some larger and longer impurities may be covered by other substances, causing the same impurity to be classified as multiple targets during detection. The robotic arm needs to pick up each impurity target, increasing the number of ineffective operations and reducing efficiency.

[0079] In view of this, this application provides an image target detection method. This method can improve the accuracy of target detection and enhance its practicality by extracting deep features of connected components obtained from image segmentation, clustering the connected components, and merging connected components belonging to related targets.

[0080] The following section first describes the implementation environment of the image target detection method provided in the embodiments of this application. (Refer to...) Figure 1 , Figure 1 A schematic diagram of an implementation environment for an image target detection method is provided. The main hardware and software components of this environment include a terminal device 110 and a server 120, which are communicatively connected. The image target detection method can be configured on the terminal device 110 side, or it can be implemented through the interaction between the terminal device 110 and the server 120.

[0081] Specifically, in this embodiment, the terminal device 110 may include, but is not limited to, any one or more of the following: smartwatch, smartphone, computer, personal digital assistant (PDA), smart voice interaction device, smart home appliance, or in-vehicle terminal. The server 120 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0082] In this embodiment of the application, a communication connection can be established between the terminal device 110 and the server 120 via a wireless network or a wired network. The wireless network or wired network uses standard communication technologies and / or protocols. The network can be set to the Internet or any other network, such as, but not limited to, any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network.

[0083] Of course, this is understandable. Figure 1 The implementation environment described in this application is merely one possible application scenario for the image target detection method provided in this embodiment; the actual application is not fixed. Figure 1 The hardware and software environment is shown below. The following section will combine... Figure 1 The implementation environment shown will be used to provide a detailed description of the image target detection method provided in the embodiments of this application.

[0084] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating an image target detection method provided in an embodiment of this application. (Refer to...) Figure 2 The image target detection method provided in this application includes, but is not limited to:

[0085] Step 210: Obtain the target image to be detected;

[0086] Step 220: Input the target image into the trained image segmentation model, and perform segmentation processing on the target image through the image segmentation model to obtain the first mask image corresponding to the target image;

[0087] Step 230: Detect the target location of each first connected component in the first mask image;

[0088] Step 240: For each first connected component, extract the image content corresponding to the first connected component from the target image according to the target location and replace it in the first mask image to obtain the first sub-image corresponding to each first connected component;

[0089] Step 250: Weight the first subgraph and the target image to obtain the second subgraph corresponding to each first connected component;

[0090] Step 260: Input each of the second subgraphs into the trained metric learning model, and determine the first feature data corresponding to the second subgraph through the metric learning model;

[0091] Step 270: Based on the first feature data, cluster the second sub-graph, merge the first connected components based on the clustering results, and determine the target detection result corresponding to the target image based on the merging results.

[0092] This application provides an image target detection method for detecting target objects in an image. The specific category of the target object is not limited in this application. For example, in some embodiments, the method can be used to detect people in an image; in other embodiments, the method can be used to detect defects or impurities in related products (such as industrial products, agricultural products, etc.) in an image. Of course, the above embodiments are merely illustrative examples and do not imply any limitation on the actual application scenarios of this application.

[0093] In this embodiment of the application, when performing object detection on an image, the image to be detected can be recorded as the target image, the target image can be obtained, and then the target image can be input into a trained image segmentation model. The image segmentation model used to segment the image can use the U-Net architecture, or use image segmentation architectures such as Mask-RCNN, FCN, YOLOv8, SegNet, etc. This application does not limit it.

[0094] The target image is segmented using an image segmentation model to obtain a mask image corresponding to the target image. In this embodiment, this mask image is referred to as the first mask image. For example, please refer to... Figure 3 , Figure 3 This illustration shows a schematic diagram of obtaining a mask image by segmenting an image according to an embodiment of this application. Figure 3The target object to be detected in the target image is impurities. Image segmentation yields a first mask image that roughly shows the location and outline of the impurities. However, due to occlusion of some image content, the first mask image corresponding to the segmented impurities includes many connected regions, which are denoted as first connected regions. In this embodiment, for the first mask image, the location of each first connected region can be detected and denoted as the target location. Then, the mask image portion corresponding to each first connected region can be extracted separately to obtain the mask sub-image corresponding to each first connected region. (Refer to...) Figure 4 , Figure 4 This illustration shows a schematic diagram of a mask subgraph provided in an embodiment of this application. In this embodiment, based on the target position of each first connected region, image content corresponding to the first connected region can be extracted from the target image using the mask subgraph, and this content can be used to replace the first mask graph. Other content is deleted, or it can be considered as directly replacing the image content corresponding to the first connected region in the mask subgraph corresponding to the first connected region. Thus, another subgraph corresponding to each first connected region is obtained, denoted as the first subgraph. (Refer to...) Figure 5 , Figure 5 A schematic diagram of a first sub-figure provided in an embodiment of this application is shown.

[0095] In this embodiment of the application, each first sub-image can be superimposed on the target image, for example, by superimposing them with a pixel value weight of 7:3 to obtain a second sub-image corresponding to each first connected component. (Refer to...) Figure 6 , Figure 6 A schematic diagram of a second sub-diagram provided in an embodiment of this application is shown.

[0096] The obtained second sub-images can be input into a trained metric learning model. The metric learning model determines the feature data corresponding to the second sub-image, which is denoted as the first feature data. In this embodiment, the metric learning model can be built using a CNN (Convolutional Neural Network) architecture. Unlike traditional CNN models, this application does not require a classification layer; only the feature extraction part is needed. (Refer to...) Figure 7 , Figure 7 The illustration shows a schematic diagram of a metric learning model provided in an embodiment of this application. In this embodiment, the metric learning model can use two CNN models, which can share parameters. For each pair of inputs, their feature data (such as vectors) can be extracted. In this embodiment, the CNN models can be replaced with architectures such as VGG, ResNet, DensNet, and Transformer, and they can be used without sharing parameters.

[0097] In this embodiment, after extracting the first feature data, the second sub-graph can be clustered. For example, the K-means clustering algorithm can be used to divide each second sub-graph into similar categories. Then, connected components of the same category can be merged, and the center coordinates of the first connected component after merging each cluster category are output, which are the coordinates of the final target object. The overall contour after merging can be regarded as the contour of the target object, thus obtaining the target detection result. Of course, in this embodiment, other clustering algorithms can also be used, such as hierarchical clustering, DBscan, and other clustering methods.

[0098] Reference Figure 8 , Figure 8 The diagram illustrates an implementation flowchart of an image target detection method provided in this application embodiment. It can be understood that the image target detection method provided in this application embodiment can obtain a first mask image by extracting image segmentation, then analyze and process the first mask image to determine a second sub-graph corresponding to each first connected component; based on the second sub-graph, determine the deep features of the first connected components, cluster the first connected components, and merge the first connected components belonging to related targets, thereby improving the accuracy of target detection and its practicality.

[0099] In some embodiments, the image segmentation model is trained through the following steps:

[0100] Obtain multiple first sample images and a first label corresponding to each first sample image; wherein, the first label is used to indicate the positional contour of the sample object in the first sample image;

[0101] The first sample image is input into the image segmentation model to be trained, and the first sample image is segmented by the image segmentation model to obtain the second mask image corresponding to the first sample image.

[0102] Based on the second mask image and the first label, determine the first loss value for training;

[0103] Based on the first loss value, the parameters of the image segmentation model are updated to obtain a trained image segmentation model.

[0104] In this embodiment, the aforementioned image classification model can be trained before use to optimize its segmentation performance. Specifically, training sets, validation sets, and test sets for image segmentation training can be obtained. In this embodiment, the data in the training set can be denoted as the first sample image, and the data in the test set can be denoted as the second sample image. Their corresponding label data are obtained and denoted as the first label and the second label, respectively. The second sample image can also be used for subsequent training of the metric learning model. Here, the label data can be used to indicate the positional contour information of sample objects in the sample image.

[0105] In this embodiment, when training the image classification model, a first sample image can be input into the image segmentation model for segmentation. The resulting image is denoted as a second mask, which shows the positional contour information of the sample objects predicted by the model in the sample image. Then, the performance of the image segmentation model can be evaluated based on the second mask and the first label, facilitating iterative updates to the model's parameters. Specifically, the training loss value can be determined based on the second mask and the first label; in this embodiment, this loss value is denoted as the first loss value. After obtaining the loss value, backpropagation training can be performed on the image segmentation model to update its internal parameters.

[0106] Specifically, for various models in the field of artificial intelligence, their prediction accuracy can be measured by a loss function. The loss function is defined on a single training data point and measures the prediction error of that data point. Specifically, the loss value of that training data point is determined by its first label and the model's prediction result. However, in actual training, a training dataset contains many data points. Therefore, a cost function is generally used to measure the overall error of the training dataset. The cost function is defined on the entire training dataset and calculates the average prediction error of all training data points, providing a better measure of the model's prediction performance. For general machine learning models, the aforementioned cost function, plus a regularization term to measure model complexity, forms the training objective function. Based on this objective function, the loss value of the entire training dataset can be calculated. Many types of loss functions are commonly used, such as 0-1 loss, squared loss, absolute loss, logarithmic loss, and cross-entropy loss, which will not be elaborated upon here.

[0107] In some embodiments, the present application may also preprocess the acquired first sample image and second sample image, such as performing denoising processing, normalization processing, etc., wherein denoising processing can reduce interference data in the image, and normalization processing can facilitate the unification of the image scale, which can help improve the performance of model processing.

[0108] In some embodiments, the metric learning model is trained through the following steps:

[0109] Obtain multiple second sample images and a second label corresponding to each second sample image; wherein, the second label is used to indicate the positional contour of the sample object in the second sample image;

[0110] The second sample image is input into the trained image segmentation model, and the image segmentation model is used to segment the second sample image to obtain the third mask image corresponding to the second sample image.

[0111] Detect the original position of each second connected component in the third mask image;

[0112] For each second connected component, the image content corresponding to the second connected component is extracted from the second sample image according to the original position and replaced in the second mask image to obtain a third sub-image corresponding to each second connected component;

[0113] The third subgraph and the second sample image are weighted and superimposed to obtain the fourth subgraph corresponding to each second connected component;

[0114] Based on the second label, determine the degree of association between any two of the fourth subgraphs;

[0115] Each of the fourth subgraphs is input into the metric learning model to be trained, and the second feature data corresponding to the fourth subgraph is determined by the metric learning model.

[0116] Calculate the Euclidean distance between any two fourth subgraphs based on the second feature data corresponding to the fourth subgraph.

[0117] The second loss value for training is determined based on the Euclidean distance and the correlation degree.

[0118] Based on the second loss value, the parameters of the metric learning model are updated to obtain the trained metric learning model.

[0119] In this embodiment, the metric learning model can be trained based on a pre-trained image segmentation model. Specifically, in this embodiment, the second sample image can be input into the pre-trained image segmentation model and segmented to obtain a third mask image. Then, based on the third mask image, the fourth sub-image corresponding to the second connected region in each third mask image can be determined. The processing flow for the fourth sub-image is similar to that of the aforementioned second sub-image, and will not be elaborated upon here.

[0120] After obtaining the fourth subgraph, its feature data can be extracted using the metric learning model to be trained, and denoted as the second feature data. Then, the Euclidean distance between them can be calculated based on the second feature data corresponding to the fourth subgraph. In this embodiment, the correlation between any two fourth subgraphs can also be determined based on the second label. For example, if the second label determines that two fourth subgraphs belong to the same sample object, the correlation between the two fourth subgraphs is determined to be 1; if the second label determines that two fourth subgraphs do not belong to the same sample object, the correlation between the two fourth subgraphs is determined to be 0. In this embodiment, the second loss value for training can be determined based on the Euclidean distance and the correlation, and the specific formula can be expressed as:

[0121]

[0122] In the formula, L(Y,X1,X2) represents the second loss value of two second image samples, n represents the index of the second image sample, N is the number of second image samples, Y represents the correlation degree, m is the threshold parameter, and D... 2 This represents the Euclidean distance between two second-image patterns.

[0123] It is understood that in this embodiment, the feature vectors of connected components are extracted through metric learning. Compared to the traditional method of using shallow features such as the shape, pixel mean, and pixel variance of connected components, this method can extract deeper information, improve the accuracy of subsequent merging, and thus improve the precision of object detection. Furthermore, in this embodiment, the data used in training the metric learning model and in actual object detection includes not only the image content within the connected components but also the pixel information of the entire image, which can further improve the object detection effect.

[0124] In this embodiment of the application, an image target detection device is also provided, comprising:

[0125] The acquisition unit is used to acquire the target image to be detected;

[0126] The segmentation unit is used to input the target image into a trained image segmentation model, and to segment the target image through the image segmentation model to obtain a first mask image corresponding to the target image;

[0127] The detection unit is used to detect the target location of each first connected component in the first mask image;

[0128] The replacement unit is configured to, for each first connected component, extract the image content corresponding to the first connected component from the target image according to the target location and replace it in the first mask image to obtain a first sub-image corresponding to each first connected component;

[0129] The overlay unit is used to weight overlay the first subgraph and the target image to obtain a second subgraph corresponding to each first connected component.

[0130] An extraction unit is used to input each of the second subgraphs into a trained metric learning model, and to determine the first feature data corresponding to the second subgraph through the metric learning model;

[0131] The merging unit is used to cluster the second sub-graph according to the first feature data, merge the first connected components according to the clustering results, and determine the target detection result corresponding to the target image according to the merging results.

[0132] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0133] Reference Figure 9 This application provides a computer device, including:

[0134] At least one processor 910;

[0135] At least one memory 920 is used to store at least one program;

[0136] When at least one program is executed by at least one processor 910, the at least one processor 910 enables... Figure 2 This illustrates an image target detection method.

[0137] Similarly, the content of the above method embodiments is applicable to the computer device embodiments. The specific functions implemented by the computer device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0138] This application embodiment also provides a computer-readable storage medium storing a program executable by a processor 910, which, when executed by the processor 910, is used to perform the above-described image target detection method.

[0139] This application also discloses a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement, for example... Figure 2 An embodiment of an image target detection method is shown.

[0140] It is understandable that, such as Figure 2 The content of the image target detection method embodiment shown is applicable to the embodiment of this computer-readable storage medium. The specific functions implemented by the embodiment of this computer-readable storage medium are the same as those shown below. Figure 2 The embodiment of the image target detection method shown is the same, and the beneficial effects achieved are the same as those described above. Figure 2 The beneficial effects achieved by the illustrated image target detection method embodiment are also the same.

[0141] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0142] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical system and / or software module, or one or more functions and / or features may be implemented in a separate physical system or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the system disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0143] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any system that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0145] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections with one or more wires (electronic systems), portable computer disk drives (magnetic systems), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic systems, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0146] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0147] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0148] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0149] The foregoing has provided a detailed description of the preferred embodiments of this application. However, this application is not limited to these embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

[0150] In the description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0151] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An image target detection method, characterized in that, The method includes: Acquire the target image to be detected; The target image is input into a trained image segmentation model, and the target image is segmented by the image segmentation model to obtain a first mask image corresponding to the target image; Detect the target location of each first connected component in the first mask image; For each first connected component, the image content corresponding to the first connected component is extracted from the target image according to the target location and replaced in the first mask image to obtain the first sub-image corresponding to each first connected component; The first subgraph and the target image are weighted and superimposed to obtain the second subgraph corresponding to each first connected component; Each of the second subgraphs is input into the trained metric learning model, and the first feature data corresponding to the second subgraph is determined through the metric learning model; Based on the first feature data, the second sub-image is clustered, the first connected components are merged based on the clustering results, and the target detection result corresponding to the target image is determined based on the merging results. The image segmentation model is trained through the following steps: Obtain multiple first sample images and a first label corresponding to each first sample image; wherein, the first label is used to indicate the positional contour of the sample object in the first sample image; The first sample image is input into the image segmentation model to be trained, and the first sample image is segmented by the image segmentation model to obtain the second mask image corresponding to the first sample image. Based on the second mask image and the first label, determine the first loss value for training; Based on the first loss value, the parameters of the image segmentation model are updated to obtain the trained image segmentation model; The metric learning model is trained through the following steps: Obtain multiple second sample images and a second label corresponding to each second sample image; wherein, the second label is used to indicate the positional contour of the sample object in the second sample image; The second sample image is input into the trained image segmentation model, and the image segmentation model is used to segment the second sample image to obtain the third mask image corresponding to the second sample image. Detect the original position of each second connected component in the third mask image; For each second connected component, the image content corresponding to the second connected component is extracted from the second sample image according to the original position and replaced in the second mask image to obtain a third sub-image corresponding to each second connected component; The third subgraph and the second sample image are weighted and superimposed to obtain the fourth subgraph corresponding to each second connected component; Based on the second label, determine the degree of association between any two of the fourth subgraphs; Each of the fourth subgraphs is input into the metric learning model to be trained, and the second feature data corresponding to the fourth subgraph is determined by the metric learning model. Calculate the Euclidean distance between any two fourth subgraphs based on the second feature data corresponding to the fourth subgraph. The second loss value for training is determined based on the Euclidean distance and the correlation degree. Based on the second loss value, the parameters of the metric learning model are updated to obtain the trained metric learning model.

2. The image target detection method according to claim 1, characterized in that, After the step of acquiring multiple first sample images, the method further includes: The first sample image is preprocessed, and the preprocessing operation includes at least one of denoising and normalization.

3. The image target detection method according to claim 1, characterized in that, The step of determining the correlation degree between any two of the fourth subgraphs based on the second label includes: If the second label determines that the two fourth subgraphs belong to the same sample object, the correlation between the two fourth subgraphs is determined to be 1; or, if the second label determines that the two fourth subgraphs do not belong to the same sample object, the correlation between the two fourth subgraphs is determined to be 0.

4. The image target detection method according to claim 1, characterized in that, Determining the first loss value for training based on the second mask image and the first label includes: Based on the second mask image and the first label, the first loss value for training is determined using the cross-entropy loss function.

5. The image target detection method according to claim 1, characterized in that, The clustering of the second subgraph includes: The second subgraph is clustered using the K-means clustering algorithm.

6. An image target detection device, characterized in that, The device includes: The acquisition unit is used to acquire the target image to be detected; The segmentation unit is used to input the target image into a trained image segmentation model, and to segment the target image through the image segmentation model to obtain a first mask image corresponding to the target image; The detection unit is used to detect the target location of each first connected component in the first mask image; The replacement unit is configured to, for each first connected component, extract the image content corresponding to the first connected component from the target image according to the target location and replace it in the first mask image to obtain a first sub-image corresponding to each first connected component; The overlay unit is used to weight overlay the first subgraph and the target image to obtain a second subgraph corresponding to each first connected component. An extraction unit is used to input each of the second subgraphs into a trained metric learning model, and to determine the first feature data corresponding to the second subgraph through the metric learning model; The merging unit is configured to cluster the second sub-graph according to the first feature data, merge the first connected components according to the clustering results, and determine the target detection result corresponding to the target image according to the merging results. The image segmentation model is trained through the following steps: Obtain multiple first sample images and a first label corresponding to each first sample image; wherein, the first label is used to indicate the positional contour of the sample object in the first sample image; The first sample image is input into the image segmentation model to be trained, and the first sample image is segmented by the image segmentation model to obtain the second mask image corresponding to the first sample image. Based on the second mask image and the first label, determine the first loss value for training; Based on the first loss value, the parameters of the image segmentation model are updated to obtain the trained image segmentation model; The metric learning model is trained through the following steps: Obtain multiple second sample images and a second label corresponding to each second sample image; wherein, the second label is used to indicate the positional contour of the sample object in the second sample image; The second sample image is input into the trained image segmentation model, and the image segmentation model is used to segment the second sample image to obtain the third mask image corresponding to the second sample image. Detect the original position of each second connected component in the third mask image; For each second connected component, the image content corresponding to the second connected component is extracted from the second sample image according to the original position and replaced in the second mask image to obtain a third sub-image corresponding to each second connected component; The third subgraph and the second sample image are weighted and superimposed to obtain the fourth subgraph corresponding to each second connected component; Based on the second label, determine the degree of association between any two of the fourth subgraphs; Each of the fourth subgraphs is input into the metric learning model to be trained, and the second feature data corresponding to the fourth subgraph is determined by the metric learning model. Calculate the Euclidean distance between any two fourth subgraphs based on the second feature data corresponding to the fourth subgraph. The second loss value for training is determined based on the Euclidean distance and the correlation degree. Based on the second loss value, the parameters of the metric learning model are updated to obtain the trained metric learning model.

7. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements an image target detection method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a processor-executable program, characterized in that: The processor-executable program, when executed by the processor, is used to implement an image target detection method as described in any one of claims 1-5.

Images