Target detection method, device, computer equipment and storage medium

Abstract

The embodiments of the present application disclose a target detection method, apparatus, computer equipment and storage medium; the embodiments of the present application can obtain a training sample image of a target detection model and the attributes of a sample target area in the training sample image; perform index generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area; add an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with an interpretability label; train the target detection model based on the training sample image with the interpretability label to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected, obtain the target area of ​​the image to be detected and the interpretability index of the target area, thereby improving the accuracy and reliability of target detection on the image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a target detection method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the gradual increase in computer costs and computing power and the development of semiconductor technology, artificial intelligence technology has also developed rapidly. With the rapid development of artificial intelligence technology, target detection algorithms have also developed rapidly.

[0003] Object detection algorithms are image segmentation techniques based on geometric and statistical features. Their purpose is to detect objects in an image and their locations within the image. Object detection algorithms based on artificial intelligence (AI) technology can segment images using AI models and, based on the segmentation results, identify objects and their locations within the image. However, existing AI-based object detection algorithms often lack interpretability, which reduces the accuracy and reliability of object detection in images. Summary of the Invention

[0004] The embodiments of the present application provide a target detection method, apparatus, computer device, and storage medium, which improve the accuracy and reliability of target detection in images.

[0005] The present invention provides a method for detecting a target, including:

[0006] Obtaining a training sample image of a target detection model and attributes of a sample target area in the training sample image;

[0007] Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0008] Adding an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with the interpretability label;

[0009] The target detection model is trained based on the training sample image with interpretable labels to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area.

[0010] Accordingly, an embodiment of the present application further provides a target detection device, comprising:

[0011] An acquisition unit, configured to acquire a training sample image of a target detection model and attributes of a sample target area in the training sample image;

[0012] an index generating unit, configured to perform index generating processing on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0013] an adding unit, configured to add an interpretability label to the sample target area according to the interpretability index, to obtain a training sample image with the interpretability label;

[0014] A training unit is used to train the target detection model based on the training sample image with interpretable labels to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area.

[0015] In one embodiment, the indicator generation unit includes:

[0016] an identification subunit, configured to identify the attribute and obtain the type of the attribute;

[0017] a determination subunit, configured to determine, according to the type of the attribute, a method for generating an indicator corresponding to the attribute;

[0018] The indicator generation subunit is used to perform indicator generation processing on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area.

[0019] In one embodiment, the indicator generation subunit includes:

[0020] An acquisition module, configured to acquire the size attribute of the training sample image;

[0021] a comparison module, configured to compare the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image;

[0022] The normalization processing module is used to perform normalization processing on the image size comparison information to obtain the size interpretability index.

[0023] In one embodiment, the indicator generation subunit includes:

[0024] An evaluation module, configured to evaluate the clarity attribute of the sample target area to obtain clarity evaluation information of the sample target area;

[0025] A comparison module, configured to compare the clarity evaluation information with a preset clarity judgment threshold, and generate comparison information based on the comparison result;

[0026] The linear operation module is used to perform linear operation processing on the contrast information to obtain the clarity interpretability index.

[0027] In one embodiment, the adding unit includes:

[0028] an adjusting subunit, configured to adjust the interpretability index to obtain an adjusted interpretability index;

[0029] a label mapping subunit, configured to perform label mapping processing on the adjusted interpretability index to obtain the interpretability label;

[0030] The association subunit is used to associate the explainability label with the sample target area to obtain the training sample image with the explainability label.

[0031] In one embodiment, the training unit includes:

[0032] an object detection subunit, configured to perform object detection on the training sample image with the interpretable label using the object detection model to obtain an initial detection area of ​​the training sample image;

[0033] an acquisition subunit, configured to acquire a detection probability of the initial detection area;

[0034] a calculation subunit, configured to calculate loss information between the detection probability and the interpretability label;

[0035] An adjustment subunit is used to adjust the target detection model using the loss information to obtain the trained target detection model.

[0036] In one embodiment, the computing subunit includes:

[0037] a comparison module, configured to compare the explanatory label with a preset threshold to obtain a comparison result;

[0038] A determination module, configured to determine a loss information calculation method corresponding to the interpretability label according to the comparison result;

[0039] A calculation module is used to calculate the loss information between the detection probability and the explainability label according to the loss information calculation method.

[0040] In one embodiment, the target detection subunit includes:

[0041] A convolution module, configured to perform convolution processing on the training sample image to obtain convolution information of the training sample image;

[0042] A feature extraction module, configured to extract features from the convolution information to obtain feature information of the training sample image;

[0043] The classification prediction module is used to perform classification prediction on the training sample image based on the feature information of the training sample image to obtain the initial detection area.

[0044] The present application also provides a sample generation method, including:

[0045] Acquire a training sample image and attributes of a sample target area in the training sample image;

[0046] Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0047] An interpretability label is added to the sample target area according to the interpretability index, and a training sample image with the interpretability label is output.

[0048] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in various optional embodiments of the above-mentioned aspect.

[0049] Correspondingly, an embodiment of the present application further provides a storage medium, which stores instructions, and when the instructions are executed by a processor, implements any target detection method provided in the embodiments of the present application.

[0050] The embodiments of the present application can obtain the training sample images of the target detection model and the attributes of the sample target area in the training sample images; perform index generation processing on the attributes of the sample target area to obtain the interpretability index of the sample target area; add interpretability labels to the sample target area according to the interpretability index to obtain the training sample images with interpretability labels; train the target detection model based on the training sample images with interpretability labels to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected, obtain the target area of ​​the image to be detected and the interpretability index of the target area, thereby improving the accuracy and reliability of target detection on the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 Schematic diagram of a target detection method according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the target detection method provided in the embodiment of the present application;

[0054] Figure 3 This is a schematic diagram of the structure of a one-stage intensive detection algorithm provided in an embodiment of the present application;

[0055] Figure 4 This is another structural diagram of the one-stage intensive detection algorithm provided in an embodiment of the present application;

[0056] Figure 5 This is a schematic diagram of a confidence scenario provided by an embodiment of the present application;

[0057] Figure 6 This is another scenario diagram of the confidence provided in the embodiment of the present application;

[0058] Figure 7 is a schematic diagram of a scene of a sample target area provided in an embodiment of the present application;

[0059] Figure 8 This is another scenario diagram of the confidence provided in the embodiment of the present application;

[0060] Figure 9 This is another scenario diagram of the confidence provided in the embodiment of the present application;

[0061] Figure 10 This is another flowchart of the target detection method provided in the embodiment of the present application;

[0062] Figure 11 This is another flowchart of the target detection method provided in the embodiment of the present application;

[0063] Figure 12 This is another flow chart of the confidence level provided in the embodiment of the present application;

[0064] Figure 13 This is another flow chart of the confidence level provided in the embodiment of the present application;

[0065] Figure 14 This is a flow chart of a sample generation method provided in an embodiment of the present application;

[0066] Figure 15 is a schematic structural diagram of a target detection device provided in an embodiment of the present application;

[0067] Figure 16 is a structural diagram of a sample generation device provided in an embodiment of the present application;

[0068] Figure 17 It is a schematic diagram of the structure of the terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. However, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0070] The embodiments of the present application provide a target detection method. The target detection method can be performed by a target detection device, which can be integrated into a computer device. The computer device can include at least one of a terminal and a server. In other words, the target detection method provided in the embodiments of the present application can be performed by a terminal, a server, or a terminal and a server that can communicate with each other.

[0071] The terminals may include smartphones, tablets, laptops, personal computers (PCs), smart homes, wearable electronic devices, VR / AR devices, in-vehicle computers, etc. The servers may be intercommunication servers or backend servers between multiple heterogeneous systems, independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, blockchain networks, and big data and artificial intelligence platforms.

[0072] In one embodiment, if Figure 1The target detection device can be integrated into a computer device such as a terminal or a server to implement the target detection method proposed in the embodiment of the present application. Specifically, the computer device can obtain the training sample image of the target detection model and the attributes of the sample target area in the training sample image; perform index generation processing on the attributes of the sample target area to obtain the interpretability index of the sample target area; add an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with an interpretability label; train the target detection model based on the training sample image with the interpretability label to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area.

[0073] The following are detailed descriptions of each embodiment. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.

[0074] The embodiments of the present application will be described from the perspective of a target detection device, which can be integrated into a computer device, which can be a server, a terminal, or other device.

[0075] like Figure 2 A target detection method is provided, and the specific process includes:

[0076] 101. Obtain a training sample image of a target detection model and attributes of a sample target area in the training sample image.

[0077] Object detection models include models that can detect objects in images, but whose object detection performance does not yet meet the requirements, so they are trained. Object detection can refer to the technology used to identify objects in an image and their locations. For example, object detection can be used to detect whether there is a face in an image and, if so, where the face is located in the image. Another example is the use of object detection to detect whether there are animals in an image and, if so, where the animals are located in the image.

[0078] In one embodiment, the target detection model can be a machine learning model. Among them, machine learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0079] For example, the target detection model can be one of the following: Region-based Convolutional Networks (RCNN), Faster Region-based Convolutional Networks (Faster RCNN), Single Shot MultiBox Detector (SSD), Fully Convolutional One-Stage Object Detection (FCOS), etc.

[0080] In one embodiment, FCOS is a pixel-by-pixel object detection algorithm based on a fully convolutional network. This algorithm implements an anchor-free and proposal-free solution and incorporates the concept of centerness. Its performance in terms of recall rate approaches or even exceeds that of many leading mainstream anchor-based object detection algorithms. Therefore, when using FCOS for object detection in images, it can improve detection efficiency while ensuring accuracy.

[0081] In one embodiment, the model structure diagram of FCOS can be as follows: Figure 3 As shown in the figure, it includes the backbone layer (Backbone), feature pyramid (Feature Pyramid) and prediction layer (Head). Among them, the Head can include classification branch (Classificaton), centerness (Centerness) and regression branch (Regression).

[0082] Among them, Backbone can be a Convolutional Neural Networks (CNN) for performing convolution processing on images. Figure 4 As shown, Backbone can be a three-layer convolutional neural network, where the three layers correspond to Figure 4 Through C3, C4 and C5, we can get the convolution information of the image in different dimensions.

[0083] Among them, Feature Pyramind is used to construct the feature information of the image at different scales. It can extract the features of the convolution information of the image at different dimensions, thereby obtaining the feature map of the image at different scales. Among them, the feature map includes the feature information of the image. For example, Figure 4 As shown in Figure 1, Feature Pyramind can extract features from C3, C4, and C5, respectively, to obtain P3, P4, and P5. Feature Pyramind can then extract features from P5 to obtain P6, and from P6 to obtain P7. P3, P4, P5, P6, and P7 are all feature maps of the image at different scales.

[0084] Next, the Head predicts the image based on its feature information at different scales, thereby detecting the objects present and their locations. Specifically, the Head predicts the object category for each pixel in the image and generates a detection area for that object category, thereby determining the location of that object category in the image. The Classification function in the Head converts the position of each pixel on the feature map into its position in the image, establishing a relationship between the pixel position on the feature map and the position of the point in the input image. Centerness then calculates the distance between each pixel in the feature map and the target pixel, thereby distinguishing pixels that require prediction from those that do not. For example, if the distance between a pixel and the target pixel is far, it is classified as a pixel that does not require prediction. However, if the distance between the pixel and the target pixel is close, it is classified as a pixel that requires prediction. Next, Regression predicts the pixels that require prediction, thereby detecting the objects present in the image and their locations.

[0085] In one embodiment, training the model may include a process of enabling the model to learn from massive amounts of data, thereby enabling the model to summarize patterns from the massive amounts of data and process any data input into the model based on the patterns.

[0086] The training sample images include training data used to train the target detection model.

[0087] In one embodiment, training sample images may be obtained based on the task of the object detection model. For example, when the task of the object detection model is to detect human faces in an image, images containing human faces may be obtained as training sample images. For another example, when the task of the object detection model is to detect animals in an image, images containing animals may be obtained as training sample images, and so on.

[0088] In one embodiment, in order to ensure the quality of training the target detection model, multiple training sample images are often obtained, and the multiple training sample images are used to form a training set, and the target detection model is trained using the training set.

[0089] In one embodiment, the method of training the model can be divided into a training method based on supervised learning and a training method based on unsupervised learning.

[0090] Among them, supervised learning can include the process of adjusting the parameters in the model using a set of samples of known categories so that the model achieves the required performance. Supervised learning is a machine learning task that infers a function from labeled training data. The training data includes a set of training examples. In supervised learning, each instance consists of an input object (usually a vector) and a desired output value (also called a supervisory signal). The supervised learning algorithm analyzes the training data and generates an inferred function that can be used to map new instances.

[0091] In one embodiment, the object detection model can be trained using a supervised learning training method. Therefore, labels can be added to the training sample images so that during the object detection model training process, the object detection model can adjust its performance based on the labels, so that the object detection model can accurately identify the target area in the image.

[0092] Typically, existing technologies only add classification labels to training sample images when training object detection models. Classification labels include labels that describe the objects in the training sample images. For example, if the object detection model's task is to detect cats and dogs in an image, the training sample images may include images of both cats and dogs. Developers can then add classification labels to these training sample images, indicating which objects are cats and which are dogs. These labeled training sample images are then used to train the object detection model, resulting in a trained object detection model.

[0093] However, since the trained target detection model only has classification labels as the basis for performance adjustment during the training process, it is easy for the trained target detection model to have low confidence in target detection.

[0094] Confidence, also known as reliability, confidence level, or confidence coefficient, is used to estimate a population parameter using sampling. Due to the randomness of the sample, the conclusion is always uncertain. Therefore, a probability statement method, known as the interval estimation method in mathematical statistics, is used. The probability that the estimated value is within a certain allowable error range relative to the population parameter is determined. This probability is called the confidence level.

[0095] For example, when an object detection model is tasked with detecting cats and dogs in an image, it uses probability theory and statistics to determine the probability of each pixel being an object. For example, if the object detection model detects a region in an image with a 90% probability of being a dog and a 20% probability of being a cat, the model will classify the object in that region as a dog.

[0096] In one embodiment, when performing model inference on an existing target detection model, the interpretability of the confidence of the output detection results is generally low.

[0097] For example, when using existing object detection models to detect objects in images, it is often easy to output high confidence for small objects and low confidence for large objects. Figure 5 As shown in , the task of the target detection model is to detect the giraffe in the image. Figure 5 The confidence level for the largest giraffe is only 0.59, while the confidence level for the largest giraffe is 0.7.

[0098] For example, when using existing object detection models to detect objects in images, it is often easy to output high confidence for objects with lower clarity, and low confidence for objects with higher clarity. Figure 6 As shown in the figure, the task of the target detection model is to detect humans and footballs in the image. In the figure, the confidence score of the human with lower clarity is 0.75, while the confidence score of the human with higher clarity is 0.8.

[0099] Explainability can include explanations of the model’s behavior. For example, confidence explainability can include explanations of the model’s confidence in the detection results output.

[0100] In one embodiment, the low interpretability of the confidence level of the output detection results can affect the performance of the target detection model, thereby reducing the accuracy and reliability of the target detection model. Therefore, in the embodiment of the present application, not only classification labels are added to the training sample images, but also interpretability labels are added to the training sample images, so that the training sample images can adjust the performance of the model based on the classification labels and interpretability labels, thereby improving the accuracy and reliability of the trained target detection model.

[0101] In one embodiment, attributes of a sample target region in a training sample image can be obtained, and the attributes of the sample target region can be evaluated to obtain an interpretability index of the sample target region. Then, an explanatory label is added to the sample target region based on the interpretability index to obtain a training sample image with the explanatory label.

[0102] The sample target area includes a location area formed by a detection object in the sample training image during target detection.

[0103] In one embodiment, the sample target area can have various forms. For example, the sample target area can be any polygon. For example, the sample target area can be a rectangle, a triangle, a pentagon, etc.

[0104] In one embodiment, since the target detection model often has more than one detection task, there may be more than one sample target region in the training sample image. For example, when the target detection task is to detect cats and dogs in an image, the sample target region may be as follows: Figure 7 As shown, one sample target region may indicate that the selected region is a cat, while another sample target region may include that the selected region is a dog.

[0105] The attributes may include properties of the image in terms of pixels, clarity, size, color, etc. For example, the attributes may include pixels, size, dimensions, color, bit depth, hue, saturation, brightness, and color channels of the image.

[0106] The pixels of an image may include the number of pixels in the width and length directions of the image.

[0107] Image clarity can also refer to image resolution. Resolution refers to the number of image pixels per unit of printed length, indicating the amount or density of digital information in the image, and determines the image's clarity. For the same area, higher image resolution means more pixels make up the image, and more pixels means higher image clarity.

[0108] The size of the image may include the length and width of the image.

[0109] The bit depth of an image, also known as image bit depth, refers to the number of bits of data describing each pixel in the image. Each pixel in an image typically contains one or more bytes of data, which store information such as the pixel's color and brightness. The more bits of data, the more color variations the image can produce.

[0110] Among them, the saturation of an image may include the depth of the image color, which indicates the purity of the color and is determined by the reflection or projection characteristics of the object.

[0111] The brightness of an image refers to the degree of brightness or darkness of the image color, which is the human eye's perception of the intensity of light and dark of an object.

[0112] The color channels of an image may be storage spaces that store the color components of the image. For example, the red (R) color channel may be a channel that stores red color information; the green (G) color channel may be a channel that stores green color channel values; and the blue (B) color channel may be a channel that stores blue color channel values. Each type of color channel may be an 8-bit storage space, with each bit storing a corresponding color channel value.

[0113] In one embodiment, since the attributes include image properties in terms of pixels, definition, size, color, etc., each aspect of the image property can be classified as a type of attribute.

[0114] For example, the pixels of an image can be regarded as a type of attribute and called pixel attributes; the clarity of an image can also be regarded as a type of attribute and called clarity attributes; the size of an image can also be regarded as a type of attribute and called size attributes; the color of an image can also be regarded as a type of attribute and called color attributes, and so on.

[0115] In one embodiment, the target detection method proposed in the embodiment of the present application can be applied to various scenarios requiring target detection technology. For example, face recognition, pedestrian detection, and large-scale scene recognition, etc. Depending on the application scenario, the target detection device proposed in the embodiment of the present application can be integrated into various different computer devices for use. For example, when the target detection method proposed in the embodiment of the present application is applied to the application scenario of face recognition, the target detection device proposed in the embodiment of the present application can be integrated into various face recognition terminals.

[0116] 102. Perform index generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area.

[0117] In one embodiment, the interpretability of an image can often be reflected through its attributes. For example, regarding the interpretability of image clarity, images with higher clarity generally have more pixels, while images with lower clarity generally have fewer pixels. Therefore, when adding an interpretability label to a sample target area, an index generation process can be performed on the image of the sample target area to obtain an interpretability index for the sample target area, and then an interpretability label can be added to the sample target area based on the interpretability index.

[0118] Among them, the interpretability index includes measurement parameters for the attributes in the sample target area. In one embodiment, the interpretability index can be equivalent to a "score" assigned to the attributes in the sample target area based on the attributes of the sample target area. For example, when the interpretability index is for the clarity of the sample target area, the interpretability index can be used to know the clarity of the sample target area. For another example, when the interpretability index is for the size of the sample target area, the interpretability index can be used to know the size of the size in the sample target area. For another example, when the interpretability index is for the brightness of the sample target area, the interpretability index can be used to know the brightness of the size in the sample target area.

[0119] In one embodiment, since attributes have multiple types, different types of attributes may correspond to one interpretability index. For example, a size attribute may correspond to a size interpretability index. For another example, a clarity attribute may correspond to a clarity interpretability index. For another example, a pixel attribute may correspond to a pixel interpretability index, and so on.

[0120] In one embodiment, since attributes have multiple types, and different types of attributes may require different evaluation and processing methods, when evaluating the attributes of the sample target area, the attributes can be identified to obtain the attribute type, and then the attributes can be evaluated based on the attribute type. Specifically, the step of "generating an indicator for the attributes of the sample target area to obtain an interpretability indicator for the sample target area" may include:

[0121] Identify the attribute and obtain the attribute type;

[0122] According to the type of attribute, determine the indicator generation method corresponding to the attribute;

[0123] The attributes of the sample target area are processed by the indicator generation method to obtain the interpretability index of the sample target area.

[0124] In one embodiment, since attributes have multiple types, the index generation methods corresponding to different types of attributes may be different.

[0125] In one embodiment, when the attribute type is a size attribute, the corresponding interpretability indicator may be a size interpretability indicator. In this case, the step of "performing an indicator generation process on the attribute of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area" may include:

[0126] Get the size attributes of the training sample image;

[0127] Comparing the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image;

[0128] The image size comparison information is normalized to obtain the size interpretability index.

[0129] In one embodiment, the comparison process may include comparing the size attributes of the sample target area with the size attributes of the training sample images. For example, the size attributes of the target area and the size attributes of the training sample images may be modulo each other to obtain image size comparison information. For another example, the size attributes of the target area and the size attributes of the training sample images may be subtracted to obtain image size comparison information. For another example, the size attributes of the target area and the size attributes of the training sample images may be subtracted and then modulo each other to obtain image size comparison information.

[0130] For example, the size attribute of an image may include the length and width of the image. Therefore, in one embodiment, the size attribute of the training sample image may be represented as H img and W img Among them, H img It can represent the length of the training sample image, W img It can represent the width of the training sample image. Similarly, the size attribute of the sample target area can be expressed as H gt and W gt Among them, H gt It can represent the length of the sample target area, W gt It can represent the width of the sample target area.

[0131] In one embodiment, when the comparison process refers to performing a modulus calculation on the size attribute of the target area and the size attribute of the training sample image, the image size comparison information may be expressed as follows:

[0132]

[0133] The symbol A may represent image size information.

[0134] In one embodiment, when the comparison process refers to subtracting the size attribute of the target area from the size attribute of the training sample image, the image size comparison information may be expressed as the following formula:

[0135] A=(H img -H gt )+(W img -W gt )

[0136] In one embodiment, the normalization process may include mapping the image size comparison information to between 0 and 1, so that the size interpretation index is within the range of 0 and 1. For example, the normalization process may include a square root operation or a division operation, etc.

[0137] For example, when the image size comparison information is processed by calculating the modulus of the size attribute of the target area and the size attribute of the training sample image, the image size comparison information can be squared to obtain the size interpretability index. Specifically, it can be shown as the following formula:

[0138]

[0139] Among them, the symbol Size score Can represent size explanatory indicators.

[0140] For another example, when the image size comparison information is the difference between the size attribute of the target area and the size attribute of the training sample image, the image size comparison information can be used to perform a starting operation to obtain a size interpretability index. Specifically, it can be shown in the following formula:

[0141]

[0142] In one embodiment, the interpretability index may include a clarity interpretability index, and the attribute type may include a clarity attribute. The step of “performing an indicator generation process on the attribute of the sample target area according to the indicator generation method to obtain an interpretability index of the sample target area” may include:

[0143] Evaluating and processing the clarity attribute of the sample target area to obtain clarity evaluation information of the sample target area;

[0144] Comparing the clarity evaluation information with a preset clarity judgment threshold, and generating comparison information based on the comparison result;

[0145] The contrast information is processed by linear operations to obtain the clarity interpretability index.

[0146] The evaluation process may include assigning a corresponding score value to the current clarity attribute of the sample target area based on the clarity attribute. For example, when the clarity of the sample target area is high, a higher score value may be assigned to the sample target area. For another example, when the clarity of the sample target area is low, a lower score value may be assigned to the sample target area. The score value may be clarity evaluation information.

[0147] In one embodiment, the clarity of the target area of ​​the sample can be known through the clarity evaluation information.

[0148] In one embodiment, the sharpness attribute of the sample target area can be evaluated using various methods. For example, the number of pixels in the sample target area can be identified, and sharpness evaluation information can be generated based on the number of pixels. In another example, the cv2.Laplacian() function in the open source computer vision library (openCV) can be used to evaluate the sharpness attribute of the sample area, thereby obtaining sharpness evaluation information.

[0149] Among them, cv2.Laplacian() can measure the variance of image gradients. The higher the variance, the higher the image clarity, and the lower the variance, the lower the image clarity.

[0150] Among them, openCV is a cross-platform computer vision and machine learning software library that can run on multiple operating systems. It also provides interfaces for multiple programming languages ​​and implements many common algorithms in the fields of image processing and computer vision.

[0151] In one embodiment, after obtaining the clarity evaluation information, the clarity evaluation information may be compared with a preset clarity evaluation threshold, and comparison information may be generated according to the comparison result.

[0152] For example, the clarity assessment information can be compared with a preset clarity assessment threshold, and the smaller value can be selected as the comparison information. For example, when the clarity assessment information is less than or equal to the preset clarity assessment threshold, the clarity assessment information can be used as the comparison information. For another example, when the clarity assessment information is greater than the preset clarity assessment threshold, the preset clarity assessment threshold can be used as the comparison information.

[0153] In one embodiment, when the clarity attribute of the sample area is evaluated using the cv2.Laplacian() function to obtain clarity evaluation information, and the clarity evaluation information is compared with a preset clarity evaluation threshold, the following formula can be used:

[0154] D=min(cv2.Laplacian(box)+k,z)

[0155] The symbol box may represent a clarity attribute, the symbol D may represent contrast information, the symbol z may represent a preset clarity evaluation threshold, and the symbol k may represent a bias term.

[0156] In one embodiment, the value of k can be 1, and the value of z can be 3. 10 .

[0157] In one embodiment, after obtaining the contrast information, linear operation processing may be performed on the contrast information to obtain a clarity interpretability index.

[0158] Among them, linear operations may include multiplication, addition, logarithm and division, etc.

[0159] For example, performing linear operations on contrast information to obtain the clarity interpretability index can be expressed as follows:

[0160] Blur score =log2(D) / m

[0161] Among them, the symbol Blur score It can represent the clarity interpretability index. The symbol m can be any non-zero value. For example, m can be set to 18.

[0162] 103. Add interpretable labels to the sample target area according to the interpretable index to obtain a training sample image with interpretable labels.

[0163] In one embodiment, after obtaining the interpretability index of the sample target region, an interpretability label may be added to the sample target region according to the interpretability index, thereby obtaining a training sample image with the interpretability label.

[0164] Among them, the explainability label includes a label generated according to the explainability index, and the label has a basis and explainability.

[0165] In one embodiment, the interpretable label may be a soft label.

[0166] In one embodiment, after obtaining the interpretability index, in order to improve the reliability of the interpretability index, the interpretability index can be adjusted to obtain an adjusted interpretability index. Then, an interpretable label is added to the sample target area based on the interpretability index. Specifically, the step of "adding interpretable labels to the sample target area based on the interpretability index to obtain a training sample image with interpretable labels" may include:

[0167] Adjust the interpretability index to obtain the adjusted interpretability index;

[0168] Perform label mapping on the adjusted interpretability index to obtain interpretability labels;

[0169] The interpretable label is associated with the sample target area to obtain a training sample image with the interpretable label.

[0170] Among them, adjusting the interpretability index may include adding a bias item to the interpretability label or padding the interpretability index, etc., so as to obtain the adjusted interpretability index.

[0171] For example, the interpretability index can be expressed as Score. Then, when the interpretability index is adjusted by adding a bias term to the interpretability index, it can be as follows:

[0172] T=Score+e

[0173] Among them, the symbol T can represent the adjusted interpretability index, and the symbol e can represent the bias term.

[0174] The bias term includes a value that can adjust the reliability of the interpretability indicator. In one embodiment, the value of the bias term is generally a decimal, which is used to fine-tune the interpretability indicator, thereby improving the reliability of the interpretability indicator.

[0175] In one embodiment, after obtaining the adjusted interpretability index, the adjusted interpretability index can be subjected to label mapping processing to obtain an interpretability label. For example, the adjusted interpretability index can be matched with a preset label mapping threshold to obtain an interpretability label. For another example, the adjusted interpretability index can be compared with a preset label mapping threshold, and an interpretability label can be generated based on the comparison result. For example, when comparing the adjusted interpretability index with a preset label mapping threshold and generating an interpretability label based on the comparison result, the following formula can be used:

[0176] label smooth =max(T,b)

[0177] Among them, the symbol label smooth The symbol b can represent an interpretability label. The symbol b can represent a preset label mapping threshold, where the preset label mapping threshold can be set to 1. In the above formula, the interpretability index and the preset label mapping threshold can be matched. When the interpretability index is greater than the preset label mapping threshold, the interpretability index can be used as the interpretability label; and when the interpretability index is less than the preset label mapping threshold, the preset label mapping threshold can be used as the interpretability label.

[0178] For example, you can also generate explainable labels according to the following formula:

[0179] label smooth =min(T,b)

[0180] In one embodiment, after obtaining the interpretable label, the interpretable label and the sample target area may be associated to obtain a training sample image with the interpretable label.

[0181] For example, the same identification information can be added to both the interpretable label and the sample target region, thereby associating the interpretable label with the sample target region. By associating the interpretable label with the sample target region, the object detection model can be trained using training sample images with interpretable labels, thereby improving the interpretability of the detection results of the object detection model and thus improving the reliability of the object detection model.

[0182] 104. Train the target detection model based on the training sample images with interpretable labels to obtain a trained target detection model.

[0183] In one embodiment, after obtaining training sample images with interpretable labels, the object detection model can be trained using the training sample images with interpretable labels to obtain a trained object detection model. Specifically, the step of "training the object detection model based on the training sample images with interpretable labels to obtain a trained object detection model" includes:

[0184] Use the target detection model to perform target detection on the training sample image with interpretable labels to obtain the initial detection area of ​​the training sample image;

[0185] Get the detection probability of the initial detection area;

[0186] Calculate the loss between detection probability and interpretable labels;

[0187] The target detection model is adjusted using the loss information to obtain the trained target detection model.

[0188] The initial detection area can be the location area of ​​the detection object in the sample training image. The difference between the sample target area and the initial detection area is that the sample target area is set manually, while the initial detection area is generated by the target detection model.

[0189] The detection probability may refer to the probability that the target detection model detects the initial detection area. In one embodiment, the detection probability may be the confidence level of the initial detection probability.

[0190] The loss information includes information that measures the degree of difference between the detection probability and the interpretable label.

[0191] The trained target detection model may include a trained model whose performance in performing target detection on an image meets preset conditions.

[0192] In one embodiment, the process of performing target detection on a training sample image with an interpretable label using a target detection model may be a process of inputting the training sample image with an interpretable label into the target detection model and outputting an initial detection area using the network structure of the target detection model.

[0193] For example, when the object detection model is FCOS, a training sample image can be input into FCOS. The Backbone in FCOS then first performs convolution on the training sample image, obtaining convolution information of the training sample image at different dimensions. The Feature Pyramid then extracts features from the convolution information of the training sample image at different dimensions, obtaining feature maps of the training sample image at different scales. Next, the Head predicts the training sample image based on the feature information at different scales and the interpretable labels, thereby obtaining the initial detection region of the training sample image.

[0194] In one embodiment, when the object detection model is FCOS, the step of “using the object detection model to perform object detection on the training sample image with interpretable labels to obtain an initial detection region of the training sample image” may include:

[0195] Perform convolution processing on the training sample image to obtain convolution information of the training sample image;

[0196] Perform feature extraction on the convolution information to obtain feature information of the training sample image;

[0197] Based on the feature information of the training sample image, the training sample image is classified and predicted to obtain the initial detection area.

[0198] For example, the Backbone in FCOS can be used to perform convolution processing on the training sample image to obtain the convolution information of the training sample image. Then, the Feature Pyramind in FCOS can be used to extract features from the convolution information to obtain the feature information of the training sample image. Finally, the Head in FCOS can be used to perform classification prediction on the training sample image to obtain the initial detection area. In one embodiment, since the detection process of the target detection model is based on the knowledge of probability theory and statistics to judge the probability of what object each pixel may be. Therefore, after obtaining the initial detection area of ​​the training sample image, the detection probability output by the target detection model can also be obtained. Next, the loss information between the detection probability and the interpretability label can be calculated, and the loss information can be used to adjust the target detection model to obtain the trained target detection model.

[0199] In one embodiment, since different training sample images may generate different interpretable labels, different loss information calculation methods are set according to the interpretable labels in this embodiment of the application. Specifically, the step of "calculating the loss information between the detection probability and the interpretable label" may include:

[0200] Compare the interpretability label with the preset threshold to obtain a comparison result;

[0201] Determine the loss calculation method corresponding to the interpretability label based on the comparison results;

[0202] The loss information between the detection probability and the interpretable label is calculated according to the loss information calculation method.

[0203] In one embodiment, the loss information calculation method may include a first loss information calculation method and a second loss information calculation method. Therefore, when comparing the explainability label with a preset threshold, when the explainability label meets the preset threshold, the loss information between the detection probability and the explainability label may be calculated according to the first loss information calculation method. When the explainability label does not meet the preset threshold, the loss information between the detection probability and the explainability label may be calculated according to the second loss information calculation method. Specifically, the step of "calculating the loss information between the detection probability and the explainability label according to the loss information calculation method" may include:

[0204] The loss information between the detection probability and the interpretability label is calculated according to the first loss information calculation method, or the loss information between the detection probability and the interpretability label is calculated according to the second loss information calculation method.

[0205] In one embodiment, the inventors discovered during practice that, in the process of adding interpretable labels to sample target areas, some sample target areas may be given interpretable labels with lower values, which can lead to an imbalance between positive and negative samples. Therefore, in order to make the target detection model pay more attention to the loss of positive samples, the embodiment of the present application solves the problem of imbalance between positive and negative samples by adding sample modulation parameters in the process of calculating loss information.

[0206] Positive samples can include sample target regions corresponding to the target detection model's task. Negative samples can include sample target regions that are not corresponding to the target detection model's task. For example, the target detection model's task is to identify cats and dogs in training sample images. Therefore, positive samples can refer to sample target regions that can accurately select cats and dogs in training sample images, while negative samples can refer to regions other than cats and dogs in training sample images.

[0207] In one embodiment, when calculating the loss information between the detection probability and the interpretability label according to the first loss information calculation method, a first sample modulation parameter and a scaling parameter may be obtained, and the first sample modulation parameter, the scaling parameter, the detection probability, and the interpretability label may be logically processed to obtain the loss information. Specifically, the step of "calculating the loss information between the detection probability and the interpretability label according to the first loss information calculation method" may include:

[0208] Obtaining a first sample modulation parameter and a scaling parameter;

[0209] A logical operation is performed on the first sample modulation parameter, the scaling parameter, the detection probability and the interpretability label to obtain the loss information.

[0210] The first sample modulation parameter includes a parameter that can modulate positive and negative samples, and the scaling parameter includes a parameter that can scale a value.

[0211] In one embodiment, the first sample modulation parameter, scaling parameter, detection probability, and interpretability label are subjected to a logical operation to obtain the loss information, which can be expressed as follows:

[0212] BQFL(p)=-α|yp| β ((1-y)log(1-p)+ylog(p))

[0213] In which, the symbol BQFL(p) can represent loss information, the symbol y can represent an interpretability label, the symbol p can represent a detection probability, the symbol α can represent a first sample modulation parameter, and the symbol β can represent a scaling parameter.

[0214] In one embodiment, when calculating the loss information between the detection probability and the interpretability label according to the second loss information calculation method, a second sample modulation parameter and a scaling parameter may be obtained, and the second sample modulation parameter, the scaling parameter, the detection probability, and the interpretability label may be logically processed to obtain the loss information. Specifically, the step of "calculating the loss information between the detection probability and the interpretability label according to the second loss information calculation method" may include:

[0215] Obtaining a second sample modulation parameter and a scaling parameter;

[0216] The second sample modulation parameter, scaling parameter, detection probability and interpretability label are subjected to logical operation processing to obtain the loss information.

[0217] In one embodiment, the second sample modulation parameter, scaling parameter, detection probability, and interpretability label are subjected to a logical operation to obtain the loss information, which can be expressed as follows:

[0218] BQFL(p)=-λ|p| β log(1-p)

[0219] The symbol λ may represent the second sample modulation parameter.

[0220] In one embodiment, the preset thresholds may be set to i and j. When the interpretability label falls within the range (i, j], the first loss calculation method may be used to calculate the loss information between the detection probability and the interpretability label. When the interpretability label is equal to i, the second loss calculation method may be used to calculate the loss information between the detection probability and the interpretability label. Specifically, it may be shown in the following formula:

[0221]

[0222] In one embodiment, i may be set to 0, and j may be set to 1. In order to simply represent the loss information calculation method proposed in the embodiment of the present application, the above formula may be referred to as a Balanced Quality Focal Loss (BQFL).

[0223] In one embodiment, after obtaining the loss information between the detection probability and the interpretability label, the loss information can be used to adjust the target detection model to obtain a trained target detection model. The process of adjusting the target detection model using the loss information can include adjusting model parameters in the target detection model using the loss information to obtain a trained target detection model. The trained target detection model is used to perform target detection on an image to be detected, and obtain a target region in the image to be detected and an interpretability index for the target region.

[0224] For example, when the target detection model is FCOS, you can adjust the parameters in the Backbone, the Feature Pyramid, or the Head. For example, the Head predicts the image's feature information, generating a prediction box whose length and width can be represented as x and y. When adjusting the parameters of the target detection model, you can adjust x and y so that the trained target detection model can accurately detect the target area in the image and the interpretability indicator of the target area.

[0225] The embodiment of the present application proposes a target detection method, which can obtain a training sample image of a target detection model and the attributes of a sample target area in the training sample image; perform index generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area; add an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with an interpretability label; train the target detection model based on the training sample image with the interpretability label to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area. By adding an interpretability label to the sample target area, the target detection model can be adjusted according to the interpretability label when training the target detection model, thereby improving the interpretability of the target area confidence, thereby improving the accuracy and reliability of the target detection model.

[0226] For example, Figure 8 As shown in the detection results of the trained target detection model, the confidence level of the largest giraffe is 0.75, and the confidence level of the smallest giraffe is 0.50. Figure 5 Compared with the detection results of the target detection model that has not been trained with interpretable labels, the detection results of the trained target detection model are more consistent with the rules. Figure 9 As shown in the detection results of the trained target detection model, the confidence level of humans with higher clarity is 0.54, and the confidence level of humans with lower clarity is 0.52, which is more in line with the rules.

[0227] Furthermore, to overcome the imbalance between positive and negative samples, the present embodiment proposes different loss information calculation methods based on interpretability labels. By adding a sample adjustment parameter to the loss information between the detection probability and the interpretability label, the imbalance between positive and negative samples can be adjusted, thereby resolving the problem and improving the accuracy and reliability of the object detection model.

[0228] The method described in the above embodiment will be further described in detail below with examples.

[0229] The embodiment of the present application will take the target detection method integrated on a computer device as an example to introduce the method of the embodiment of the present application.

[0230] In one embodiment, if Figure 10 As shown in FIG, a target detection method, the specific process is as follows:

[0231] 201. The server obtains a training sample image of a target detection model and attributes of a sample target area in the training sample image.

[0232] The training sample image may include at least one sample target area. Figure 11 As shown, the training sample image may include a sample target area 1, a sample target area 2, and a sample target area 3.

[0233] The target detection model can be FCOS. For example, Figure 11 As shown, Figure 11 The target detection model in the figure may be FCOS. The classification branch in the figure may include Classificaton, and the positioning branch in the figure may include Centerness.

[0234] The attributes of the sample target area may include the size and definition of the sample target area.

[0235] The size of the sample target area may include the length and width of the sample target area.

[0236] In one embodiment, data from the Common Objects in Context (COCO) dataset may be used as training sample images. The COCO dataset is a dataset that can be used for image recognition.

[0237] 202. The server performs an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area.

[0238] In one embodiment, if Figure 11 As shown, the embodiment of the present application can add an interpretability evaluation branch based on FCOS, and the interpretability evaluation branch can perform index generation processing on the attributes of the sample target area to obtain the interpretability index of the sample target area.

[0239] In one embodiment, the server may perform interpretability evaluation on the sample target area based on the size attribute and the clarity attribute, thereby obtaining a size interpretability index and a clarity interpretability index of the sample target area.

[0240] In one embodiment, the server may perform an index generation process on the size attribute of the sample target area based on the following formula, thereby obtaining a size interpretability index of the sample target area:

[0241]

[0242] In one embodiment, the server may perform an index generation process on the clarity attribute of the sample target area based on the following formula, thereby obtaining a clarity interpretability index of the sample target area:

[0243] Blur score =log2(min(cv2.Laplacian(box)+1,3 10 )) / 18

[0244] 203. The server adds an interpretability label to the sample target area according to the interpretability index, and obtains a training sample image with the interpretability label.

[0245] In one embodiment, after obtaining the interpretability index, the server may add an interpretability label to the sample target area according to the interpretability index, thereby obtaining a training sample image with the interpretability label.

[0246] In one embodiment, for the size interpretability index, an interpretability label corresponding to the size may be generated according to the following method:

[0247] label smooth =max(Size score +0.5, 1)

[0248] In one embodiment, for the clarity interpretability index, an interpretability label corresponding to the clarity may be generated according to the following method:

[0249] label smooth =min(Blur score +0.2, 1)

[0250] After obtaining the interpretability labels, the server can associate the interpretability labels with the sample target regions, thereby obtaining training sample images with interpretability labels. For example, after obtaining interpretability labels for sample target regions 1, 2, and 3, the server can add the same identification information to the sample target regions and their corresponding interpretability labels, thereby associating the interpretability labels with the sample target regions.

[0251] In one embodiment, if Figure 11 As shown, the server may also combine the interpretability index and the category annotation to obtain a soft interpretability index, wherein the category annotation may be an annotation that describes the category of the object in the sample target area.

[0252] 204. The server trains the target detection model based on the training sample images with interpretable labels to obtain a trained target detection model.

[0253] In one embodiment, after obtaining the interpretable label, the server may train the target detection model using the training sample images with the interpretable label to obtain a trained target detection model.

[0254] In one embodiment, the target detection model can be trained and tested under the mmdetection code framework. During the training process, a 2x (training 24 epochs) learning strategy can be adopted.

[0255] In one embodiment, during the training process, Backbone can be set to ResNet50 network and ResNet101 network for training respectively.

[0256] In one embodiment, the target model with the interpretability evaluation branch added can be called QSFCOS. Furthermore, BQFL can be used in the training of the classification branch, and the FCOS-based object detection model can be named QSFCOS+BQFL. The test results and performance of QSFCOS+BQFL, QSFCOS, and FCOS on the COCO validation set can be compared. The comparison results can be shown in the following table:

[0257]

[0258]

[0259] Table 1: Performance comparison

[0260] Among them, mAP, AP_50, AP_75, AP_M and AP_L in the table can be the performance evaluation criteria in the COCO dataset.

[0261] As shown in Table 1, the QSFCOS model achieves comparable overall detection performance to the FCOS model, and significantly improves detection of medium-sized and large objects (AP_M increases by an average of 1.2% and AP_L increases by an average of 1.25%). The QSFCOS+BQFL model, while improving overall detection performance, further improves detection of medium-sized and large objects (AP_M increases by an average of 1.6% and AP_L increases by an average of 2%).

[0262] At the same time, in the embodiment of the present application, the FCOS model and the QSFCOS model are visualized in the COCO validation set, and the relationship between the confidence of the sample target area in the visualization detection result and the interpretable index of its size attribute can be shown as follows: Figure 12 and Figure 13 As shown. Among them, Figure 12 and 13 In , the horizontal axis is the interpretability index of the size attribute of the sample target area, and the vertical axis is the confidence of the target detection model for the sample target detection.

[0263] exist Figure 12 In , the sample target area framed by area 002 belongs to the area of ​​"small size but high confidence", and the sample target area framed by area 001 belongs to the area of ​​"large size and high confidence". Figure 13 In the example, the sample target area framed by area 004 belongs to the area of ​​"small size but high confidence", and the sample target area framed by area 003 belongs to the area of ​​"large size and high confidence". Figure 12 and Figure 13 A comparison shows that the addition of the size-based interpretable metric reduces the model's confidence in small objects during inference, while increasing the number of high-confidence detections of large objects. Combined with a comparison of detection performance, we can see that the addition of the size-based interpretable metric makes the QSFCOS model more interpretable during inference than the FCOS model and significantly improves detection performance for medium-sized and large objects.

[0264] In an embodiment of the present application, the server obtains a training sample image of a target detection model and the attributes of a sample target area in the training sample image; the server performs an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area; the server adds an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with an interpretability label; the server trains the target detection model based on the training sample image with the interpretability label to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area. By adding an interpretability label to the sample target area, the target detection model can be adjusted according to the interpretability label when training the target detection model, thereby improving the interpretability of the target area confidence, thereby improving the accuracy and reliability of the target detection model.

[0265] The present application also provides a sample generation method, which can be performed by a sample generation device, which can be integrated into a computer device, wherein the computer device can include at least one of a terminal and a server.

[0266] In one embodiment, the sample generation device can be integrated on a computer device such as a terminal or server to implement the sample generation method proposed in the embodiment of the present application. Specifically, the computer device can obtain the training sample image of the target detection model and the attributes of the sample target area in the training sample image; perform indicator generation processing on the attributes of the sample target area to obtain the interpretability index of the sample target area; add an interpretability label to the sample target area according to the interpretability index, and output the training sample image with the interpretability label.

[0267] The following are detailed descriptions of each embodiment. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.

[0268] The embodiments of the present application will be described from the perspective of a sample generation device, which may be integrated into a computer device, which may be a server, a terminal, or other device.

[0269] like Figure 14 As shown, a sample generation method is provided, and the specific process includes:

[0270] 301. Obtain a training sample image and attributes of a sample target area in the training sample image.

[0271] The training sample images include training data used to train the target detection model.

[0272] In one embodiment, training sample images may be obtained based on the task of the object detection model. For example, when the task of the object detection model is to detect human faces in an image, images containing human faces may be obtained as training sample images. For another example, when the task of the object detection model is to detect animals in an image, images containing animals may be obtained as training sample images, and so on.

[0273] The sample target area includes a location area formed by a detection object in the sample training image during target detection.

[0274] Attributes may include image properties in terms of pixels, clarity, size, and color.

[0275] 302. Perform index generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area.

[0276] Among them, the interpretability index includes measurement parameters for the attributes in the sample target area. In one embodiment, the interpretability index can be equivalent to a "score" assigned to the attributes in the sample target area based on the attributes of the sample target area. For example, when the interpretability index is for the clarity of the sample target area, the interpretability index can be used to know the clarity of the sample target area. For another example, when the interpretability index is for the size of the sample target area, the interpretability index can be used to know the size of the size in the sample target area. For another example, when the interpretability index is for the brightness of the sample target area, the interpretability index can be used to know the brightness of the size in the sample target area.

[0277] In one embodiment, the index generation process for the attributes of the sample target area can refer to step 102, which will not be repeated here.

[0278] 303. Add interpretability labels to the sample target area according to the interpretability index, and output the training sample image with the interpretability label.

[0279] Among them, the explainability label includes a label generated according to the explainability index, and the label has a basis and explainability.

[0280] In one embodiment, adding an explainability label to the sample target area according to the explainability indicator can refer to step 103, which will not be repeated here.

[0281] In one embodiment, after adding an explainable label to the sample target area, a training sample image with the explainable label can be output, so that the training sample image with the explainable label can be applied to various model training scenarios, thereby improving the accuracy and reliability of the model's task execution.

[0282] An embodiment of the present application proposes a sample generation method, which can obtain the attributes of a training sample image and a sample target area in the training sample image; perform indicator generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area; add an interpretability label to the sample target area according to the interpretability index, and output a training sample image with an interpretability label, so that the training sample image with an interpretability label can be applied to various model training scenarios, thereby improving the accuracy and reliability of the model's task execution.

[0283] In addition, to better implement the target detection method provided in the embodiments of this application, an object detection device is also provided in one embodiment. The object detection device can be integrated into a computer device. The meanings of the terms herein are the same as those in the target detection method described above. For specific implementation details, please refer to the description in the method embodiment.

[0284] In one embodiment, a target detection device is provided. The target detection device can be integrated into a computer device, such as Figure 15 As shown, the target detection device includes: an acquisition unit 401, an indicator generation unit 402, an adding unit 403 and a training unit 404, which are specifically as follows:

[0285] An acquisition unit 401 is configured to acquire a training sample image of a target detection model and attributes of a sample target region in the training sample image;

[0286] An index generating unit 402 is configured to perform index generating processing on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0287] An adding unit 403 is configured to add an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with the interpretability label;

[0288] The training unit 404 is used to train the target detection model based on the training sample image with the interpretable label to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area.

[0289] In one embodiment, the indicator generation unit includes:

[0290] an identification subunit, configured to identify the attribute and obtain the type of the attribute;

[0291] a determination subunit, configured to determine, according to the type of the attribute, a method for generating an indicator corresponding to the attribute;

[0292] The indicator generation subunit is used to perform indicator generation processing on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area.

[0293] In one embodiment, the indicator generation subunit includes:

[0294] An acquisition module, configured to acquire the size attribute of the training sample image;

[0295] a comparison module, configured to compare the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image;

[0296] The normalization processing module is used to perform normalization processing on the image size comparison information to obtain the size interpretability index.

[0297] In one embodiment, the indicator generation subunit includes:

[0298] An evaluation module, configured to evaluate the clarity attribute of the sample target area to obtain clarity evaluation information of the sample target area;

[0299] A comparison module, configured to compare the clarity evaluation information with a preset clarity judgment threshold, and generate comparison information based on the comparison result;

[0300] The linear operation module is used to perform linear operation processing on the contrast information to obtain the clarity interpretability index.

[0301] In one embodiment, the adding unit includes:

[0302] an adjusting subunit, configured to adjust the interpretability index to obtain an adjusted interpretability index;

[0303] a label mapping subunit, configured to perform label mapping processing on the adjusted interpretability index to obtain the interpretability label;

[0304] The association subunit is used to associate the explainability label with the sample target area to obtain the training sample image with the explainability label.

[0305] In one embodiment, the training unit includes:

[0306] an object detection subunit, configured to perform object detection on the training sample image with the interpretable label using the object detection model to obtain an initial detection area of ​​the training sample image;

[0307] an acquisition subunit, configured to acquire a detection probability of the initial detection area;

[0308] a calculation subunit, configured to calculate loss information between the detection probability and the interpretability label;

[0309] An adjustment subunit is used to adjust the target detection model using the loss information to obtain the trained target detection model.

[0310] In one embodiment, the computing subunit includes:

[0311] A comparison module is used to compare the explanatory label with a preset threshold to obtain a comparison result; a determination module is used to determine a loss information calculation method corresponding to the explanatory label based on the comparison result;

[0312] A calculation module is used to calculate the loss information between the detection probability and the explainability label according to the loss information calculation method.

[0313] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0314] The above-mentioned target detection device can improve the accuracy and reliability of target detection in images.

[0315] In one embodiment, a sample generating device is also provided. The sample generating device can be integrated into a computer device, such as Figure 16 As shown, the sample generation device includes: an attribute acquisition unit 501, an interpretability index generation unit 502 and a label adding unit 503, which are specifically as follows:

[0316] The attribute acquisition unit 501 is used to acquire attributes of a training sample image and a sample target area in the training sample image;

[0317] An interpretability index generating unit 502 is configured to perform index generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0318] The label adding unit 503 is configured to add interpretable labels to the sample target area based on the interpretability index and output a training sample image with the interpretable labels. In specific implementations, each of the above units can be implemented as an independent entity or in any combination as one or more entities. The specific implementation of each of the above units can be found in the previous method embodiments and will not be further described here.

[0319] The above-mentioned sample generation device can output training sample images with interpretable labels.

[0320] The embodiment of the present application also provides a computer device, which may include a terminal or a server. For example, the computer device may be used as a target detection terminal, which may be a mobile phone, a tablet computer, etc.; for another example, the computer device may be a server, such as a target detection server. Figure 17 As shown, it shows a schematic diagram of the structure of the terminal involved in the embodiment of the present application, specifically:

[0321] The computer device may include one or more processing core processors 601, one or more computer readable storage media memories 602, a power supply 603, an input unit 604 and other components. Those skilled in the art will understand that Figure 17 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0322] Processor 601 is the control center of the computer device. It utilizes various interfaces and lines to connect the various components of the entire computer device. By running or executing software programs and / or modules stored in memory 602 and accessing data stored in memory 602, it performs various functions of the computer device and processes data, thereby performing overall testing of the computer device. Optionally, processor 601 may include one or more processing cores; preferably, processor 601 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interfaces, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 601.

[0323] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0324] The computer device also includes a power supply 603 for supplying power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 603 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0325] The computer device may further include an input unit 604, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0326] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the computer device will load the executable files corresponding to one or more application processes into the memory 602 according to the following instructions, and the processor 601 will run the application stored in the memory 602 to implement various functions as follows:

[0327] Obtaining a training sample image of a target detection model and attributes of a sample target area in the training sample image;

[0328] Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0329] Adding an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with the interpretability label;

[0330] The target detection model is trained based on the training sample image with interpretable labels to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area.

[0331] Or implement the following functions:

[0332] Acquire a training sample image and attributes of a sample target area in the training sample image;

[0333] Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0334] An interpretability label is added to the sample target area according to the interpretability index, and a training sample image with the interpretability label is output.

[0335] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0336] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0337] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0338] To this end, an embodiment of the present application further provides a storage medium storing a computer program that can be loaded by a processor to execute the steps of any target detection method provided in the embodiment of the present application. For example, the computer program can execute the following steps:

[0339] Obtaining a training sample image of a target detection model and attributes of a sample target area in the training sample image;

[0340] Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0341] Adding an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with the interpretability label;

[0342] The target detection model is trained based on the training sample image with interpretable labels to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain the target area of ​​the image to be detected and the interpretability index of the target area.

[0343] Or perform the following steps:

[0344] Acquire a training sample image and attributes of a sample target area in the training sample image;

[0345] Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area;

[0346] An interpretability label is added to the sample target area according to the interpretability index, and a training sample image with the interpretability label is output.

[0347] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0348] Since the computer program stored in the storage medium can execute the steps in any target detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any target detection method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0349] The above is a detailed introduction to a target detection method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A target detection method, characterized in that: include: Obtaining a training sample image of a target detection model and attributes of a sample target area in the training sample image; Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area; Adding an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with the interpretability label; The target detection model is trained based on the training sample image with the interpretable label to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain a target area of ​​the image to be detected and an interpretability index of the target area; The performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area includes: Identify the attribute and obtain the type of the attribute; Determine, based on the type of the attribute, a method for generating an indicator corresponding to the attribute; Performing an indicator generation process on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area; The interpretability index includes a size interpretability index, and the type of the attribute includes a size attribute; performing an indicator generation process on the attribute of the sample target area according to the indicator generation method to obtain the interpretability index of the sample target area includes: Obtaining the size attribute of the training sample image; Comparing the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image; Normalizing the image size comparison information to obtain the size interpretability index.

2. The target detection method according to claim 1, wherein: The interpretability index includes a clarity interpretability index, and the type of attribute includes a clarity attribute; The step of performing an indicator generation process on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area includes: Performing an evaluation process on the clarity attribute of the sample target area to obtain clarity evaluation information of the sample target area; Comparing the clarity evaluation information with a preset clarity judgment threshold, and generating comparison information based on the comparison result; Performing linear operation on the contrast information to obtain the clarity interpretability index.

3. The target detection method according to claim 1, wherein: Adding an interpretability label to the sample target area according to the interpretability index to obtain a training sample image with the interpretability label includes: Adjusting the interpretability index to obtain an adjusted interpretability index; Performing label mapping processing on the adjusted interpretability index to obtain the interpretability label; The interpretability label is associated with the sample target area to obtain the training sample image with the interpretability label.

4. The target detection method according to claim 1, wherein: The object detection model is trained based on the training sample images with interpretable labels to obtain a trained object detection model, including: Performing target detection on the training sample image with the interpretable label using the target detection model to obtain an initial detection area of ​​the training sample image; Obtaining a detection probability of the initial detection area; Calculating loss information between the detection probability and the interpretability label; The target detection model is adjusted using the loss information to obtain the trained target detection model.

5. The target detection method according to claim 4, wherein: The calculating loss information between the detection probability and the interpretability label includes: Comparing the interpretability label with a preset threshold to obtain a comparison result; Determining a loss information calculation method corresponding to the explainability label according to the comparison result; The loss information between the detection probability and the explainability label is calculated according to the loss information calculation method.

6. The target detection method according to claim 4, wherein: The step of performing target detection on the training sample image with the interpretable label using the target detection model to obtain an initial detection area of ​​the training sample image includes: Performing convolution processing on the training sample image to obtain convolution information of the training sample image; Performing feature extraction on the convolution information to obtain feature information of the training sample image; Based on the feature information of the training sample image, classification prediction is performed on the training sample image to obtain the initial detection area.

7. A sample generation method, characterized in that: include: Obtaining a training sample image and attributes of a sample target area in the training sample image; Performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area; Adding an interpretability label to the sample target area according to the interpretability index, and outputting a training sample image with the interpretability label; The performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area includes: Identify the attribute and obtain the type of the attribute; Determine, based on the type of the attribute, a method for generating an indicator corresponding to the attribute; Performing an indicator generation process on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area; The interpretability index includes a size interpretability index, and the type of the attribute includes a size attribute; performing an indicator generation process on the attribute of the sample target area according to the indicator generation method to obtain the interpretability index of the sample target area includes: Obtaining the size attribute of the training sample image; Comparing the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image; Normalizing the image size comparison information to obtain the size interpretability index.

8. A computer device, characterized in that: It includes a memory and a processor; the memory stores an application, and the processor is used to run the application in the memory to perform the operations in the target detection method according to any one of claims 1 to 6 or the sample generation method according to claim 7.

9. A target detection device, characterized in that: include: An acquisition unit, configured to acquire a training sample image of a target detection model and attributes of a sample target area in the training sample image; an index generating unit, configured to perform index generating processing on the attributes of the sample target area to obtain an interpretability index of the sample target area; an adding unit, configured to add an interpretability label to the sample target area according to the interpretability index, to obtain a training sample image with the interpretability label; A training unit is configured to train the target detection model based on the training sample images with interpretable labels to obtain a trained target detection model, wherein the trained target detection model is used to perform target detection on the image to be detected to obtain a target area of ​​the image to be detected and an interpretability index of the target area; The indicator generating unit includes: an identification subunit, configured to identify the attribute and obtain the type of the attribute; a determination subunit, configured to determine, according to the type of the attribute, a method for generating an indicator corresponding to the attribute; an indicator generation subunit, configured to perform indicator generation processing on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area; The interpretability index includes a size interpretability index, and the attribute type includes a size attribute; the index generation subunit includes: An acquisition module, configured to acquire the size attribute of the training sample image; a comparison module, configured to compare the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image; The normalization processing module is used to perform normalization processing on the image size comparison information to obtain the size interpretability index.

10. A sample generating device, characterized in that: include: An attribute acquisition unit, configured to acquire attributes of a training sample image and a sample target area in the training sample image; an interpretability index generating unit, configured to perform index generation processing on the attributes of the sample target area to obtain an interpretability index of the sample target area; a label adding unit, configured to add an interpretability label to the sample target area according to the interpretability index, and output a training sample image with the interpretability label; The performing an index generation process on the attributes of the sample target area to obtain an interpretability index of the sample target area includes: Identify the attribute and obtain the type of the attribute; Determine, based on the type of the attribute, a method for generating an indicator corresponding to the attribute; Performing an indicator generation process on the attributes of the sample target area according to the indicator generation method to obtain an interpretability indicator of the sample target area; The interpretability index includes a size interpretability index, and the type of the attribute includes a size attribute; performing an indicator generation process on the attribute of the sample target area according to the indicator generation method to obtain the interpretability index of the sample target area includes: Obtaining the size attribute of the training sample image; Comparing the size attributes of the sample target area with the size attributes of the training sample image to obtain image size comparison information between the sample target area and the training sample image; Normalizing the image size comparison information to obtain the size interpretability index.

Images