Target detection method and device, electronic equipment and storage medium

Abstract

The application discloses a target detection method and device, electronic equipment and storage medium, and belongs to the technical field of image processing. The target detection method comprises the following steps: acquiring a to-be-detected image, performing target detection on the to-be-detected image to obtain a target detection result, performing negative example filtering on the target detection result to obtain a detection result corresponding to a target user, and obtaining an image retrieval feature library corresponding to the target user by preprocessing and feature extraction on a negative example image determined for the target user. The application can obtain a detection result of a target detection event defined for the target user without separately customizing a target detection model for the target user, only needs to perform negative example filtering on the target detection result according to the image retrieval feature library corresponding to the target user, can improve the efficiency of target detection, and reduces the implementation cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a target detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Object detection is a popular area of ​​computer vision and digital image processing, widely used in fields such as robot navigation, intelligent video surveillance, industrial inspection, and aerospace. Reducing the consumption of human capital through computer vision has significant practical implications. The basic task of object detection is to determine the category of objects in an image, and to use rectangular bounding boxes to determine the location and size of the objects, providing corresponding confidence scores. As a fundamental problem in computer vision, object detection is also the foundation for many computer vision tasks such as image segmentation, object tracking, and image captioning.

[0003] In real-world scenarios, there exists a subjective target detection scenario. Taking urban management violations as an example, some users believe that placing tables and chairs on the roadside constitutes a violation and needs to be detected promptly so that law enforcement officers can take appropriate action. Other users believe that this is not a violation and should not be detected. Furthermore, users cannot describe all the events that should be detected (positive examples) and those that should not be detected (negative examples) at once. Current technologies require each user to customize a detection model individually, and each time a user changes their requirements, a new detection model must be customized to meet those requirements, resulting in low efficiency and high costs. Summary of the Invention

[0004] This invention provides a target detection method, apparatus, electronic device, and storage medium to solve the problems of low efficiency and high cost in the prior art, which involves customizing detection models for each user individually.

[0005] This invention provides a target detection method, comprising:

[0006] Acquire the image to be detected, perform target detection on the image to be detected, and obtain the target detection result;

[0007] Based on the image retrieval feature library corresponding to the target user, negative examples are filtered on the target detection results to obtain the detection results corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of the negative example images determined by the target user.

[0008] According to a target detection method provided by the present invention, the step of filtering negative examples of the target detection results based on an image retrieval feature library corresponding to the target user to obtain the detection results corresponding to the target user includes:

[0009] The target detection results are preprocessed and feature extracted to obtain the first image features corresponding to the target detection results;

[0010] Search for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold;

[0011] If the second image feature exists, the target detection result is not output; or, if the second image feature does not exist, the target detection result is output.

[0012] According to a target detection method provided by the present invention, the target detection method further includes:

[0013] Obtain the negative example images determined by the target user and construct an image dataset;

[0014] Traverse the image dataset, perform the preprocessing and feature extraction on each image in the image dataset, and obtain the feature vector or matrix corresponding to each image;

[0015] By associating the feature vector or matrix corresponding to each image with each image, an image retrieval feature library corresponding to the target user is obtained.

[0016] According to a target detection method provided by the present invention, the step of preprocessing and feature extraction of the target detection result to obtain a first image feature corresponding to the target detection result includes:

[0017] The target detection results are preprocessed, including: enhancing contrast and removing noise;

[0018] The preprocessed target detection results are subjected to Scale Invariant Feature Transform (SIFT) feature extraction to obtain the SIFT features corresponding to the target detection results. The preprocessed target detection results are then subjected to high-level semantic feature extraction to obtain the high-level semantic features corresponding to the target detection results.

[0019] By fusing the SIFT features and high-level semantic features, the first image feature corresponding to the target detection result is obtained.

[0020] According to a target detection method provided by the present invention, the step of performing Scale Invariant Feature Transform (SIFT) feature extraction on the preprocessed target detection result to obtain the SIFT features corresponding to the target detection result includes:

[0021] Constructing scale space;

[0022] The feature points in the preprocessed target detection results are determined, and the extreme points with low contrast and unstable edge response points are removed from the feature points. Then, the extreme points with significant errors are removed from the feature points by combining the random sampling consensus algorithm, and the first feature point set is obtained.

[0023] Determine the orientation of each feature point in the first feature point set, and rotate each feature point in the first feature point set to obtain the SIFT feature corresponding to the target detection result.

[0024] According to a target detection method provided by the present invention, the step of extracting high-level semantic features from the preprocessed target detection result to obtain the high-level semantic features corresponding to the target detection result includes:

[0025] The preprocessed target detection results are input into a CNN feature extraction network for high-level semantic feature extraction.

[0026] The output vector of the fully connected layer of the CNN feature extraction network is obtained to obtain the high-level semantic features corresponding to the target detection result.

[0027] According to a target detection method provided by the present invention, the step of searching for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold includes:

[0028] Based on a preset similarity measurement method, the similarity between the first image feature and the feature vector or matrix corresponding to each image in the image retrieval feature library is calculated;

[0029] Determine whether there exists a feature vector or matrix corresponding to the second image whose similarity to the features of the first image exceeds the preset threshold;

[0030] If it exists, then the feature vector or matrix corresponding to the second image is determined to be a feature of the second image;

[0031] The preset similarity measurement methods include: quadratic distance measurement method, Manhattan distance measurement method, Euclidean distance measurement method, or cosine similarity measurement method.

[0032] The present invention also provides a target detection device, comprising:

[0033] The target detection unit is used to acquire an image to be detected, perform target detection on the image to be detected, and obtain a target detection result.

[0034] The negative example filtering unit is used to perform negative example filtering on the target detection results to obtain the detection results corresponding to the target user.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target detection method as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target detection method as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the target detection method as described above.

[0038] The target detection method, apparatus, electronic device, and storage medium provided by this invention perform target detection on the image to be detected, obtain the target detection result, and then perform negative example filtering on the target detection result based on the image retrieval feature library corresponding to the target user. It is not necessary to customize the target detection model separately for the target user. It is only necessary to perform negative example filtering on the target detection result according to the image retrieval feature library corresponding to the target user to obtain the detection result for the target detection event defined by the target user, which can improve the efficiency of target detection and reduce the implementation cost. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a diagram illustrating the application environment in which the target detection method provided in this embodiment of the invention can operate.

[0041] Figure 2 A schematic flowchart of the target detection method provided by the present invention;

[0042] Figure 3 This is a schematic diagram of the process for filtering negative examples of target detection results provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the process for preprocessing and feature extraction of target detection results provided in an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the process for extracting SIFT features from preprocessed target detection results according to an embodiment of the present invention;

[0045] Figure 6This is a schematic diagram of the target detection device provided in an embodiment of the present invention;

[0046] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0049] Existing technologies require custom-designed detection models for each user, and each time a user changes their requirements, a new detection model must be customized, resulting in low efficiency and high cost. To address this, this invention provides a target detection method, apparatus, electronic device, and storage medium. By performing target detection on the image to be detected, a target detection result is obtained. Then, negative examples are filtered from the target detection result based on an image retrieval feature library corresponding to the target user. This eliminates the need for custom-designed target detection models for each target user; simply filtering negative examples from the target detection result using the target user's image retrieval feature library is sufficient to obtain detection results for target detection events defined by the target user. This improves the efficiency of target detection and reduces implementation costs.

[0050] Figure 1 This diagram illustrates the application environment in which the target detection method provided in this embodiment of the invention can operate. For example... Figure 1As shown, the application environment includes terminal 110 and server 120. Terminal 110 and server 120 communicate via a network, which can be a wireless communication network or a wired communication network. The number of terminals and servers is unlimited. The wireless communication network can include, but is not limited to, at least one of the following: WIFI (Wireless Fidelity) and Bluetooth. The wired communication network can include, but is not limited to, at least one of the following: wide area network, metropolitan area network, and local area network.

[0051] In some embodiments, terminal 110 (terminal device) includes various handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem with wireless communication capabilities, such as mobile phones, tablets, desktop laptops, and smart devices capable of running applications, including the central console of a smart car. Specifically, it can refer to user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device. Terminal devices can also be satellite phones, cellular phones, smartphones, wireless data cards, wireless modems, machine-type communication devices, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices or wearable devices, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical care, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, terminal devices in 5G networks or future communication networks, etc. Terminals can be battery-powered or attached to and powered by the power system of a vehicle or vessel. The power system of a vehicle or ship can also charge the terminal's battery to extend the terminal's communication time.

[0052] Server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0053] It should be noted that the target detection method in this invention can be implemented directly on the terminal 110, or directly on the server 120, or it can be implemented on the server 120 and then the server 120 sends the processing result to the terminal 110.

[0054] Figure 2 This is a schematic flowchart of the target detection method provided by the present invention. Figure 2 As shown, a target detection method is provided, which is applied to... Figure 1 The following steps are used as an example of the terminal in the example: Step 210 and Step 220.

[0055] Step 210: Obtain the image to be detected, perform target detection on the image to be detected, and obtain the target detection result;

[0056] Optionally, the image to be detected can be one or more frames from the video to be detected, or one or more frames captured by a camera.

[0057] Object detection refers to determining the location and size of an object in an image using rectangular bounding boxes. Therefore, the result of object detection can be understood as an image containing rectangular bounding boxes.

[0058] Optionally, target detection is performed on the image to be detected to obtain target detection results, including:

[0059] The image to be detected is input into the target detection model, and the target detection result output by the target detection model is obtained.

[0060] It should be noted that this invention does not limit the target detection model, nor does it limit the specific type of target detection results.

[0061] The target detection model detects all suspected targets in the image to be detected. Then, based on the target user's needs for the target, all suspected targets are filtered for negative examples, thus finally obtaining the detection result corresponding to the target user. The detection result corresponding to the target user removes the negative examples considered by the target user.

[0062] For example, in the detection of violations by urban management officers, some users believe that tables and chairs placed on the roadside do not need to be detected. For these users, such detected images should be filtered out as negative examples. Other users believe that tables and chairs placed on the roadside are detrimental to the city's appearance and need to be detected. For these users, the detection results should be retained as positive examples.

[0063] Step 220: Based on the image retrieval feature library corresponding to the target user, perform negative example filtering on the target detection results to obtain the detection results corresponding to the target user.

[0064] The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of negative example images determined by the target user.

[0065] The negative example images identified by the target user are those from the target detection results based on the user's subjective judgment. By preprocessing and feature extraction on these negative example images, multiple features are obtained and combined to form an image retrieval feature library.

[0066] It is understandable that image retrieval feature libraries are targeted at specific users. Different users may have different or the same image retrieval feature libraries, depending on whether different users have the same definition of target detection events. Here, a target detection event refers to an event in which the detected target belongs to the positive examples recognized by the user, that is, an event that should be detected.

[0067] For a query image in the target detection results, the same preprocessing and feature extraction are performed on the query image to obtain its features. The similarity is calculated with each feature in the image retrieval feature library according to a suitable similarity measurement method. When the similarity exceeds a preset threshold, the query image is considered to belong to the target user-defined negative example, and thus the query image is not output, thereby filtering user-defined negative examples in the target detection results.

[0068] This invention utilizes an image retrieval feature library corresponding to the target user to achieve negative example filtering of target detection results.

[0069] Furthermore, the features of the selected negative examples can be stored in an image retrieval feature library.

[0070] In this embodiment of the invention, target detection is performed on the image to be detected to obtain the target detection result. Then, negative examples are filtered based on the image retrieval feature library corresponding to the target user. Without modifying the target detection model, negative examples that the user does not need to be detected can be effectively filtered. There is no need to customize the target detection model separately for the target user. It is only necessary to filter the negative examples of the target detection result according to the image retrieval feature library corresponding to the target user to obtain the detection result for the target detection event defined by the target user. When the user's needs change, compared with customizing the target detection model separately for the user, it can quickly respond to the user's needs, reduce costs, improve detection accuracy, and greatly enhance the user experience.

[0071] Figure 3 This is a schematic diagram illustrating the process of negative example filtering for target detection results provided in an embodiment of the present invention. In some embodiments, such as Figure 3 As shown, step 220 includes:

[0072] Step 221: Preprocess and extract features from the target detection results to obtain the first image features corresponding to the target detection results;

[0073] Since images can be affected by factors such as noise, color difference, and brightness, the quality of extracted features may decrease, which can affect the effectiveness of subsequent image retrieval based on image features. To address this issue, this invention first preprocesses the target detection results.

[0074] In this embodiment, the preprocessing includes enhancing contrast and removing noise. Other methods that can improve the quality of image feature extraction can also be used, and the present invention does not limit them.

[0075] Image feature extraction involves identifying certain information that represents an image, such as objects, scenery, or other things in the picture, different colors, or different textures. These extracted features can then be used as tags for retrieval and matching. This embodiment does not limit the method of feature extraction.

[0076] Step 222: Search for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold;

[0077] That is, by using an appropriate similarity measurement method, the similarity between the first image feature and each feature in the image retrieval feature library corresponding to the target user is calculated, and a second image feature whose similarity to the first image feature exceeds a preset threshold is found.

[0078] Step 223: If the second image feature exists, the target detection result is not output; or, if the second image feature does not exist, the target detection result is output.

[0079] If the second image feature exists, that is, the similarity between the first image feature and the second image feature exceeds a preset threshold, then the target detection result corresponding to the first image feature is considered to be a negative example defined by the target user, and thus the target detection result corresponding to the first image feature is not output, thereby filtering out user-defined negative examples in the target detection result.

[0080] If the second image feature does not exist, the similarity between the first image feature and each feature in the image retrieval feature library does not exceed the preset threshold. Therefore, the target detection result corresponding to the first image feature is considered not to be a negative example defined by the target user. Thus, the target detection result corresponding to the first image feature is not filtered, and the target detection result corresponding to the first image feature is output.

[0081] In this embodiment of the invention, negative examples of the target detection results are filtered based on the image retrieval feature library corresponding to the target user. Without modifying the target detection model, negative example images that the user does not need to be detected can be effectively filtered, which can quickly respond to user needs, reduce costs, and improve detection accuracy.

[0082] In some embodiments, the target detection method further includes: constructing an image retrieval feature library corresponding to the target user.

[0083] Optionally, constructing an image retrieval feature library corresponding to the target user includes:

[0084] Obtain the negative example images determined by the target user and construct an image dataset;

[0085] The image dataset is traversed, and the preprocessing and feature extraction are performed on each image in the image dataset to obtain the feature vector or matrix corresponding to each image.

[0086] By associating the feature vector or matrix corresponding to each image with each image, an image retrieval feature library corresponding to the target user is obtained.

[0087] The main steps for constructing the image retrieval feature library corresponding to the target user are as follows: First, obtain the negative example images determined by the target user and construct an image dataset. Then, traverse the image dataset and perform preprocessing and feature extraction as in step 221 on each image in the image dataset to obtain the feature vector or matrix corresponding to each image. Then, associate the feature vector or matrix corresponding to each image with the original image (i.e., each image itself), that is, establish a mapping relationship between the image and its feature vector or matrix. Use the extracted feature vector or matrix as the label of the entire image, thereby constructing the image retrieval feature library corresponding to the target user.

[0088] In this embodiment of the invention, by constructing an image retrieval feature library corresponding to the target user, negative examples are filtered for the target detection results based on the image retrieval feature library corresponding to the target user. Without modifying the target detection model, negative example images that the user does not need to be detected can be effectively filtered, which can quickly respond to user needs, reduce costs, and improve detection accuracy.

[0089] Figure 4 This is a schematic diagram illustrating the process of preprocessing and feature extraction of target detection results provided in an embodiment of the present invention. In some embodiments, such as Figure 4 As shown, step 221 includes:

[0090] Step 2211: Preprocess the target detection results, the preprocessing including: enhancing contrast and removing noise;

[0091] Step 2212: Perform Scale Invariant Feature Transform (SIFT) feature extraction on the preprocessed target detection results to obtain the SIFT features corresponding to the target detection results; perform high-level semantic feature extraction on the preprocessed target detection results to obtain the high-level semantic features corresponding to the target detection results.

[0092] Because single features typically express image content from only one aspect, neglecting other attributes, their representation is often inaccurate. To fully express image content, complementary features can be fused for better results. Therefore, this invention addresses the issue of insufficient expressive power of single features by employing a multi-feature fusion scheme, selecting appropriate image features for fusion to create the image's feature vector.

[0093] In multi-feature fusion, this invention focuses on the complementary advantages of various individual features rather than simply having as many features as possible, thereby avoiding meaningless feature stacking and information redundancy. The features selected in this invention are scale-invariant feature transform (SIFT) features and high-level semantic features.

[0094] SIFT feature extraction constructs visual words by clustering SIFT descriptors and extracts discriminative representations, which belong to the low-level features.

[0095] High-level semantic feature extraction can transform the input image into a fixed-length vector to obtain the high-level semantic features of the image. This embodiment does not limit the method of high-level semantic feature extraction.

[0096] SIFT features and high-level semantic features have different focuses. SIFT features have good stability and universality, and are not easily affected by external conditions, but they lack high-level semantic information. High-level semantic features reflect the high-level semantic information of an image, which can achieve better retrieval results, but they have high requirements for the dataset, ignore the relationship between the whole and the parts, and still have certain problems in image retrieval. The two types of features belong to different levels and are complementary, so feature fusion can be performed.

[0097] Step 2213: Fuse the SIFT features and high-level semantic features to obtain the first image features corresponding to the target detection result.

[0098] Optionally, SIFT features and high-level semantic features can be concatenated to achieve feature fusion, or SIFT features and high-level semantic features can be weighted and summed to achieve feature fusion, or other feature fusion methods can be used. This invention does not limit these methods.

[0099] It should be noted that when constructing the image retrieval feature library corresponding to the target user, the method for preprocessing and feature extraction of each image in the image dataset is the same as steps 2211 to 2213 in this embodiment.

[0100] In this embodiment of the invention, by performing Scale Invariant Feature Transform (SIFT) feature extraction and high-level semantic feature extraction on the preprocessed target detection results, and fusing the extracted SIFT features and high-level semantic features, the accuracy of feature matching can be effectively improved, thereby effectively filtering out negative example images that the user does not need to be detected and improving the detection accuracy.

[0101] In some embodiments, such as Figure 5 As shown, the step of performing Scale Invariant Feature Transform (SIFT) feature extraction on the preprocessed target detection results to obtain the SIFT features corresponding to the target detection results includes:

[0102] Step 500: Construct scale space;

[0103] The scale space of an image can be represented as:

[0104] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0105] Where L(x, y, σ) is the scale space function representation at this scale, G(x, y, σ) represents a scale-varying Gaussian function, and I(x, y) represents the target detection result after the preprocessing, simply referred to as the image. Here, "*" indicates convolution operation in the x and y directions.

[0106] The specific formula for G(x, y, σ) is:

[0107]

[0108] Here, (x, y) represents a point located in the image. σ is the scale-space parameter; different values ​​of σ result in different levels of blur in the image. The smaller the value of σ, the higher the resolution of the image, and the better the details can be observed.

[0109] However, due to the low detection efficiency of LoG (Local Gaussian Geometry), the Difference of Gaussian (DG) DoG pyramid method is used instead of LoG detection to improve efficiency. The specific calculations are as follows:

[0110] D(x,y,σ)=(G(x,y,kσ)-G(x,y,0))*I(x,y)

[0111] =L(x,y,kσ)-L(x,y,σ)

[0112] In the formula, D(x, y, σ) represents the image from which SIFT features need to be extracted, and k represents the multiple of the image from which SIFT features need to be extracted and its neighboring scale space.

[0113] Step 510: Determine the feature points in the preprocessed target detection results, remove the extreme points with low contrast and unstable edge response points from the feature points, and combine the random sampling consensus algorithm to remove the extreme points with significant errors from the feature points to obtain the first feature point set.

[0114] SIFT feature points are local extrema in the DoG space. The search and determination of these extrema need to be performed in adjacent scale spaces. To improve the reliability of feature points and enhance the speed and accuracy of feature matching, extracted spatial extrema need to be deleted using a specific method. This invention combines a Random Sample Consensus (RANSA) algorithm to remove extrema with significant errors. Specifically, the following steps are included:

[0115] (a) Remove extreme points with low contrast and accurately locate key points.

[0116] Curve fitting of the LoG function using a three-dimensional quadratic formula is shown below.

[0117]

[0118] Here, x refers to a pixel located at a certain point in the image, and X refers to the 26 pixels surrounding that point.

[0119] By differentiating the above equation and setting the differentiated expression equal to zero, we obtain the extreme point:

[0120]

[0121] The equation corresponding to the extreme point is:

[0122]

[0123] Set a threshold; if D(x) is greater than the threshold, retain the point. Otherwise, if it is less than the threshold, consider the extreme point as a low-contrast extreme point and discard it.

[0124] (b) Remove unstable edge response points

[0125] Points in edge regions exhibit poor decision-making ability and are susceptible to noise. The difference of Gaussian function produces a strong response in edge regions, so points with this property should be removed. The curvature of extreme points can be used to determine whether a point is an edge point. The principal curvature of edge response points typically exhibits a property: the principal curvature is large across the edge and small perpendicular to the edge. The principal curvature is calculated using the second-order Hessian matrix.

[0126]

[0127] In the formula, the eigenvalues ​​α and β of H represent the gradients in the x and y directions, respectively, that is:

[0128] Tr(H)=D xx +D yy =a+β

[0129] Det(H) = D xx D yy -(D xy ) 2 =αβ

[0130]

[0131] Where r is the ratio of eigenvalues ​​α and β.

[0132] When the expression holds true, retain the extreme point; otherwise, treat the extreme point as an edge response point and discard it.

[0133]

[0134] (c) Use the Ransaca algorithm to remove feature points with significant errors.

[0135] Directly using image feature points extracted by the SIFT algorithm for feature matching yields poor results because the feature points extracted from each image have some deviation. This invention incorporates the Ransac algorithm to remove feature points with significant errors extracted from each image. Feature matching is then performed on the remaining feature points after algorithm processing. The implementation steps of the Ransac algorithm are as follows:

[0136] ①Suppose we want to fit a model determined by at least n points to P data points {x1,x2,…,xn} (p>=n, n=3 for a circle);

[0137] ② Let the iteration count k = 1;

[0138] ③ Randomly select n points from p to fit a model, denoted as M1. n is 3 at the beginning and will increase over time.

[0139] ④ Given the tolerance error, calculate the number of residuals of the data points {x1,x2,…,xn} that are within the deviation relative to the model. If the number of inliers is greater than the threshold t, the algorithm terminates. Afterward, the model can be refitted based on the set of inliers (the least squares method or its variants can be used).

[0140] ⑤ Let k = k + 1. If k is less than the preset k, repeat step 3. The new set of interior points and the model are denoted as s1* and M1*, respectively. Otherwise, the model with the most interior points is used, or the algorithm fails.

[0141] The combination of Ransac and SIFT algorithms has significant advantages. It can remove noise interference points in images, and the combination greatly enhances the feature matching ability of SIFT algorithm, making SIFT algorithm more robust.

[0142] Step 520: Determine the orientation of each feature point in the first feature point set, and rotate each feature point in the first feature point set to obtain the SIFT feature corresponding to the target detection result.

[0143] The above method can determine the feature points of an image. However, to improve the matching accuracy and robustness of the algorithm, it is necessary to determine a direction for the determined feature points. The magnitude m(x,y) and argument θ(x,y) of the pixel gradient are respectively:

[0144]

[0145]

[0146] Where L(x,y) represents the direction of the pixel (x,y).

[0147] Furthermore, to improve the robustness of the generated feature vectors, the coordinate axes of the generated feature points need to be rotated. The neighborhood radius chosen for this rotation is r, which is calculated as follows:

[0148]

[0149] The new coordinates after rotation are:

[0150]

[0151] Where σ is the scale space parameter and d is a constant, which can be 4.

[0152] In this embodiment of the invention, by combining the Ransac algorithm and the SIFT algorithm, the speed and accuracy of feature matching can be effectively improved, the robustness of the algorithm can be enhanced, and negative images that the user does not need to be detected can be effectively filtered, thereby improving the detection accuracy.

[0153] In some embodiments, the step of extracting high-level semantic features from the preprocessed target detection results to obtain the high-level semantic features corresponding to the target detection results includes:

[0154] The preprocessed target detection results are input into a CNN feature extraction network for high-level semantic feature extraction.

[0155] The output vector of the fully connected layer of the CNN feature extraction network is obtained to obtain the high-level semantic features corresponding to the target detection result.

[0156] Optionally, this invention uses Convolutional Neural Networks (CNNs) to extract high-level semantic features. CNN feature extraction transforms the input image into a fixed-length vector through a series of processes such as convolution and pooling, thereby obtaining the high-level semantic features of the image.

[0157] Optionally, since ResNet avoids the degradation problem in deep networks through its residual structure and solves the problems of gradient vanishing and gradient exploding through batch normalization (BN) layers, it performs well in multiple visual tasks. Therefore, this embodiment of the invention uses ResNet50 as the CNN feature extraction network. The entire network structure consists of 50 layers, composed of four large modules: the first large module has 3 small modules, the second large module has 4 small modules, the third large module has 6 small modules, and the fourth large module has 3 small modules. Each small module consists of 3 convolutional kernels. In addition, the first convolutional layer and the last fully connected layer of the network constitute the 50-layer network structure.

[0158] All negative images are fed into a pre-trained ResNet50 model in batches. The high-level semantic feature vector of the image is the output vector of the last fully connected layer of the ResNet50 model. Then, the SIFT feature vector and the high-level semantic feature vector are fused to form a complete feature vector containing both low-level and high-level semantic information of the image. This feature vector is stored in the image retrieval feature library. The similarity between the feature vector of the target detection result and the features in the image retrieval feature library is then calculated to obtain the similarity between the target detection result and the negative images in the image retrieval feature library. This completes the filtering of negative images.

[0159] In this embodiment of the invention, high-level semantic features of the target detection results are extracted by CNN feature extraction and then fused with SIFT features, which can effectively improve the speed and accuracy of feature matching, thereby effectively filtering negative images that users do not need to be detected and improving the detection accuracy.

[0160] In some embodiments, step 222 includes:

[0161] Based on a preset similarity measurement method, the similarity between the first image feature and the feature vector or matrix corresponding to each image in the image retrieval feature library is calculated;

[0162] Determine whether there exists a feature vector or matrix corresponding to the second image whose similarity to the features of the first image exceeds the preset threshold;

[0163] If it exists, then the feature vector or matrix corresponding to the second image is determined to be a feature of the second image;

[0164] The preset similarity measurement methods include: quadratic distance measurement method, Manhattan distance measurement method, Euclidean distance measurement method, or cosine similarity measurement method.

[0165] Image similarity measurement is essentially a comparison between feature vectors. Feature vectors are treated as points in a multi-dimensional space, and the distance between points is calculated using appropriate measurement methods. Different similarity measurement methods need to be selected according to different scenarios. Available similarity measurement methods include quadratic distance, Manhattan distance, Euclidean distance, and cosine similarity.

[0166] (1) Euclidean distance is a common distance metric that provides the absolute distance between two vectors in Euclidean space. When the features extracted from an image satisfy the following two conditions: (a) the components of the feature vectors are orthogonal; and (b) the dimensions of the feature vectors are of equal importance, Euclidean distance can be used to calculate the distance between the feature vectors. The method for calculating the Euclidean distance between vectors A and B is as follows:

[0167]

[0168] In the above formula, N is the dimension of the image feature vector.

[0169] (2) Quadratic distance is mainly used in image retrieval that utilizes color histograms to extract image features because it comprehensively considers color similarity and can effectively represent the distance difference between different colors. For any two color histograms C and D, the quadratic distance D is calculated as follows:

[0170] D = (CD) T A(CD)

[0171] Where A is a symmetric matrix, A = [a ij ], a ij It expresses the similarity between colors.

[0172] (3) Manhattan distance is the sum of the distances between the line segments formed by projecting two points onto the coordinate axes. The Manhattan distance between two points usually changes as the coordinate axes change. Assuming there are two two-dimensional points i(x1,y1) and j(x2,y2), the Manhattan distance between them is calculated as follows:

[0173] D(i,j)=|x1-x2|+|y1-y2|

[0174] (4) Cosine similarity, typically used in high-dimensional feature spaces, is unrelated to the magnitude of the vectors and only measures the similarity of their directions. First, the feature vectors are mapped to the corresponding feature space. Then, the cosine of the angle between the two vectors is calculated, and the similarity between the vectors is compared. The calculation formula is as follows:

[0175]

[0176] The value of D(A,B) ranges from 0 to 1. When used for similarity measurement, a larger value indicates a higher similarity between the two images.

[0177] In this embodiment of the invention, different similarity measurement methods are selected according to different scenarios. The second image feature with a similarity exceeding a preset threshold with the first image feature is searched in the image retrieval feature library corresponding to the target user. This can effectively filter out negative images that the user does not need to be detected and improve the detection accuracy.

[0178] The target detection device provided by the present invention is described below. The target detection device described below and the target detection method described above can be referred to in correspondence.

[0179] Figure 6 This is a schematic diagram of the target detection device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes: a target detection unit 610 and a negative example filtering unit 620, wherein,

[0180] The target detection unit 610 is used to acquire an image to be detected, perform target detection on the image to be detected, and obtain a target detection result.

[0181] The negative example filtering unit 620 is used to filter the target detection results based on the image retrieval feature library corresponding to the target user, so as to obtain the detection results corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of the negative example images determined by the target user.

[0182] Optionally, the step of filtering negative examples from the target detection results based on the image retrieval feature library corresponding to the target user to obtain the detection results corresponding to the target user includes:

[0183] The target detection results are preprocessed and feature extracted to obtain the first image features corresponding to the target detection results;

[0184] Search for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold;

[0185] If the second image feature exists, the target detection result is not output; or, if the second image feature does not exist, the target detection result is output.

[0186] Optionally, the target detection device further includes: a feature library construction unit, the feature library construction unit being used for:

[0187] Obtain the negative example images determined by the target user and construct an image dataset;

[0188] The image dataset is traversed, and the preprocessing and feature extraction are performed on each image in the image dataset to obtain the feature vector or matrix corresponding to each image.

[0189] By associating the feature vector or matrix corresponding to each image with each image, an image retrieval feature library corresponding to the target user is obtained.

[0190] Optionally, the step of preprocessing and feature extraction of the target detection result to obtain the first image feature corresponding to the target detection result includes:

[0191] The target detection results are preprocessed, including: enhancing contrast and removing noise;

[0192] The preprocessed target detection results are subjected to Scale Invariant Feature Transform (SIFT) feature extraction to obtain the SIFT features corresponding to the target detection results. The preprocessed target detection results are then subjected to high-level semantic feature extraction to obtain the high-level semantic features corresponding to the target detection results.

[0193] By fusing the SIFT features and high-level semantic features, the first image feature corresponding to the target detection result is obtained.

[0194] Optionally, the step of performing Scale Invariant Feature Transform (SIFT) feature extraction on the preprocessed target detection results to obtain the SIFT features corresponding to the target detection results includes:

[0195] Constructing scale space;

[0196] The feature points in the preprocessed target detection results are determined, and the extreme points with low contrast and unstable edge response points are removed from the feature points. Then, the extreme points with significant errors are removed from the feature points by combining the random sampling consensus algorithm, and the first feature point set is obtained.

[0197] Determine the orientation of each feature point in the first feature point set, and rotate each feature point in the first feature point set to obtain the SIFT feature corresponding to the target detection result.

[0198] Optionally, the step of extracting high-level semantic features from the preprocessed target detection results to obtain the high-level semantic features corresponding to the target detection results includes:

[0199] The preprocessed target detection results are input into a CNN feature extraction network for high-level semantic feature extraction.

[0200] The output vector of the fully connected layer of the CNN feature extraction network is obtained to obtain the high-level semantic features corresponding to the target detection result.

[0201] Optionally, the step of searching for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold includes:

[0202] Based on a preset similarity measurement method, the similarity between the first image feature and the feature vector or matrix corresponding to each image in the image retrieval feature library is calculated;

[0203] Determine whether there exists a feature vector or matrix corresponding to the second image whose similarity to the features of the first image exceeds the preset threshold;

[0204] If it exists, then the feature vector or matrix corresponding to the second image is determined to be a feature of the second image;

[0205] The preset similarity measurement methods include: quadratic distance measurement method, Manhattan distance measurement method, Euclidean distance measurement method, or cosine similarity measurement method.

[0206] It should be noted that the target detection device provided in this embodiment of the invention can implement all the method steps implemented in the above-described target detection method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0207] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a target detection method, which includes: acquiring an image to be detected; performing target detection on the image to be detected to obtain a target detection result; and filtering negative examples from the target detection result based on an image retrieval feature library corresponding to the target user to obtain a detection result corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and extracting features from negative example images determined by the target user.

[0208] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0209] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target detection method provided in the above-described method embodiments. The method includes: acquiring an image to be detected; performing target detection on the image to be detected to obtain a target detection result; and filtering negative examples of the target detection result based on an image retrieval feature library corresponding to the target user to obtain a detection result corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of negative example images determined by the target user.

[0210] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the target detection method provided by the above methods. The method includes: acquiring an image to be detected; performing target detection on the image to be detected to obtain a target detection result; and filtering negative examples of the target detection result based on an image retrieval feature library corresponding to a target user to obtain a detection result corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of negative example images determined by the target user.

[0211] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0212] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method, characterized in that, include: Acquire the image to be detected, perform target detection on the image to be detected, and obtain the target detection result; Based on the image retrieval feature library corresponding to the target user, negative examples are filtered on the target detection results to obtain the detection results corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of the negative example images determined by the target user. The negative example images determined by the target user are negative example images in the target detection results subjectively judged by the target user. The step of filtering negative examples from the image retrieval feature library corresponding to the target user to obtain the detection result corresponding to the target user includes: The target detection results are preprocessed and feature extracted to obtain the first image features corresponding to the target detection results; Search for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold; If the second image feature exists, the target detection result is not output; or, if the second image feature does not exist, the target detection result is output.

2. The target detection method according to claim 1, characterized in that, The target detection method further includes: Obtain the negative example images determined by the target user and construct an image dataset; The image dataset is traversed, and the preprocessing and feature extraction are performed on each image in the image dataset to obtain the feature vector or matrix corresponding to each image. By associating the feature vector or matrix corresponding to each image with each image, an image retrieval feature library corresponding to the target user is obtained.

3. The target detection method according to claim 1 or 2, characterized in that, The step of preprocessing and feature extraction of the target detection result to obtain the first image feature corresponding to the target detection result includes: The target detection results are preprocessed, including: enhancing contrast and removing noise; The preprocessed target detection results are subjected to Scale Invariant Feature Transform (SIFT) feature extraction to obtain the SIFT features corresponding to the target detection results. The preprocessed target detection results are then subjected to high-level semantic feature extraction to obtain the high-level semantic features corresponding to the target detection results. By fusing the SIFT features and high-level semantic features, the first image feature corresponding to the target detection result is obtained.

4. The target detection method according to claim 3, characterized in that, The step of performing Scale Invariant Feature Transform (SIFT) feature extraction on the preprocessed target detection results to obtain the SIFT features corresponding to the target detection results includes: Constructing scale space; The feature points in the preprocessed target detection results are determined, and the extreme points with low contrast and unstable edge response points are removed from the feature points. Then, the extreme points with significant errors are removed from the feature points by combining the random sampling consensus algorithm, and the first feature point set is obtained. Determine the orientation of each feature point in the first feature point set, and rotate each feature point in the first feature point set to obtain the SIFT feature corresponding to the target detection result.

5. The target detection method according to claim 3, characterized in that, The step of extracting high-level semantic features from the preprocessed target detection results to obtain the high-level semantic features corresponding to the target detection results includes: The preprocessed target detection results are input into a CNN feature extraction network for high-level semantic feature extraction. The output vector of the fully connected layer of the CNN feature extraction network is obtained to obtain the high-level semantic features corresponding to the target detection result.

6. The target detection method according to claim 1, characterized in that, The step of searching for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold includes: Based on a preset similarity measurement method, the similarity between the first image feature and the feature vector or matrix corresponding to each image in the image retrieval feature library is calculated; Determine whether there exists a feature vector or matrix corresponding to the second image whose similarity to the features of the first image exceeds the preset threshold; If it exists, then the feature vector or matrix corresponding to the second image is determined to be a feature of the second image; The preset similarity measurement methods include: quadratic distance measurement method, Manhattan distance measurement method, Euclidean distance measurement method, or cosine similarity measurement method.

7. A target detection device, characterized in that, include: The target detection unit is used to acquire an image to be detected, perform target detection on the image to be detected, and obtain a target detection result. The negative example filtering unit is used to filter the target detection results based on the image retrieval feature library corresponding to the target user, so as to obtain the detection results corresponding to the target user. The image retrieval feature library corresponding to the target user is obtained by preprocessing and feature extraction of the negative example images determined by the target user. The negative example images determined by the target user are negative example images in the target detection results judged subjectively by the target user. The negative example filtering unit is specifically used for: The target detection results are preprocessed and feature extracted to obtain the first image features corresponding to the target detection results; Search for a second image feature in the image retrieval feature library corresponding to the target user whose similarity to the first image feature exceeds a preset threshold; If the second image feature exists, the target detection result is not output; or, if the second image feature does not exist, the target detection result is output.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the target detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the target detection method as described in any one of claims 1 to 6.

Images