Image detection method and device

Abstract

The embodiment of the invention provides an image detection method and device, and the method comprises the steps: obtaining a first depth feature map of image data and a plurality of second depth feature maps corresponding to a plurality of pieces of target template image data, and carrying out the matching of each second depth feature map with the first depth feature map, determining a first number of candidate regions in the image data, determining attitude information of the target template image data in the candidate regions, correcting the candidate regions according to each attitude information, obtaining a third depth feature map of each corrected candidate region and a fourth depth feature map corresponding to the target template image data, and obtaining a second depth feature map of each corrected candidate region; according to the method, the specific template image is detected in a complex environment and under the condition that no training sample exists, and the accuracy and robustness of specific template image detection are improved under the conditions of insufficient training samples, illumination, scale, rotation, view angle changes, complex backgrounds and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of AI image processing and big data analysis, and in particular to a method and device for image detection. Background Art

[0002] In today's information society, computer vision and image processing technologies have penetrated into various industries and fields. The detection and detection of specific images, such as promotional posters, trademarks, logos, etc., are of great significance in practical applications. For example, in advertising placement monitoring, the publisher needs to know whether the advertisement is exposed in the designated location as planned, as well as its exposure frequency and duration; in augmented reality (AR) applications, it is necessary to accurately detect specific images in real scenes to overlay virtual information; in image retrieval and security monitoring, detecting specific images helps to quickly locate the target and improve the efficiency and safety of the system.

[0003] However, issues such as the size of model training samples and the complexity of the real world have brought huge challenges to image detection. Insufficient training samples, illumination changes, scale changes, rotation, perspective changes, occlusion, and complex backgrounds can all affect the accuracy and robustness of image detection. How to accurately and quickly detect specific images under these complex conditions is a problem with important research value and application prospects. Summary of the invention

[0004] In view of the above problems, a method and device for image detection is proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] A method for image detection, the method comprising:

[0006] Acquire image data to be detected and multiple target template image data; wherein the multiple target template image data include multiple target template images obtained by changing an initial template image;

[0007] Acquire a first depth feature map of the image data and a plurality of second depth feature maps corresponding to a plurality of target template image data;

[0008] Determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map;

[0009] Determine, according to each candidate region and the corresponding target template image data, the posture information of the target template image data in the candidate region;

[0010] Correcting the candidate region according to each posture information;

[0011] Obtain a third depth feature map of each corrected candidate region and a fourth depth feature map of the corresponding target template image data;

[0012] A similarity calculation is performed on each third depth feature map and the corresponding fourth depth feature map, and a target detection result is obtained according to the result of the similarity calculation.

[0013] Optionally, the acquiring of a plurality of target template image data comprises:

[0014] Performing a three-dimensional rotation operation on the initial template image to obtain a plurality of first images corresponding to a plurality of angles;

[0015] Adjust each first image according to multiple brightness and contrast ratios to obtain multiple second images corresponding to each first image under multiple lighting effects;

[0016] According to each second image, a plurality of target template images corresponding to a plurality of scaling ratios are obtained.

[0017] Optionally, determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map includes:

[0018] In the depth feature space, each second depth feature map is matched with the first depth feature map by a preset template matching algorithm to obtain a matching result and a plurality of matching positions in the image data;

[0019] According to the matching result, determining a first number of preliminary detection positions from the multiple matching positions, and drawing a preliminary detection frame at each preliminary detection position;

[0020] The first number of preliminary detection frames are expanded according to a preset ratio to obtain a first number of candidate regions.

[0021] Optionally, matching each second depth feature map with the first depth feature map by a preset template matching algorithm to obtain a matching result and a plurality of matching positions in the image data includes:

[0022] Each second depth feature map is matched with the first depth feature map through a preset template matching algorithm to obtain corresponding multiple matching scores and corresponding multiple matching positions in the image data.

[0023] Optionally, determining a first number of preliminary detection positions from the multiple matching positions according to the matching result includes:

[0024] Obtaining a matching score greater than a first preset threshold and a corresponding matching position;

[0025] The matching scores greater than the first preset threshold are sorted from high to low, and the matching positions corresponding to the first number of matching scores in the front are obtained.

[0026] Optionally, determining the posture information of the target template image data in the candidate area according to each candidate area and the corresponding target template image data includes:

[0027] Extract feature points of each candidate region and corresponding target template image data;

[0028] Matching the feature points of each candidate area with the feature points of the corresponding target template image data to obtain matching feature point pairs;

[0029] According to the matched feature point pairs, the posture information of the target template image data in the candidate area is obtained; wherein the posture information includes a rotation vector and a translation vector.

[0030] Optionally, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining the target detection result according to the result of the similarity calculation, includes:

[0031] Performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map to obtain corresponding multiple similarity scores;

[0032] Acquire candidate regions corresponding to similarity scores greater than a second preset threshold, and filter them according to preset filtering rules to obtain target candidate regions;

[0033] A target detection frame is drawn in the target candidate area in the image data to obtain a target detection result.

[0034] A device for image detection, comprising:

[0035] A data acquisition module, used to acquire image data to be detected and multiple target template image data; wherein the multiple target template image data include multiple target template images obtained by changing an initial template image;

[0036] A first feature map acquisition module, used to acquire a first depth feature map of the image data and a plurality of second depth feature maps corresponding to a plurality of target template image data;

[0037] a candidate region acquisition module, configured to determine a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map;

[0038] A posture information acquisition module, used to determine the posture information of the target template image data in the candidate area according to each candidate area and the corresponding target template image data;

[0039] A correction module, used for correcting the candidate area according to each posture information;

[0040] A second feature map acquisition module, used to acquire a third depth feature map of each corrected candidate area and a fourth depth feature map corresponding to the target template image data;

[0041] The detection result acquisition module is used to perform similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtain the target detection result according to the result of the similarity calculation.

[0042] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the audio and video recording method based on quantum key as described above is implemented.

[0043] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the audio and video recording method based on quantum key as described above is implemented.

[0044] The embodiments of the present invention have the following advantages:

[0045] In an embodiment of the present invention, by acquiring image data to be detected and multiple target template image data, the multiple target template image data include multiple target template images obtained by changing an initial template image, acquiring a first depth feature map of the image data and multiple second depth feature maps corresponding to the multiple target template image data, determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map, determining posture information of the target template image data in the candidate region according to each candidate region and the corresponding target template image data, correcting the candidate region according to each posture information, acquiring a third depth feature map of each corrected candidate region and a fourth depth feature map of the corresponding target template image data, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining a target detection result according to the result of the similarity calculation, thereby realizing the detection of specific template images in complex environments and without training samples, and improving the accuracy and robustness of specific template image detection in situations such as insufficient training samples, changes in illumination, scale, rotation, viewing angle, and complex background. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0047] Figure 1 is a flowchart of the steps of an image detection method provided by some embodiments of the present invention;

[0048] Figure 2 is a flow chart of steps of a template image detection process provided by some embodiments of the present invention;

[0049] Figure 3 is a flow chart of data loading and preprocessing steps provided by some embodiments of the present invention;

[0050] Figure 4 is a flowchart of steps for extracting depth features and performing template matching provided by some embodiments of the present invention;

[0051] Figure 5 It is a flowchart of steps for performing fine matching and posture estimation, inverse affine transformation and cropping images based on SIFT features provided by some embodiments of the present invention;

[0052] Figure 6 is a flowchart of steps for calculating feature similarity and determining whether the similarity is greater than a threshold provided by some embodiments of the present invention;

[0053] Figure 7 is a flowchart of steps for performing non-maximum suppression and visualizing the results provided by some embodiments of the present invention;

[0054] Figure 8 is a schematic diagram of a preliminary detection frame provided by some embodiments of the present invention;

[0055] Fig. 9 It is a schematic diagram of the visualized relationship between target template image data and candidate region similarity calculation provided by some embodiments of the present invention;

[0056] Fig.10 is a schematic diagram of target detection results provided by some embodiments of the present invention;

[0057] Fig.11 It is a structural block diagram of an image detection device provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] At present, traditional feature matching algorithms, such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF), rely on local feature points of images and have certain scale and rotation invariance. However, the performance of these algorithms will be significantly reduced when dealing with illumination changes, perspective changes, occlusions, and complex backgrounds. This is because the detection and description of local feature points are easily affected by environmental factors, resulting in matching errors or failure to match.

[0060] Deep learning methods have made significant progress in image feature extraction and matching. Deep neural networks can learn high-level semantic features of images and are robust to changes in illumination, scale, and rotation. However, deep learning models usually require a large amount of labeled data for training, and the generalization ability of the model is limited when there is only a single target image. In addition, the complexity and computational complexity of deep models also increase the difficulty of practical applications.

[0061] Therefore, the existing technology is difficult to achieve accurate and robust detection in complex environments when only a single template image is processed, which limits its effectiveness and wide application in practical applications.

[0062] It should be noted that the SIFT algorithm is an algorithm for image processing, proposed by David Lowe in 1999 and improved in 2004. The SIFT algorithm detects key points by constructing a scale space and generates descriptors with rotation, scale and illumination invariance. In OpenCV (Open Source Computer Vision Library, an open source computer vision and machine learning software library), the SIFT_create(), detect() and compute() functions are used for key point detection and descriptor calculation, which are suitable for applications such as image classification.

[0063] SURF is an algorithm for feature detection and image matching, proposed by Herbert Bay in 2006. The SURF algorithm is an improvement on the SIFT algorithm, aiming to improve the computational speed and robustness. The SURF algorithm uses the concepts of Haar (Haar-like features, a feature description method for target detection) features and integral images, which greatly speeds up the running time of the program, while maintaining the scale-invariant and rotation-invariant properties of the SIFT algorithm, and is highly robust to changes in illumination and radiation.

[0064] ORB is a fast local feature point extraction and description algorithm proposed by Ethan Rublee et al. in 2011. The ORB algorithm combines FAST (Features from Accelerated Segment Test, a feature point detection algorithm used in image processing and computer vision) feature point detection and BRIEF (Binary Robust Independent Elementary Features, a feature description algorithm used in image processing and computer vision) feature descriptor, and improves and optimizes them. The ORB algorithm is characterized by fast speed and certain rotation invariance and illumination invariance.

[0065] In one embodiment of the present invention, a plurality of target template images are obtained based on changes made to a template image, and the image to be detected is detected by combining the global features extracted by the deep learning model and the local features of the traditional SIFT algorithm.

[0066] Reference Figure 1 , shows a flowchart of a method for image detection provided by some embodiments of the present invention, which may specifically include the following steps:

[0067] Step 101, obtaining image data to be detected and a plurality of target template image data; wherein the plurality of target template image data include a plurality of target template images obtained by changing an initial template image.

[0068] Before image detection begins, image data to be detected and multiple target template image data may be acquired first. The multiple target template image data may include multiple target template images obtained by changing an initial template image.

[0069] The image data to be processed and the specified template image, ie, the initial template image, can be obtained from the folder. The image data to be processed can be one or more real-shot images.

[0070] As an example, Figure 2As shown, after obtaining the corresponding data, the image data to be processed and the initial template image can be preprocessed, specifically, as Figure 3 As shown in the figure, for real-shot images, their edge information can be enhanced mainly through sharpening. Sharpening can strengthen the edge features in the image and make the details more obvious. Since the actual shot images may be interfered by various factors, edge sharpening can compensate for the adverse effects of some image blur or low contrast. For the initial template image, some changes can be made to it to obtain multiple target template image data.

[0071] In some embodiments of the present invention, Figure 2 As shown, the obtaining of multiple target template image data includes:

[0072] Step 11: Perform a three-dimensional rotation operation on the initial template image to obtain a plurality of first images corresponding to a plurality of angles.

[0073] The initial template image can be subjected to a three-dimensional rotation operation to obtain multiple first images corresponding to multiple angles. By generating images at different rotation angles, the model's ability to cope with perspective deformation and viewing angle deviation can be improved, so that it can more accurately match images taken at different angles in actual scenes.

[0074] Step 12: Adjust each first image according to multiple brightness and contrast ratios to obtain multiple second images corresponding to each first image under multiple lighting effects.

[0075] After acquiring multiple first images, each first image can be adjusted according to multiple brightness and contrast to obtain multiple second images corresponding to each first image under multiple lighting effects. The brightness and contrast of the initial template image are adjusted to simulate the actual shooting effect under different lighting conditions. This enhancement operation can help the target image to be effectively identified in the face of lighting changes during the image detection process.

[0076] Step 13: Acquire multiple target template images corresponding to multiple scaling ratios according to each second image.

[0077] After acquiring multiple second images, multiple target template images corresponding to multiple zoom ratios can be acquired according to each second image, that is, each second image is transformed with different zoom ratios to simulate different shooting distances.

[0078] Specifically, the initial template image can be scaled to 1 / 2 of the size of the real-world image. This operation is to ensure that real-world images of different resolutions can be covered in subsequent scaling. Then multi-scale scaling is performed according to the pre-defined scaling ratio. In practical applications, a series of scaling ratio arrays of the initial template images can be set according to the needs of the actual application, such as [2.0, 1.7, 1.4, 1.2, 1.0, 0.9, 0.8, 0.7, ..., 0.05], to adapt to the different sizes and distances that the initial template image may appear in the real-world image.

[0079] In summary, the change processing of the initial template image can adapt it to the visual differences caused by different lighting conditions and viewing angles in actual shooting. Multi-lighting and multi-angle processing not only enhances the diversity of target template image data, but also improves the adaptability of the image detection of the present invention in complex environments.

[0080] Step 102: Obtain a first depth feature map of the image data and a plurality of second depth feature maps corresponding to a plurality of target template image data.

[0081] You can use the pre-trained ResNet50 deep neural network model and remove its top fully connected layer to obtain a high-level deep feature representation. The conv4_block6_out layer is selected as the feature extraction layer. This layer has a large receptive field and high feature abstraction ability, can capture the global information of the image, and is conducive to improving the robustness of feature matching.

[0082] ResNet50 is a specific network structure in the ResNet (Residual Network) series, which consists of 50 convolutional layers and aims to solve the "degradation" problem of deep convolutional networks when the depth increases. ResNet50 introduces residual blocks, which enable the network to learn the residual between input and output, thereby effectively avoiding the problem of gradient disappearance and explosion.

[0083] Then, if Figure 4 As shown, a first depth feature map of image data and multiple second depth feature maps corresponding to multiple target template image data can be obtained through a pre-trained ResNet50 deep neural network model, i.e., a backbone network.

[0084] Step 103: Determine a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map.

[0085] Specifically, Figure 2As shown, after obtaining a first depth feature map of image data and multiple second depth feature maps corresponding to multiple target template image data, a first number of candidate regions in the image data can be determined by matching each second depth feature map with the first depth feature map.

[0086] In some embodiments of the present invention, determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map includes:

[0087] Sub-step 21, in the depth feature space, using a preset template matching algorithm, matches each second depth feature map with the first depth feature map to obtain a matching result and a plurality of matching positions in the image data.

[0088] In some embodiments of the present invention, matching each second depth feature map with the first depth feature map by a preset template matching algorithm to obtain a matching result and a plurality of matching positions in the image data includes:

[0089] Each second depth feature map is matched with the first depth feature map through a preset template matching algorithm to obtain corresponding multiple matching scores and corresponding multiple matching positions in the image data.

[0090] like Figure 4 As shown, each second depth feature map can be matched with the first depth feature map. Specifically, in the depth feature space, a template matching algorithm (such as match_template function, a template matching function) is used to perform sliding window matching on the depth feature map of each target template image (i.e., the second depth feature map) and the depth feature map of the real image (i.e., the first depth feature map), and the matching score and the best matching position are calculated, so that corresponding multiple matching scores and corresponding multiple matching positions in the image data can be obtained, that is, matching results and multiple matching positions in the image data are obtained.

[0091] Sub-step 22, determining a first number of preliminary detection positions from the multiple matching positions according to the matching results, and drawing a preliminary detection frame at each preliminary detection position.

[0092] like Figure 4 As shown, after the matching is completed, the preliminary detection results can be obtained and visualized. First, a first number of preliminary detection positions (i.e., preliminary detection results) can be determined from multiple matching positions according to the matching results, and then a preliminary detection frame can be drawn at each preliminary detection position to obtain a visualized preliminary detection result, wherein the first number can be matching positions with better matching effects.

[0093] In practical applications, according to the matching results, a green rectangular frame can be drawn at the corresponding initial matching position on the corresponding real-shot image to mark the initial template image position that is initially detected, such as Figure 8 As shown. This helps to visually observe the effect of the preliminary detection. At this point, the first stage of image detection is completed, and the detection results of the first stage of image detection, that is, the preliminary detection results, are obtained.

[0094] Sub-step 23, expanding the first number of preliminary detection frames according to a preset ratio to obtain a first number of candidate regions.

[0095] After drawing a preliminary detection frame for each preliminary detection position, the first number of preliminary detection frames may be expanded according to a preset ratio, and the areas selected by the expansion are candidate areas, so that the first number of candidate areas may be obtained.

[0096] Specifically, the area of ​​the preliminary detection frame may be expanded according to a certain ratio (eg, expanded by 1 / 2), new area coordinates may be calculated, and candidate areas may be obtained to prepare for image detection after the first stage.

[0097] In some embodiments of the present invention, determining a first number of preliminary detection positions from the plurality of matching positions according to the matching results includes:

[0098] Sub-step 31, obtaining a matching score greater than a first preset threshold and a corresponding matching position.

[0099] A first preset threshold may be set so that a matching score greater than the first preset threshold and a corresponding matching position may be obtained.

[0100] Specifically, a first preset threshold value (such as 0.2) of the matching score is set to filter out matching positions with matching scores higher than the threshold value, thereby effectively eliminating obviously mismatched areas and reducing the amount of calculation for subsequent image detection processing.

[0101] Sub-step 32: sort the matching scores that are greater than the first preset threshold from high to low, and obtain the matching positions corresponding to the matching scores that are ranked in the front by a first number.

[0102] After obtaining the matching scores greater than the first preset threshold and the corresponding matching positions, the matching scores greater than the first preset threshold can be sorted from high to low, and then the matching positions corresponding to the first number of matching scores can be selected as the first number of preliminary detection positions, that is, the preliminary detection results.

[0103] The image data may include one or more real-shot images. Each real-shot image is sorted according to the matching score, and the top 10 matching positions with the highest scores are retained. Then, the preliminary detection results of all real-shot images are summarized and saved in the specified result file for subsequent fine matching and verification.

[0104] Step 104 : determining the posture information of the target template image data in the candidate region according to each candidate region and the corresponding target template image data.

[0105] After obtaining the first number of candidate regions, Figure 2 As shown, the posture information of the target template image data in the corresponding candidate area can be determined based on each candidate area and the corresponding target template image data.

[0106] In some embodiments of the present invention, Figure 5 As shown, the step of determining the posture information of the target template image data in the candidate area according to each candidate area and the corresponding target template image data includes:

[0107] Sub-step 41, extracting feature points of each candidate region and the corresponding target template image data.

[0108] Feature points in each candidate area and feature points in the corresponding target template image data can be extracted.

[0109] Specifically, the candidate area can be cropped from the real shot image, SIFT feature points and feature descriptors can be extracted for each candidate area, and SIFT feature points and feature descriptors can also be extracted for the target template image data corresponding to each candidate area. Extracting SIFT features includes extracting SIFT feature points and feature descriptors for the target template image. SIFT features are invariant to scale and rotation, can capture local key point information of the image, and can provide support for fine matching.

[0110] Sub-step 42, matching the feature points of each candidate region with the feature points of the corresponding target template image data to obtain matching feature point pairs.

[0111] After obtaining the feature points of each candidate region and the feature points of the corresponding target template image data, the feature points of each candidate region may be matched with the feature points of the corresponding target template image data to obtain matched feature point pairs.

[0112] Specifically, the FLANN (Fast Library for Approximate Nearest Neighbors) matcher or BFMatcher (Brute-Force Matcher, an image matching algorithm based on brute-force matching) can be used to match the target template image data with the SIFT features of the corresponding candidate area, and a ratio test (such as Lowe's ratio test) can be applied to screen out excellent matching points (i.e., matching SIFT feature point pairs) to eliminate false matches.

[0113] FLANN can support a variety of search algorithms, including kd-tree and octree, etc. FLANN is mainly used for searching high-dimensional data, especially in the fields of image processing and machine learning, where it can quickly find the approximate nearest neighbors between data points.

[0114] BFMatcher can find the similarity between two images by comparing their feature points, and then match them based on the similarity. The working principle of this algorithm is to first select a feature point, then find other feature points in the image that are closest to it, and so on, until all matching feature points are found.

[0115] Sub-step 43, obtaining the posture information of the target template image data in the candidate area according to the matched feature point pairs; wherein the posture information includes a rotation vector and a translation vector.

[0116] Specifically, the number of valid matching points can be counted first. If the number of matching points is sufficient, usually greater than 4, the matching points are considered reliable and the candidate area is skipped. Then, the solvePnPRansac method (a common function in a computer vision library) is used to calculate the pose information of the corresponding target template image data in the candidate area based on the matched feature point pairs, combined with the PnP (Perspective-n-Point, a pose estimation method) algorithm and the RANSAC (Random Sample Consensus) algorithm, that is, the pose information in the real shot image, including the rotation vector and the translation vector.

[0117] The introduction of the RANSAC (Random Sample Consensus) algorithm can remove abnormal matching points and enhance the robustness of posture estimation (i.e., posture information).

[0118] The PNP algorithm is a commonly used attitude estimation method in computer vision. Its core function is to estimate the attitude of the camera based on known three-dimensional space points and their corresponding positions on the two-dimensional image. Specifically, the PNP algorithm calculates the rotation matrix and translation vector of the camera through a set of n 3D space points and their corresponding 2D points on the image, as well as the camera's intrinsic parameter matrix, thereby determining the position and orientation of the camera.

[0119] At the same time, it should be noted that the aforementioned operation of expanding the first number of preliminary detection frames according to a preset ratio can ensure that more possible matching feature points are included, thereby increasing the stability of obtaining matching points. Moreover, the first number of candidate regions can be cropped out.

[0120] Step 105: Correct the candidate region according to each posture information.

[0121] like Figure 2 and Figure 5 As shown in the figure, after obtaining the posture information of each target template image data, the affine transformation matrix can be calculated according to each posture information, and the corresponding cropped candidate area can be inversely transformed to correct it to the same perspective and scale as the template image. This can eliminate the influence of perspective and perspective transformation, and facilitate similarity verification in the subsequent steps of image detection.

[0122] At this step, the second stage of image detection can be considered to be completed, and the remaining steps can be considered to be the third stage of image detection.

[0123] Step 106, obtaining a third depth feature map of each corrected candidate region and a fourth depth feature map corresponding to the target template image data.

[0124] like Figure 6 As shown, the pre-trained ResNet50 model can be used again to extract the third depth feature map of the corrected candidate area and the fourth depth feature map of the corresponding target template image, respectively, and ensure that the features are compared in the same feature space.

[0125] It is worth noting that each corrected candidate region has the same size and orientation (i.e., angle) as the corresponding target template image data, and may only differ due to factors such as lighting. In actual operation, due to occlusion, matching errors, etc., there will be a certain degree of deviation.

[0126] Step 107, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining a target detection result according to the result of the similarity calculation.

[0127] like Figure 2 and Figure 6As shown, after obtaining the third depth feature map of each corrected candidate area and the fourth depth feature map of the corresponding target template image data, a similarity calculation can be performed on each third depth feature map and the corresponding fourth depth feature map to obtain the result of the similarity calculation, and based on the result of the similarity calculation, the target detection result is obtained.

[0128] In some embodiments of the present invention, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining the target detection result according to the result of the similarity calculation, includes:

[0129] Sub-step 51, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map to obtain corresponding multiple similarity scores.

[0130] In the deep feature space, cosine similarity or template matching method can be used to calculate the similarity score between the corrected candidate region and the corresponding target template image data, that is, the similarity score between the third deep feature map and the corresponding fourth deep feature map, so as to obtain corresponding multiple similarity scores. Cosine similarity can measure the directional consistency of two feature vectors. The higher the score, the more similar the two are. Its visualization effect is as follows: Fig. 9 shown.

[0131] Sub-step 52, obtaining candidate regions corresponding to similarity scores greater than a second preset threshold, and filtering according to preset filtering rules to obtain target candidate regions.

[0132] like Figure 2 and Figure 6 As shown, a second preset threshold value (such as 0.3) of the similarity score can be set. If the similarity score is higher than the threshold, the corresponding corrected candidate area is considered to be a valid detection or a valid match, and the relevant information is recorded; otherwise, the corresponding corrected candidate area is considered to be an invalid detection, that is, the target template image data is not tested, and the candidate area needs to be discarded.

[0133] Therefore, the corrected candidate area corresponding to the similarity score greater than the second preset threshold can be obtained first, that is, the corrected candidate area considered valid can be obtained. The corrected candidate area is obtained by cutting out the candidate area and then correcting it, so the effective detection can be projected onto the corresponding candidate area of ​​the image data.

[0134] Then, if Figure 7As shown, filtering can be performed according to preset filtering rules, non-maximum suppression can be performed, the best candidate regions can be retained, and repeated detection and overlapping candidate regions can be avoided. That is, for each valid candidate region in the image data, the IOU (Intersection over Union) with other valid candidate regions can be calculated. That is, IOU screening, IOU screening is that if the IOU of two candidate regions is greater than a set third preset threshold (such as 0.5), the candidate region with a higher matching score is retained.

[0135] And perform area difference judgment, that is, if the area difference between the two detection frames is large, and the intersection part accounts for a large proportion (such as more than 90%), then retain the detection frame with a larger area to reduce false detection, that is, avoid the situation where one detection frame is completely covered by another frame, and ensure that the detection result is more concise and accurate.

[0136] After filtering according to the preset filtering rules, the target candidate region can be obtained, that is, the target template image data is detected to be included in the image data to be detected. There may also be a situation where the target candidate region is not obtained, that is, the target template image data is not detected.

[0137] Sub-step 53, drawing a target detection frame in the target candidate area in the image data to obtain a target detection result.

[0138] When the target candidate region is obtained, the target detection frame can be drawn in the target candidate region in the image data, that is, Fig.10 As shown, a red rectangular box is drawn on the real image to mark the final detection result, that is, the target detection result, so that the exact position of the template image in the real image can be intuitively displayed.

[0139] In one example, the real-shot image with the target detection frame drawn can be saved in the final_results folder, and the file name is consistent with the original image data for easy retrieval and viewing. The method for handling no detection results is to directly copy the original image data to the final_results folder for the target template image data that is not detected, indicating that the target template image data is not found in these image data.

[0140] The method of the present invention has a wide range of application scenarios, mainly including but not limited to the following fields:

[0141] Advertisement placement monitoring: During the advertisement placement process, the advertiser needs to ensure that the advertisement content appears at the designated location and time as planned. This method can detect the content such as promotional posters, trademarks and advertising materials in the actual scene in real time, determine whether they are correctly placed, and count the exposure frequency and duration.

[0142] Augmented reality (AR) applications: In AR applications, it is necessary to accurately identify specific images in real scenes in order to overlay virtual information. This method can accurately locate the target template image in complex environments, improving the accuracy of the AR experience and the user interaction effect.

[0143] Image retrieval and security monitoring: In large public places such as airports, stations and shopping malls, security personnel need to quickly locate specific signs. This method can help quickly retrieve target images and improve the efficiency of security monitoring systems.

[0144] Autonomous driving and robot vision: In autonomous vehicles or service robots, it is necessary to recognize road signs, signs or other visual information. This method can enhance the robustness of the machine vision system and improve the recognition accuracy.

[0145] In an embodiment of the present invention, by acquiring image data to be detected and multiple target template image data, the multiple target template image data include multiple target template images obtained by changing an initial template image, acquiring a first depth feature map of the image data and multiple second depth feature maps corresponding to the multiple target template image data, determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map, determining posture information of the target template image data in the candidate region according to each candidate region and the corresponding target template image data, correcting the candidate region according to each posture information, acquiring a third depth feature map of each corrected candidate region and a fourth depth feature map of the corresponding target template image data, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining a target detection result according to the result of the similarity calculation, thereby realizing the detection of specific template images in complex environments and without training samples, and improving the accuracy and robustness of specific template image detection in situations such as insufficient training samples, changes in illumination, scale, rotation, viewing angle, and complex background.

[0146] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0147] Reference Fig.11 , shows a schematic diagram of the structure of an image detection device provided by some embodiments of the present invention, which may specifically include the following modules:

[0148] The data acquisition module 111 is used to acquire the image data to be detected and a plurality of target template image data; wherein the plurality of target template image data includes a plurality of target template images obtained by changing an initial template image;

[0149] A first feature map acquisition module 112, used to acquire a first depth feature map of the image data and a plurality of second depth feature maps corresponding to a plurality of target template image data;

[0150] A candidate region acquisition module 113, configured to determine a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map;

[0151] A posture information acquisition module 114, configured to determine the posture information of the target template image data in the candidate area according to each candidate area and the corresponding target template image data;

[0152] A correction module 115, configured to correct the candidate region according to each posture information;

[0153] A second feature map acquisition module 116, configured to acquire a third depth feature map of each corrected candidate region and a fourth depth feature map corresponding to the target template image data;

[0154] The detection result acquisition module 117 is used to perform similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtain the target detection result according to the result of the similarity calculation.

[0155] In one embodiment of the present invention, the data acquisition module 111 includes:

[0156] A multi-angle image acquisition submodule, used to perform a three-dimensional rotation operation on the initial template image to obtain a plurality of first images corresponding to a plurality of angles;

[0157] Multiple illumination image acquisition submodules, used to adjust each first image according to multiple brightness and contrast, to obtain multiple second images corresponding to each first image under multiple illumination effects;

[0158] The multiple zoom ratio image acquisition module is used to acquire multiple target template images corresponding to multiple zoom ratios according to each second image.

[0159] In one embodiment of the present invention, the candidate region acquisition module 113 includes:

[0160] A matching result and position acquisition submodule, used to match each second depth feature map with the first depth feature map in the depth feature space by a preset template matching algorithm to obtain a matching result and a plurality of matching positions in the image data;

[0161] a preliminary detection position determination submodule, configured to determine a first number of preliminary detection positions from the plurality of matching positions according to the matching result, and draw a preliminary detection frame at each preliminary detection position;

[0162] The candidate region acquisition submodule is used to expand the first number of preliminary detection frames according to a preset ratio to obtain a first number of candidate regions.

[0163] In one embodiment of the present invention, the matching result and position acquisition submodule includes:

[0164] A matching score and position acquisition unit is used to match each second depth feature map with the first depth feature map through a preset template matching algorithm to obtain corresponding multiple matching scores and corresponding multiple matching positions in the image data.

[0165] In one embodiment of the present invention, the preliminary detection position determination submodule includes:

[0166] A first preset threshold comparison unit, used to obtain a matching score greater than a first preset threshold and a corresponding matching position;

[0167] The matching position determining unit is used to sort the matching scores greater than the first preset threshold from high to low, and obtain the matching positions corresponding to the matching scores that are ranked in the front by a first number.

[0168] In one embodiment of the present invention, the posture information acquisition module 114 includes:

[0169] A feature point extraction submodule is used to extract feature points of each candidate area and the corresponding target template image data;

[0170] A feature point pair acquisition submodule is used to match the feature points of each candidate area with the feature points of the corresponding target template image data to obtain matching feature point pairs;

[0171] The posture information acquisition submodule is used to obtain the posture information of the target template image data in the candidate area according to the matched feature point pairs; wherein the posture information includes a rotation vector and a translation vector.

[0172] In one embodiment of the present invention, the detection result acquisition module 107 includes:

[0173] A similarity score calculation submodule, used to perform similarity calculation on each third depth feature map and the corresponding fourth depth feature map to obtain corresponding multiple similarity scores;

[0174] A target candidate region acquisition submodule is used to acquire candidate regions corresponding to similarity scores greater than a second preset threshold value, and filter them according to preset filtering rules to obtain target candidate regions;

[0175] The detection result acquisition submodule is used to draw a target detection frame in the target candidate area in the image data to obtain a target detection result.

[0176] In an embodiment of the present invention, by acquiring image data to be detected and multiple target template image data, the multiple target template image data include multiple target template images obtained by changing an initial template image, acquiring a first depth feature map of the image data and multiple second depth feature maps corresponding to the multiple target template image data, determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map, determining posture information of the target template image data in the candidate region according to each candidate region and the corresponding target template image data, correcting the candidate region according to each posture information, acquiring a third depth feature map of each corrected candidate region and a fourth depth feature map of the corresponding target template image data, performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining a target detection result according to the result of the similarity calculation, thereby realizing the detection of specific template images in complex environments and without training samples, and improving the accuracy and robustness of specific template image detection in situations such as insufficient training samples, changes in illumination, scale, rotation, viewing angle, and complex background.

[0177] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the above method is implemented when the computer program is executed by the processor.

[0178] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0179] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above method when executed by a processor.

[0180] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0181] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0182] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0183] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0184] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0187] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0188] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0189] The above is a detailed introduction to the provided image detection method and device. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for image detection, characterized in that: The method comprises: Acquire image data to be detected and multiple target template image data; wherein the multiple target template image data include multiple target template images obtained by changing an initial template image; Acquire a first depth feature map of the image data and a plurality of second depth feature maps corresponding to a plurality of target template image data; Determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map; Determine, according to each candidate region and the corresponding target template image data, the posture information of the target template image data in the candidate region; Correcting the candidate region according to each posture information; Obtain a third depth feature map of each corrected candidate region and a fourth depth feature map of the corresponding target template image data; A similarity calculation is performed on each third depth feature map and the corresponding fourth depth feature map, and a target detection result is obtained according to the result of the similarity calculation.

2. The method according to claim 1, characterized in that The step of acquiring a plurality of target template image data comprises: Performing a three-dimensional rotation operation on the initial template image to obtain a plurality of first images corresponding to a plurality of angles; Adjust each first image according to multiple brightness and contrast ratios to obtain multiple second images corresponding to each first image under multiple lighting effects; According to each second image, a plurality of target template images corresponding to a plurality of scaling ratios are obtained.

3. The method according to claim 1, characterized in that The determining a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map comprises: In the depth feature space, each second depth feature map is matched with the first depth feature map by a preset template matching algorithm to obtain a matching result and a plurality of matching positions in the image data; According to the matching result, determining a first number of preliminary detection positions from the multiple matching positions, and drawing a preliminary detection frame at each preliminary detection position; The first number of preliminary detection frames are expanded according to a preset ratio to obtain a first number of candidate regions.

4. The method according to claim 3, characterized in that The method of matching each second depth feature map with the first depth feature map by a preset template matching algorithm to obtain a matching result and a plurality of matching positions in the image data includes: Each second depth feature map is matched with the first depth feature map through a preset template matching algorithm to obtain corresponding multiple matching scores and corresponding multiple matching positions in the image data.

5. The method according to claim 4, characterized in that Determining a first number of preliminary detection positions from the plurality of matching positions according to the matching results includes: Obtaining a matching score greater than a first preset threshold and a corresponding matching position; The matching scores greater than the first preset threshold are sorted from high to low, and the matching positions corresponding to the first number of matching scores in the front are obtained.

6. The method according to claim 1, characterized in that The step of determining the posture information of the target template image data in the candidate area according to each candidate area and the corresponding target template image data includes: Extract feature points of each candidate region and corresponding target template image data; Matching the feature points of each candidate area with the feature points of the corresponding target template image data to obtain matching feature point pairs; According to the matched feature point pairs, the posture information of the target template image data in the candidate area is obtained; wherein the posture information includes a rotation vector and a translation vector.

7. The method according to any one of claims 1 to 6, characterized in that: The performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtaining the target detection result according to the result of the similarity calculation, includes: Performing similarity calculation on each third depth feature map and the corresponding fourth depth feature map to obtain corresponding multiple similarity scores; Acquire candidate regions corresponding to similarity scores greater than a second preset threshold, and filter them according to preset filtering rules to obtain target candidate regions; A target detection frame is drawn in the target candidate area in the image data to obtain a target detection result.

8. An image detection device, characterized in that: The device comprises: A data acquisition module, used to acquire image data to be detected and multiple target template image data; wherein the multiple target template image data include multiple target template images obtained by changing an initial template image; A first feature map acquisition module, used to acquire a first depth feature map of the image data and a plurality of second depth feature maps corresponding to a plurality of target template image data; a candidate region acquisition module, configured to determine a first number of candidate regions in the image data by matching each second depth feature map with the first depth feature map; A posture information acquisition module, used to determine the posture information of the target template image data in the candidate area according to each candidate area and the corresponding target template image data; A correction module, used for correcting the candidate area according to each posture information; A second feature map acquisition module, used to acquire a third depth feature map of each corrected candidate area and a fourth depth feature map corresponding to the target template image data; The detection result acquisition module is used to perform similarity calculation on each third depth feature map and the corresponding fourth depth feature map, and obtain the target detection result according to the result of the similarity calculation.

9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the image detection method according to any one of claims 1 to 7 is implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the image detection method according to any one of claims 1 to 7 is implemented.

Images