Position detection method and device for tiny target object in image, computer equipment, readable storage medium and program product

Abstract

The invention relates to a position detection method and device for a tiny target object in an image, equipment, a storage medium and a program product, and relates to the technical field of computer vision. By adopting the method, the detection precision and accuracy of the tiny target object can be improved. The method comprises the following steps: identifying a large object logically related to a tiny target object in a to-be-detected image according to scene description information; inputting the target image of the large object into a convolutional neural network to obtain a feature map, and determining a plurality of first candidate regions in the feature map; carrying out confidence coefficient prediction on the plurality of first candidate regions, sorting the first candidate regions according to confidence coefficient scores, and screening out a plurality of second candidate regions of which the confidence coefficient scores meet a score threshold condition; predicting the probability and the position of a tiny target object in the second candidate area, and determining a target area in the second candidate area according to a prediction result; and performing boundary regression in the target area to obtain target position information of the tiny target object.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, apparatus, computer device, computer-readable storage medium and computer program product for detecting the position of a tiny target object in an image. Background Art

[0002] In computer vision technology, the detection of tiny objects is of great significance. For example, in an inflammable and explosive industrial environment, timely detection of fire sources such as cigarette butts, matches or lighters produced by smoking and generating early warnings can reduce casualties and property losses caused by fires.

[0003] Traditional detection methods usually use methods such as manual local cropping, stretching, and magnification of images to retain only tiny target objects in the detected image. However, this method may reduce detection precision and accuracy due to image deformation, low pixel count, or failure to consider actual scene applications. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for detecting the position of tiny target objects in an image in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting the position of a tiny target object in an image, comprising:

[0006] Acquire an image to be detected and scene description information of the image to be detected, and identify a large object logically related to a small target object in the image to be detected according to the scene description information;

[0007] Inputting a target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determining a plurality of first candidate regions in the feature map;

[0008] Performing confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sorting the first candidate regions according to the confidence scores, and screening out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions;

[0009] Predicting the probability and position of the tiny target object in a plurality of the second candidate regions, and determining a target region in the second candidate regions according to the obtained prediction results;

[0010] Boundary regression is performed in the target area to obtain target position information of the tiny target object.

[0011] In one of the embodiments, determining a plurality of first candidate regions in the feature map includes:

[0012] Use a pre-trained convolutional layer to perform feature analysis on the feature map to obtain a bounding box regression coefficient for each target position in the feature map and an anchor point offset for the first candidate region; generate multiple anchor points at each target position based on the anchor point offset; adjust the multiple anchor points based on the bounding box regression coefficient, and use the adjusted anchor points as the first candidate region.

[0013] In one embodiment, after sorting the first candidate regions according to the confidence scores, the method further includes:

[0014] The non-maximum suppression technique is used to calculate the degree of overlap of each of the first candidate regions; in each of the first candidate regions, highly overlapping regions whose degree of overlap exceeds a preset overlap threshold are identified, and the highly overlapping regions are removed.

[0015] In one embodiment, predicting the probability and position of the tiny target object in a plurality of the second candidate regions, and determining the target region in the second candidate regions according to the obtained prediction results, includes:

[0016] Input a number of the second candidate areas into a pre-trained deep learning model, and obtain the probability prediction value and position prediction information of the existence of the tiny target object output by the deep learning model as the prediction result; sort the second candidate areas according to the probability prediction value, and screen out the target area whose probability prediction value meets the probability threshold condition from the sorted second candidate areas.

[0017] In one embodiment, performing boundary regression and feature classification in the target area to obtain target position information of the tiny target object includes:

[0018] According to the position prediction information of the tiny target object in the target area, the boundary box where the target area is located is adjusted using boundary regression technology; feature extraction is performed on the adjusted target area to obtain feature extraction results; and according to the feature extraction results, the target position information of the tiny target object is determined.

[0019] In one embodiment, determining the target position information of the tiny target object according to the feature extraction result includes:

[0020] According to the feature extraction result, the adjusted target area is feature classified, and the classification score corresponding to the target area after feature classification is obtained; the target areas after feature classification are sorted according to the classification scores, and the current target area with the highest classification score is screened out; in the current target area, the target position information of the tiny target object is determined.

[0021] In a second aspect, the present application also provides a device for detecting the position of a tiny target object in an image, comprising:

[0022] An object recognition module is used to obtain an image to be detected and scene description information of the image to be detected, and to recognize a large object logically related to a small target object in the image to be detected according to the scene description information;

[0023] A feature extraction module, used for inputting a target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determining a plurality of first candidate regions in the feature map;

[0024] A first screening module is used to perform confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and screen out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions;

[0025] A second screening module is used to predict the probability and position of the tiny target object in a plurality of the second candidate regions, and determine the target region in the second candidate regions according to the obtained prediction results;

[0026] The position determination module is used to perform boundary regression in the target area to obtain the target position information of the tiny target object.

[0027] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0028] Acquire an image to be detected and scene description information of the image to be detected, and identify a large object logically related to a tiny target object in the image to be detected according to the scene description information; input the target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determine a plurality of first candidate regions in the feature map; perform confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and screen out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions; predict the probability and position of the tiny target object in a plurality of the second candidate regions, and determine a target region in the second candidate regions according to the obtained prediction results; perform boundary regression in the target region to obtain target position information of the tiny target object.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0030] Acquire an image to be detected and scene description information of the image to be detected, and identify a large object logically related to a tiny target object in the image to be detected according to the scene description information; input the target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determine a plurality of first candidate regions in the feature map; perform confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and screen out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions; predict the probability and position of the tiny target object in a plurality of the second candidate regions, and determine a target region in the second candidate regions according to the obtained prediction results; perform boundary regression in the target region to obtain target position information of the tiny target object.

[0031] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0032] Acquire an image to be detected and scene description information of the image to be detected, and identify a large object logically related to a tiny target object in the image to be detected according to the scene description information; input the target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determine a plurality of first candidate regions in the feature map; perform confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and screen out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions; predict the probability and position of the tiny target object in a plurality of the second candidate regions, and determine a target region in the second candidate regions according to the obtained prediction results; perform boundary regression in the target region to obtain target position information of the tiny target object.

[0033] The above-mentioned method, device, computer equipment, computer-readable storage medium and computer program product for detecting the position of a tiny target object in an image analyzes scene description information to identify large objects in the image that have a logical relationship with the tiny target, and uses a pre-trained convolutional neural network to extract the features of large objects and tiny targets in the image. In the region proposal stage, large objects that are logically related to the tiny target object are labeled, and then the feature map is analyzed using a pre-trained convolutional layer to generate a series of candidate regions for the large object; in the classification stage, the candidate region is labeled and classified to obtain the probability of whether the candidate box contains the target object; and its position is optimized through boundary regression to obtain more accurate object position information; finally, the final detection of the target object is completed by combining relational reasoning and contextual verification. In summary, the entire application scheme comprehensively utilizes the two-stage detection of neural networks, physical position relationships, behavioral relationships, natural logical relationships and contextual relationships to detect tiny target objects in images, thereby effectively improving the detection precision and accuracy of tiny target objects in complex scenes of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] FIG1 is a diagram showing an application environment of a method for detecting a position of a tiny target object in an image according to an embodiment;

[0036] Figure 2is a schematic flow chart of a method for detecting the position of a tiny target object in an image in one embodiment;

[0037] Figure 3 A schematic diagram of a flow chart of a step of determining a first candidate region in an embodiment;

[0038] Figure 4 A schematic diagram of a process for obtaining target location information in one embodiment;

[0039] Figure 5 is a flow chart of a method for detecting the position of a tiny target object in an image in a specific embodiment;

[0040] Figure 6 A schematic diagram of the steps of a method for detecting the position of a tiny target object in an image in a specific embodiment;

[0041] Figure 7 is a structural block diagram of a device for detecting the position of a tiny target object in an image in one embodiment;

[0042] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The method for detecting the position of a tiny target object in an image provided by an embodiment of the present application can be applied in an application environment as shown in FIG1. ​​The terminal communicates with the server via a network. The data storage system can store data that the server needs to process. The data storage system can be integrated on the server or placed on a cloud or other network server.

[0045] Specifically, the method for detecting the position of a tiny target object in an image provided by an embodiment of the present application can be executed by a server.

[0046] Exemplarily, the server obtains an image to be detected and scene description information of the image to be detected, and identifies a large object that is logically related to the tiny target object in the image to be detected according to the scene description information; the server inputs the target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determines multiple first candidate areas in the feature map; the server performs confidence prediction on the multiple first candidate areas to obtain corresponding confidence scores, sorts the first candidate areas according to the confidence scores, and screens out several second candidate areas whose confidence scores meet the score threshold condition from the sorted first candidate areas; the server predicts the probability and position of the presence of tiny target objects in several second candidate areas, and determines the target area in the second candidate areas according to the obtained prediction results; the server performs boundary regression and feature classification in the target area to obtain the target position information of the tiny target object.

[0047] In the application environment shown in FIG1 , the terminal may be, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers. The server may be implemented as an independent server or a server cluster consisting of multiple servers.

[0048] In one embodiment, Figure 2 As shown, a method for detecting the position of a tiny target object in an image is provided, and the method is applied to the server in FIG1 as an example for explanation, and includes the following steps:

[0049] Step S201 , obtaining an image to be detected and scene description information of the image to be detected, and identifying a large object logically related to a small target object in the image to be detected according to the scene description information.

[0050] It should be noted that, considering that in real business scenarios, there are often inseparable relationships between things, this application sorts out the inevitable related relationships for tiny target objects based on the "logical association" principle, so as to determine the large objects that need to be detected in the first stage.

[0051] Among them, the scene description information can be the association information between different things in a real business scene. For example, in a computer room environment where smoking is prohibited to avoid fire, it is necessary to detect operators who smoke in violation of the regulations. For the cigarette being sucked in this scene, the large object associated with the tiny target object "cigarette" can be determined to be a person through the specific smoking methods of smokers, such as holding cigarettes in hands, clamping cigarettes in ears, and smoking with mouths.

[0052] Specifically, the server obtains the image to be detected and the scene description information of the image to be detected, sorts out the necessarily related associations for the tiny target object based on the scene description information, and identifies the large object logically related to the tiny target object in the image to be detected according to the associations.

[0053] Step S202: input the target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determine multiple first candidate regions in the feature map.

[0054] Among them, the pre-trained convolutional neural network can be a feature extractor in Faster R-CNN (Faster Region-Based Convolutional Neural Network, a deep learning model for target detection), such as VGG16, ResNet, etc.

[0055] Specifically, the target image corresponding to the large object of the server is input into the pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network. A small convolution layer is used in the feature map to further process the features and predict the bounding box regression coefficient and the offset of the candidate box relative to the anchor point (i.e., the center offset of the x, y coordinates and the change in width and height) for each position; at each position in the feature map, multiple anchor points, i.e., potential candidate boxes of different sizes and shapes, are generated according to the preset anchor point size and aspect ratio as the first candidate area.

[0056] Step S203, confidence prediction is performed on multiple first candidate regions to obtain corresponding confidence scores, the first candidate regions are sorted according to the confidence scores, and several second candidate regions whose confidence scores meet the score threshold condition are screened out from the sorted first candidate regions.

[0057] Among them, the confidence score can be an indicator used to measure the possibility of a candidate region containing a target object in the target detection task.

[0058] The score threshold condition may be set by those skilled in the art according to actual conditions.

[0059] Specifically, the server uses the Faster R-CNN model to predict the confidence of multiple first candidate regions, obtains the confidence scores of each first candidate region generated by the output layer of the classifier, sorts the first candidate regions in descending order according to the confidence scores, and screens out several second candidate regions whose confidence scores meet the score threshold conditions from the sorted first candidate regions.

[0060] Step S204: predicting the probability and position of the tiny target object in a plurality of second candidate regions, and determining the target region in the second candidate regions according to the obtained prediction results.

[0061] Specifically, the server inputs several of the second candidate areas into a pre-trained deep learning model, obtains the probability prediction value and position prediction information of the existence of the tiny target object output by the deep learning model as the prediction result; sorts the second candidate areas according to the probability prediction value, and screens out the target area whose probability prediction value meets the probability threshold condition from the sorted second candidate areas.

[0062] Among them, the pre-trained deep learning model can be a convolutional neural network (CNN or ConvNet).

[0063] Among them, the probability threshold condition can be set by those skilled in the art according to actual conditions.

[0064] Step S205 , performing boundary regression and feature classification in the target area to obtain target position information of the tiny target object.

[0065] Among them, boundary regression is a key technology in target detection, which is mainly used to accurately adjust the candidate region to better surround tiny target objects. It is achieved by learning the transformation from the anchor point to the real bounding box.

[0066] Among them, feature classification is the process of analyzing and extracting the features of the target area and assigning these target areas to different categories using a specific classification algorithm, such as the classification of foreground and background.

[0067] Specifically, the server uses ROI Pooling or ROI Align to extract features in the target area, classifies these target areas using the extracted features, and further regresses the positions of these target areas to obtain more accurate target position information.

[0068] In the above-mentioned method for detecting the position of tiny target objects in the image, by analyzing the scene description information, the large objects in the image that have a logical relationship with the tiny target are identified, and the pre-trained convolutional neural network is used to extract the features of the large objects and tiny targets in the image. In the region proposal stage, the large objects that are logically related to the tiny target objects are labeled, and then the feature map is analyzed using the pre-trained convolutional layer to generate a series of candidate regions for the large objects; in the classification stage, the candidate regions are labeled and classified to obtain the probability of whether the candidate box contains the target object; and its position is optimized through boundary regression to obtain more accurate object position information; finally, the final detection of the target object is completed by combining relational reasoning and contextual verification. In summary, the entire application scheme comprehensively utilizes the two-stage detection of neural networks, physical position relationships, behavioral relationships, natural logical relationships, and contextual relationships to detect tiny target objects in images, thereby effectively improving the detection precision and accuracy of tiny target objects in complex scenes of image detection.

[0069] In one embodiment, if Figure 3 As shown, in the above step S201, multiple first candidate regions are determined in the feature map, which specifically includes the following steps:

[0070] Step S301, using a pre-trained convolutional layer to perform feature analysis on the feature map, to obtain the bounding box regression coefficient of each target position in the feature map and the anchor point offset of the first candidate region.

[0071] Step S302: generating multiple anchor points at each target position according to the anchor point offset.

[0072] Step S303: adjust multiple anchor points according to the bounding box regression coefficients, and use the adjusted anchor points as the first candidate region.

[0073] Among them, the anchor point can be a predefined rectangular box, which is used to generate a candidate region to detect tiny target objects in the image.

[0074] Specifically, the server performs feature analysis by sliding a small convolutional layer on the feature map to obtain the bounding box regression coefficients of each target position in the feature map and the anchor point offset of the first candidate region (i.e., the center offset of the x, y coordinates and the changes in width and height). At each position in the feature map, multiple anchor points, i.e., potential candidate boxes of different sizes and shapes, are generated according to the anchor point offset. The bounding boxes of multiple anchor points are adjusted according to the predicted bounding box regression coefficients, and the adjusted anchor points are used as the first candidate region.

[0075] In this embodiment, a small convolutional layer is slid on the feature map to generate a set of anchor points, and it is predicted whether each anchor point contains the target and its corresponding bounding box adjustment, and finally multiple first candidate regions are accurately determined, which helps to improve the detection accuracy of tiny target objects.

[0076] In one embodiment, after sorting the first candidate regions according to the confidence scores, the solution of the present application further includes the following steps:

[0077] The non-maximum suppression technique is used to calculate the degree of overlap of each first candidate region; in each first candidate region, highly overlapping regions whose degree of overlap exceeds a preset overlap threshold are identified and removed.

[0078] Among them, non-maximum suppression is a post-processing technique commonly used in target detection tasks. It is used to remove redundant candidate regions to retain the most relevant detection results. It is mainly used to solve the problem of generating multiple overlapping regions on the same target to ensure that each target has only one final detection region.

[0079] Specifically, the server sorts the anchor points according to their scores, uses non-maximum suppression (NMS) to remove highly overlapping candidate regions, and selects candidate regions with high scores as the final output detection results.

[0080] In this embodiment, by removing candidate regions with high overlap, NMS significantly reduces the number of candidate regions, and retaining candidate regions with high scores can effectively improve the accuracy and usability of detection.

[0081] In one embodiment, in the above step S204, the probability and position of the existence of the tiny target object in the plurality of second candidate regions are predicted, and the target region is determined in the second candidate regions according to the obtained prediction results, which specifically includes the following steps:

[0082] Several second candidate regions are input into a pre-trained deep learning model, and the probability prediction value and position prediction information of the existence of tiny target objects are obtained as prediction results output by the deep learning model; the second candidate regions are sorted according to the probability prediction values, and the target regions whose probability prediction values ​​meet the probability threshold conditions are screened out from the sorted second candidate regions.

[0083] Among them, the pre-trained deep learning model can be a convolutional neural network (CNN or ConvNet), which is mainly used for image processing and computer vision tasks such as image classification, target detection, image segmentation, etc.

[0084] Specifically, the server slides a small convolutional neural network (ConvNe) within the detection range of several second candidate areas to predict the probability of the existence of tiny target objects and the location of the objects, and screens out target areas whose probability prediction values ​​meet the probability threshold conditions based on the predicted probabilities or scores.

[0085] In this embodiment, by simulating the working principle of the biological visual system, the probability of the existence of a tiny target object and the position of the object are predicted, thereby effectively extracting the spatial features in the image and achieving efficient pattern recognition.

[0086] In one embodiment, if Figure 4 As shown, in the above step S205, boundary regression and feature classification are performed in the target area to obtain the target position information of the tiny target object, which specifically includes the following steps:

[0087] Step S401 , based on the position prediction information of the tiny target object in the target area, the boundary regression technology is used to adjust the boundary box where the target area is located.

[0088] Step S402, performing feature extraction on the adjusted target area to obtain a feature extraction result.

[0089] Step S403: determining the target position information of the tiny target object according to the feature extraction result.

[0090] Among them, boundary regression technology is a key technology in target detection, which is mainly used to accurately adjust the candidate region to better surround the target object, which can be achieved by learning the transformation from the anchor point to the real bounding box.

[0091] Specifically, the server performs boundary regression on the filtered target area and adjusts its position to more accurately frame the tiny target object. It uses ROI Pooling or ROI Align to extract features within the target area, and uses the extracted features to classify these target areas (such as foreground and background classification). At the same time, it further regresses the position of the tiny target object to obtain more accurate target position information.

[0092] In this embodiment, by accurately adjusting the candidate region, the difference between the target region and the real region can be effectively reduced, and better feature alignment can be provided, so that more information can be retained, thereby improving detection accuracy.

[0093] In one embodiment, in the above step S403, determining the target position information of the tiny target object according to the feature extraction result specifically includes the following steps:

[0094] According to the feature extraction result, the adjusted target area is feature classified, and the classification score corresponding to the target area after feature classification is obtained; the target areas after feature classification are sorted according to the classification scores, and the current target area with the highest classification score is screened out; in the current target area, the target position information of the tiny target object is determined.

[0095] Specifically, the server again uses the non-maximum suppression technique to remove overlapping detection areas, and determines the target position information of the tiny target object in the current target area based on the classification score and the bounding box regression result, thereby improving the accuracy and robustness of tiny target object detection.

[0096] In one embodiment, Figure 5 and Figure 6 As shown, a method for detecting the position of a tiny target object in an image in a specific embodiment is provided, which specifically includes the following steps:

[0097] Step S601, obtaining an image to be detected and scene description information of the image to be detected, and identifying a large object that is logically related to a tiny target object in the image to be detected according to the scene description information; inputting the target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network.

[0098] Step S602, use the pre-trained convolutional layer to perform feature analysis on the feature map, obtain the bounding box regression coefficient of each target position in the feature map and the anchor point offset of the first candidate area; at each target position, generate multiple anchor points according to the anchor point offset; adjust the multiple anchor points according to the bounding box regression coefficient, and use the adjusted anchor points as the first candidate area.

[0099] Step S603, perform confidence prediction on multiple first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and use non-maximum suppression technology to calculate the degree of overlap of each first candidate region; in each first candidate region, identify highly overlapping regions whose degree of overlap exceeds a preset overlap threshold, and remove the highly overlapping regions.

[0100] Step S604, screen out several second candidate areas whose confidence scores meet the score threshold condition from the sorted first candidate areas, input the several second candidate areas into the pre-trained deep learning model, and obtain the probability prediction value and position prediction information of the existence of tiny target objects output by the deep learning model as the prediction result.

[0101] Step S605, sorting the second candidate areas according to the probability prediction values, screening out the target areas whose probability prediction values ​​meet the probability threshold conditions from the sorted second candidate areas, and adjusting the bounding box of the target area using the boundary regression technology based on the position prediction information of the tiny target objects in the target area; performing feature extraction on the adjusted target area to obtain the feature extraction result.

[0102] Step S606, based on the feature extraction result, feature classification is performed on the adjusted target area, and the classification score corresponding to the target area after feature classification is obtained; the target areas after feature classification are sorted according to the classification scores, and the current target area with the highest classification score is screened out; in the current target area, the target position information of the tiny target object is determined.

[0103] The beneficial effects brought by the above embodiments are as follows:

[0104] This application is based on the two-stage detection, and utilizes the physical position relationship, behavioral relationship, natural logical relationship and contextual relationship between objects to determine the candidate area. On the basis of anchoring the candidate area, it detects tiny target objects, performs contextual reasoning on the results, and outputs the detection results of tiny target objects, thereby effectively improving the detection precision and accuracy of tiny target objects in complex scenes of image detection.

[0105] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0106] Based on the same inventive concept, the embodiment of the present application also provides a device for detecting the position of a tiny target object in an image for implementing the method for detecting the position of a tiny target object in an image mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method mentioned above, so the specific limitations in the embodiments of the device for detecting the position of a tiny target object in one or more images provided below can refer to the limitations of the method for detecting the position of a tiny target object in an image mentioned above, and will not be repeated here.

[0107] In an exemplary embodiment, Figure 7 As shown, a device for detecting the position of a tiny target object in an image is provided, comprising:

[0108] The object recognition module 701 is used to obtain the image to be detected and the scene description information of the image to be detected, and to identify the large object logically related to the small target object in the image to be detected according to the scene description information;

[0109] A feature extraction module 702 is used to input a target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determine a plurality of first candidate regions in the feature map;

[0110] The first screening module 703 is used to perform confidence prediction on multiple first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and screen out several second candidate regions whose confidence scores meet the score threshold condition from the sorted first candidate regions;

[0111] The second screening module 704 is used to predict the probability and position of the tiny target object in the plurality of second candidate regions, and determine the target region in the second candidate regions according to the obtained prediction results;

[0112] The position determination module 705 is used to perform boundary regression and feature classification in the target area to obtain the target position information of the tiny target object.

[0113] In one embodiment, the feature extraction module 702 is also used to perform feature analysis on the feature map using a pre-trained convolutional layer to obtain the bounding box regression coefficient of each target position in the feature map and the anchor point offset of the first candidate area; at each target position, multiple anchor points are generated according to the anchor point offset; the multiple anchor points are adjusted according to the bounding box regression coefficient, and the adjusted anchor points are used as the first candidate area.

[0114] In one embodiment, the position detection device of a tiny target object in an image also includes a screening and removal module, which is used to calculate the degree of overlap of each first candidate area using a non-maximum suppression technique; in each first candidate area, highly overlapping areas whose degree of overlap exceeds a preset overlap threshold are identified, and the highly overlapping areas are removed.

[0115] In one embodiment, the second screening module 704 is also used to input several second candidate areas into a pre-trained deep learning model to obtain the probability prediction value and position prediction information of the existence of tiny target objects output by the deep learning model as a prediction result; sort the second candidate areas according to the probability prediction values, and screen out target areas whose probability prediction values ​​meet the probability threshold conditions from the sorted second candidate areas.

[0116] In one embodiment, the position determination module 705 is also used to adjust the boundary box of the target area using boundary regression technology based on the position prediction information of the tiny target object in the target area; perform feature extraction on the adjusted target area to obtain feature extraction results; and determine the target position information of the tiny target object based on the feature extraction results.

[0117] In one embodiment, the position determination module 705 is also used to perform feature classification on the adjusted target area according to the feature extraction result, and obtain the classification score corresponding to the target area after feature classification; sort the target areas after feature classification according to the classification scores, and filter out the current target area with the highest classification score; in the current target area, determine the target position information of the tiny target object.

[0118] Each module in the above-mentioned device for detecting the position of a tiny target object in an image can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0119] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data to be detected. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting the position of a small target object in an image is implemented.

[0120] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0122] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0123] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0124] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0125] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0126] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0127] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting the position of a tiny target object in an image, characterized in that: The method comprises: Acquire an image to be detected and scene description information of the image to be detected, and identify a large object logically related to a small target object in the image to be detected according to the scene description information; Inputting a target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determining a plurality of first candidate regions in the feature map; Performing confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sorting the first candidate regions according to the confidence scores, and screening out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions; Predicting the probability and position of the tiny target object in a plurality of the second candidate regions, and determining a target region in the second candidate regions according to the obtained prediction results; Boundary regression and feature classification are performed in the target area to obtain target position information of the tiny target object.

2. The method according to claim 1, characterized in that Determining a plurality of first candidate regions in the feature map includes: Performing feature analysis on the feature map using a pre-trained convolutional layer to obtain a bounding box regression coefficient for each target position in the feature map and an anchor point offset of the first candidate region; At each of the target positions, generating a plurality of anchor points according to the anchor point offsets; The plurality of anchor points are adjusted according to the bounding box regression coefficients, and the adjusted anchor points are used as the first candidate regions.

3. The method according to claim 1, characterized in that After sorting the first candidate regions according to the confidence scores, the method further includes: Using non-maximum suppression technology, calculating the degree of overlap of each of the first candidate regions; In each of the first candidate regions, a highly overlapping region whose overlapping degree exceeds a preset overlapping degree threshold is identified, and the highly overlapping region is removed.

4. The method according to any one of claims 1 to 3, characterized in that: The predicting the probability and position of the tiny target object in a plurality of the second candidate regions, and determining the target region in the second candidate regions according to the obtained prediction results, comprises: Inputting a plurality of the second candidate regions into a pre-trained deep learning model, and obtaining a probability prediction value and position prediction information of the tiny target object output by the deep learning model as the prediction result; The second candidate regions are sorted according to the probability prediction values, and the target regions whose probability prediction values ​​meet the probability threshold condition are screened out from the sorted second candidate regions.

5. The method according to claim 4, characterized in that The performing boundary regression and feature classification in the target area to obtain target position information of the tiny target object includes: According to the position prediction information of the tiny target object in the target area, a boundary regression technology is used to adjust the boundary box where the target area is located; Perform feature extraction on the adjusted target area to obtain a feature extraction result; The target position information of the tiny target object is determined according to the feature extraction result.

6. The method according to claim 5, characterized in that Determining the target position information of the tiny target object according to the feature extraction result includes: According to the feature extraction result, feature classification is performed on the adjusted target area, and a classification score corresponding to the target area after feature classification is obtained; sorting the target areas after the feature classification according to the classification scores, and selecting the current target area with the highest classification score; In the current target area, the target position information of the tiny target object is determined.

7. A device for detecting the position of a tiny target object in an image, characterized in that: The device comprises: An object recognition module is used to obtain an image to be detected and scene description information of the image to be detected, and to recognize a large object logically related to a small target object in the image to be detected according to the scene description information; A feature extraction module, used for inputting a target image corresponding to the large object into a pre-trained convolutional neural network to obtain a feature map extracted by the convolutional neural network, and determining a plurality of first candidate regions in the feature map; A first screening module is used to perform confidence prediction on the plurality of first candidate regions to obtain corresponding confidence scores, sort the first candidate regions according to the confidence scores, and screen out a plurality of second candidate regions whose confidence scores meet a score threshold condition from the sorted first candidate regions; A second screening module is used to predict the probability and position of the tiny target object in a plurality of the second candidate regions, and determine the target region in the second candidate regions according to the obtained prediction results; The position determination module is used to perform boundary regression and feature classification in the target area to obtain the target position information of the tiny target object.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images