Target detection method, device, equipment, storage medium and program product

Abstract

The present application relates to a target detection method, device, equipment, storage medium and program product. The method comprises: obtaining an image to be detected; the image to be detected comprises at least one target to be detected; inputting the image to be detected into a preset target detection network to obtain detection results of each target to be detected; wherein the target detection network comprises a probability heat map head module; the probability heat map head module comprises a background channel; the probability heat map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors. The use of this method can output target detection results with higher accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target detection method, device, equipment, storage medium and program product. Background Art

[0002] With the development of computer vision, a variety of deep learning target detection methods based on convolutional neural networks have emerged. Image target detection refers to the use of algorithms to mark the area of ​​key targets in an image, usually with a rectangular box to indicate the area where the target is located.

[0003] The application fields of image object detection include but are not limited to face detection, pedestrian detection, detection of natural images, etc. In the related technology, the commonly used methods for object detection include the RCNN series of "two-stage" methods, the YOLO series of "one-stage" methods, etc. These methods require pre-setting rectangular boxes of different aspect ratios and sizes when selecting candidate areas, and sequentially representing the candidate areas with the same center as the grid points on the feature map, so as to complete the object detection in the image.

[0004] However, when performing target detection, the target detection method in the related art may classify the same object into different categories at the same time, resulting in low accuracy of the output target detection result. Summary of the invention

[0005] Based on this, it is necessary to provide a target detection method, device, equipment, storage medium and program product that can output target detection results with higher accuracy in response to the above technical problems.

[0006] In a first aspect, the present application provides a target detection method, the method comprising:

[0007] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0008] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0009] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0010] In one embodiment, the target detection network further includes a feature map generation unit and a positioning information head module; accordingly, the image to be detected is input into a preset target detection network to obtain the detection results of each target to be detected, including:

[0011] The image to be detected is input into the target detection network, and the feature map to be detected corresponding to the image to be detected is extracted by the feature map generation unit;

[0012] Inputting the feature map to be detected into the probability heat map head module, and determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected by the probability heat map head module; and inputting the feature map to be detected into the positioning information head module, and determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected by the positioning information head module;

[0013] According to the classification results and positioning information, the detection results of each target to be detected are determined.

[0014] In one embodiment, the probability heat map head module further includes a non-background channel; accordingly, determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected by the probability heat map head module includes:

[0015] The maximum values ​​of all grid point areas in the feature map to be detected on all channels of the background channel and the non-background channel are used to determine the initial candidate grid point area;

[0016] Determine a preset number of grid point regions belonging to non-background channels in the initial candidate grid point region as candidate grid point regions;

[0017] The candidate grid point regions corresponding to the same target to be detected in the feature map to be detected are merged to obtain the classification result of the target to be detected.

[0018] In one embodiment, the positioning information head module includes a position information head module and a size information head module; accordingly, determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected by the positioning information head module includes:

[0019] Determine the position information of the center point of the target to be detected from the feature map to be detected through the position information head module;

[0020] Obtain the size information of the target to be detected in the feature map to be detected through the size information head module;

[0021] The positioning information of the target to be detected is determined based on the position information and size information.

[0022] In one embodiment, the process of constructing the target detection network includes:

[0023] Obtain multiple training images of labeled objects;

[0024] Inputting a plurality of training images of labeled targets into an initial target detection network, determining classification results of training candidate targets through an initial probability heat map head module in the initial target detection network, and determining training positioning parameters through an initial positioning information head module in the initial target detection network;

[0025] Substitute the classification results and training positioning parameters of the candidate training targets into the preset loss function;

[0026] According to the calculation results of the loss function, back propagation is performed, and the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module are iteratively optimized until the preset number of iterations is reached to obtain the target detection network.

[0027] In one embodiment, before inputting a plurality of training images of labeled objects into an initial object detection network, the method includes:

[0028] Data augmentation preprocessing is performed on training images of multiple labeled targets; the data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

[0029] In a second aspect, the present application also provides a target detection device, the device comprising:

[0030] An image acquisition module is used to acquire an image to be detected; the image to be detected includes at least one marked target to be detected;

[0031] The image detection module is used to input the image to be detected into a preset target detection network to obtain the classification results of each target to be detected;

[0032] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to classify each target to be detected, and remove interference factors in the classified targets to be detected through the background channel.

[0033] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0034] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0035] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0036] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0038] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0039] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0040] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0041] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0042] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0043] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0044] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0045] The above-mentioned target detection method, device, equipment, storage medium and program product, the terminal obtains an image to be detected including at least one target to be detected; inputs the image to be detected into a preset target detection network to obtain the detection results of each target to be detected, wherein the target detection network includes a probability heat map head module; the probability heat map head module includes a background channel; the probability heat map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors. Since when detecting the target in the image to be detected, it is detected by using a pre-trained target detection network, the target detection network is provided with a probability heat map head module, and the probability heat map head module is provided with a background channel. When detecting the target, the background channel in the probability heat map head module can be used to remove the interference factors generated when the background information in the image to be detected is classified, thereby increasing the classification accuracy of the target to be detected, and then increasing the accuracy of the detection results of each target to be detected in the image to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A diagram showing an application environment of a target detection method in an embodiment;

[0047] Figure 2 Schematic diagram of a target detection method in one embodiment;

[0048] Figure 3 A schematic diagram of the architecture of a target detection method in an embodiment;

[0049] Figure 4 A schematic diagram of a flow chart of a probability heat map header module in one embodiment;

[0050] Figure 5 is a schematic diagram of positioning information in an embodiment;

[0051] Figure 6 is a schematic diagram of a marking of a rotating target frame in an embodiment;

[0052] Figure 7 is a schematic diagram of test results in an embodiment;

[0053] Figure 8 is a structural block diagram of a target detection device in one embodiment;

[0054] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] The target detection method provided in the present application can be applied to a computer device, which can be a terminal, a server, or a device cluster consisting of a terminal and a server, for example, including but not limited to various personal computers, laptops, tablet computers, wearable devices, etc. The present application embodiment does not limit the type of computer device. Figure 1 As shown, a schematic diagram of the internal structure of a computer device is provided. Figure 1 The processor in the device is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database is used to store relevant data of the feature selection process. The network interface is used to communicate with other external devices through a network connection. When the computer program is executed by the processor, a target detection method is implemented.

[0057] In one embodiment, a target detection method is provided. This embodiment uses the method applied to a terminal as an example. Figure 2 As shown, the method comprises the following steps:

[0058] Step 102, obtaining an image to be detected; the image to be detected includes at least one target to be detected.

[0059] The image to be detected is a pre-acquired image including an object to be detected by classification. The object to be detected is an object to be detected by classification in the image to be detected.

[0060] Specifically, the terminal obtains an image to be detected including at least one target to be detected. This embodiment does not limit the type of the image to be detected, as long as it meets the processing format of the target detection network. Optionally, the image to be detected is an image obtained by aerial photography or a remote sensing image.

[0061] Step 104, input the image to be detected into a preset target detection network to obtain the detection results of each target to be detected; wherein, the target detection network includes a probability thermal map head module; the probability thermal map head module includes a background channel; the probability thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0062] Among them, the preset target detection network is a detection network trained based on convolutional neural network and CenterNet (center point based method). The probability heat map head module is a module for classifying the target to be detected in the image to be detected. The background channel is a preset channel in the probability head module used to determine the part of the image to be detected that belongs to the background. The detection result is the area corresponding to the target to be detected and the corresponding classification result.

[0063] Specifically, the terminal inputs the image to be detected into the target detection network trained based on the convolutional neural network and CenterNet, and classifies each target to be detected through the probability heat map head module in the target detection network. During the classification process, the background channel of the probability heat map head module is used to determine the part of the image to be detected that belongs to the background, so as to remove the interference factors in the target to be detected. The terminal performs relevant processing on the classification results through the target detection network to obtain the detection results of each target to be detected.

[0064] In the above target detection method, the terminal obtains an image to be detected including at least one target to be detected; inputs the image to be detected into a preset target detection network to obtain the detection results of each target to be detected, wherein the target detection network includes a probability heat map head module; the probability heat map head module includes a background channel; the probability heat map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors. Since when detecting the target in the image to be detected, it is detected by using a pre-trained target detection network, the target detection network is provided with a probability heat map head module, and the probability heat map head module is provided with a background channel. When detecting the target, the background channel in the probability heat map head module can be used to remove the interference factors generated when the background information in the image to be detected is classified, thereby increasing the classification accuracy of the target to be detected, and then increasing the accuracy of the detection results of each target to be detected in the image to be detected.

[0065] In one embodiment, the target detection network further includes a feature map generation unit and a positioning information head module; accordingly, the image to be detected is input into a preset target detection network to obtain the detection results of each target to be detected, including:

[0066] The image to be detected is input into the target detection network, and the feature map to be detected corresponding to the image to be detected is extracted by the feature map generation unit.

[0067] Among them, the feature map generation unit is an image processing unit.

[0068] Specifically, the terminal inputs the image to be detected into a preset target detection network, and the feature map generation unit in the target detection network downsamples the image to be detected at different resolutions to generate a feature map of the corresponding resolution, and then performs multi-scale fusion on the feature maps of the corresponding resolutions to generate a feature map to be detected.

[0069] In the specific implementation process, the terminal inputs the image to be detected into a preset target detection network, and uses the convolutional neural network in the target detection network, such as Figure 3 The Resnet101 (101-layer residual network) in the image to be detected downsamples the image to be detected at different resolutions, and saves the feature maps corresponding to the convolution blocks with different downsampling rates. The feature maps corresponding to the convolution blocks with different downsampling rates are then fused through short-cut (a structure under the CNN model) to obtain a multi-scale feature map, i.e., the feature map to be detected.

[0070] The feature map to be detected is input into the probability heat map head module, and the classification result of the target to be detected in the feature map to be detected is determined from the grid area of ​​the feature map to be detected by the probability heat map head module; and the feature map to be detected is input into the positioning information head module, and the positioning information of the target to be detected in the feature map to be detected is determined from the feature map to be detected by the positioning information head module.

[0071] The grid area of ​​the feature map to be detected is a grid area divided according to the feature map to be detected. The positioning information head module is a module for determining the area of ​​the target to be detected. The positioning information refers to the area of ​​the target to be detected in the image to be detected.

[0072] The probability heat map head module and the positioning information head module in the target detection network are two modules arranged in parallel. The feature map to be detected output from the feature map generation unit will enter the probability heat map head module and the positioning information head module respectively. Specifically, the feature map to be detected is input into the probability heat map head module, so that the classification result of the target to be detected in the feature map to be detected can be determined from the grid area divided according to the feature map to be detected; at the same time, the feature map to be detected enters the positioning information head module, so that the area of ​​the target to be detected in the image to be detected can be determined from the feature map to be detected.

[0073] For example, in one embodiment, an implementation method of determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected by the probability heat map head module includes the following process:

[0074] The initial candidate grid point area is determined by taking the maximum values ​​of all grid point areas in the feature map to be detected on all channels of the background channel and the non-background channel.

[0075] In this embodiment, the probability heat map header module also includes a non-background channel.

[0076] The channel is the classification of target detection. For example, if the target detection requirement is to detect the type of ship in the aerial image, then all the classifications of the ship correspond to a non-background channel.

[0077] The terminal takes all the grid point areas in the feature map to be detected, and takes the channels corresponding to the maximum probability of each grid point area in the background channel and the non-background channel in turn, and then determines each grid point area and the background channel or non-background channel where the corresponding probability maximum value is located as the initial candidate grid point area.

[0078] A preset number of grid point regions belonging to non-background channels in the initial candidate grid point region are determined as candidate grid point regions.

[0079] Specifically, the terminal determines the initial candidate grid point area, i.e., each grid point area and the corresponding probability maximum value in the background channel or non-background channel, where the channel where each grid point area and the corresponding probability maximum value is a non-background channel, as a secondary candidate grid point area, and screens the secondary candidate grid point area according to preset conditions, and determines the secondary grid point area that meets the preset conditions as the candidate grid point area. The present invention does not limit the preset conditions, and the setting of the preset conditions can be adaptively modified according to the needs. Optionally, the preset condition is to sort the probability values ​​of the secondary candidate grid point area in descending order, and select a part of the sorting as the candidate grid point area according to the needs; optionally, the preset condition is to compare the probability value of the secondary candidate grid point area with the standard threshold, and if the probability value is greater than the standard value, it is determined to be a candidate grid point area.

[0080] The candidate grid point regions corresponding to the same target to be detected in the feature map to be detected are merged to obtain the classification result of the target to be detected.

[0081] Specifically, the terminal merges the candidate grid point regions corresponding to the same target to be detected in the feature map to be detected through a maximum suppression function to obtain a classification result of the target to be detected.

[0082] In the specific implementation process, Figure 4 As shown in FIG. 1 , if the size of the feature map to be detected outputted by the terminal after the image to be detected is input into the feature map generation unit in the preset target detection network is M*N, the feature map to be detected is divided into M*N grid areas. The terminal obtains C classification types divided according to the requirements. Accordingly, the number of non-background channels is C, the number of background channels is 1, and each channel is Figure 4 The terminal inputs the feature map to be detected including the M*N grid point area into the C+1 channels of the probability heat map head module, obtains the channel with the maximum probability value corresponding to the M*N grid point area, and uses it as the initial candidate grid point area. The above initial candidate grid point area is Figure 4 The channels in the M*N candidate grid point regions are marked with asterisks. The grid point regions whose channels correspond to the maximum probability values ​​of the grid point regions are background channels are determined as secondary candidate grid point regions, and the secondary candidate grid point regions are sorted from large to small according to the probability values ​​therein, and the secondary candidate grid point regions with the largest probability values ​​in the secondary candidate grid point regions are determined as candidate grid point regions. If different candidate grid point regions represent the same target to be detected, the non-maximum suppression function is used to classify the different candidate grid point regions representing the same target to be detected.

[0083] The above-mentioned initial candidate grid point areas are determined by the maximum values ​​of all grid point areas in all channels of the background channel and the non-background channel through the feature map to be detected, so as to classify each grid point area. The maximum value of each channel may be 1 / the number of channels, so that the sum of the classification probabilities of the same grid point area is 1, and the classification division is mutually exclusive. By determining a preset number of grid point areas belonging to the non-background channel in the initial candidate grid point area as candidate grid point areas, that is, removing the interference factors generated by the classification of the background in the target to be detected through the background channel, the classification accuracy of the target to be detected is increased; by merging the candidate grid point areas corresponding to the same target to be detected in the feature map to be detected, the target candidate grid point area is determined to obtain the classification result of the target to be detected.

[0084] For example, in one embodiment, an implementation method of determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected by the positioning information head module includes the following process:

[0085] In this embodiment, the positioning information head module includes a position information head module and a size information head module.

[0086] The position information of the center point of the target to be detected is determined from the feature map to be detected through the position information head module.

[0087] Among them, the position information head module is used to obtain the center point position coordinates of the target to be detected in the image to be detected.

[0088] Specifically, the terminal obtains the index information of the target grid area from the probability heat map head module through the position information head module to obtain the center point coordinate range of the target in the image to be detected, and combines the coordinate offset obtained by the size information head module itself to obtain the specific center point position coordinates.

[0089] The size information of the target to be detected is determined in the feature map to be detected by the size information head module.

[0090] Among them, the size information head module is used to obtain the annotation box of the target to be detected.

[0091] Specifically, the terminal calls the position information head module through the size information head module to obtain the center point position coordinates of the target to be detected in the image to be detected, and regressively calculates the offset of the center point position coordinates relative to the relative center of the four midpoints to obtain the position coordinate information of the annotation box of the target to be detected.

[0092] According to the position information and size information, the positioning information of the target to be detected is determined.

[0093] Specifically, the terminal determines the positioning information of the target to be detected in the image to be detected according to the position coordinates of the center point of the target to be detected and the position coordinate information of the annotation box. The positioning information can be understood as follows: Figure 5 Annotation boxes for images of ships in .

[0094] The terminal determines the positioning information of the target to be detected based on the center point position information and size information of the target to be detected, reducing the work of pre-marking the target to be detected, reducing the amount of manual work in the target detection process and increasing the degree of intelligence.

[0095] After the above process, the classification result and positioning information of the target to be detected can be obtained, which is equivalent to merging each candidate grid point corresponding to the same target to be detected in the feature map to be detected of the image to be detected, that is, the candidate grid points belonging to the same target are divided together, and when the candidate grid points belonging to the same target are divided together, the positioning information of the target to be detected is determined, and the outline size of each target to be detected and the coordinate position of the center point and other information can be known. Therefore, the positioning information of each target to be detected in the image to be detected can be determined, and combined with the classification result, the detection result of each target to be detected can be determined.

[0096] In addition, in some scenes, after determining the detection results of each target to be detected, the terminal marks each target to be detected in the image to be detected, for example, by means of a rotating rectangular frame, and outputs it in the terminal interface. In other scenes, the terminal determines the detection results of each target to be detected, detects whether a user's request to view the detection results is received, and the request indicates that the user needs to view each detection target detected in the image to be detected at this time. After receiving the request, the terminal then performs the steps of marking each target to be detected in the image to be detected and outputting it in the terminal interface, so that the user can view it conveniently. The embodiment of the present application does not limit this process, and can be set according to actual needs.

[0097] In the above target detection method, the feature map to be detected corresponding to the image to be detected is extracted by the feature map generation unit in the target detection network. The feature map to be detected can represent targets to be detected of different sizes, so that the terminal can input the feature map to be detected into the probability thermal map head module and the positioning information head module, and can more accurately process targets to be detected of different sizes. The processing includes removing the interference factors generated by the classification of the background in the target to be detected through the background channel in the probability thermal map head module, thereby increasing the classification accuracy of the target to be detected; determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected through the positioning information head module, thereby reducing the work of pre-marking the target to be detected, thereby reducing the amount of manual work in the process of target detection and increasing the degree of intelligence; finally, determining the detection results of each target to be detected based on the classification results and positioning information.

[0098] Based on any of the above embodiments, in one embodiment, the process of constructing the target detection network includes: obtaining a plurality of training images of labeled targets.

[0099] Among them, the training image is annotated with the target to be detected by rotating the target box, such as Figure 6 shown.

[0100] Specifically, the terminal obtains a plurality of training images of annotated targets. After the terminal annotates the target to be detected in the training image by rotating the rectangular box, it processes the four vertices of the rotating rectangular box, obtains the center points of the four sides of the rotating rectangular box and the center point of the rotating rectangular box, and obtains the offset vector from the center point of the rotating rectangular box to the center points of the four sides of the rotating rectangular box according to the above five center points. The terminal determines the category of the target to be detected according to the requirements.

[0101] Inputting a plurality of training images of labeled targets into an initial target detection network, determining classification results of training candidate targets through an initial probability heat map head module in the initial target detection network, and determining training positioning parameters through an initial positioning information head module in the initial target detection network;

[0102] Specifically, the terminal sets the number of channels of the initial probability head module according to the category of the target to be detected. If the category is C, the number of channels is C+1, where C channels correspond to one channel for each category of the target to be detected, and the other channel is the background channel. Among them, the execution step of "the terminal inputs multiple labeled training images into the initial target detection network, determines the classification results of the training candidate targets through the initial probability heat map head module in the initial target detection network, and determines the training positioning parameters through the initial positioning information head module in the initial target detection network" in the training process has corresponding technical features to the corresponding detection steps in the target detection process, which will not be repeated here.

[0103] The classification results and training positioning parameters of the training candidate targets are substituted into the preset loss function; back propagation is performed according to the calculation results of the loss function, and the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module are iteratively optimized until the preset number of iterations is reached to obtain the target detection network.

[0104] The preset number of iterations is the preset number of iterations corresponding to when the loss function reaches a local optimum.

[0105] Specifically, the terminal substitutes the classification results and training positioning parameters of the candidate training targets into the preset loss function, and analyzes the classification results and training positioning parameters of the candidate training targets and the labeled classification results and the data of the rotation target frame. If the difference does not meet the expectation, the terminal performs back propagation according to the calculation result of the loss function, and iteratively optimizes the initial feature map generation unit, the initial probability heat map head module, and the initial positioning information head module until the preset number of iterations is reached to obtain the target detection network. The expectation is a local optimal value or a relatively stable value or range.

[0106] In the above target detection method, in the process of constructing the target detection network at the terminal, the classification results and training positioning parameters of the training candidate targets are substituted into the preset loss function; back propagation is performed according to the calculation result of the loss function, and the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module are iteratively optimized to improve the detection rate of target detection; the training images of multiple labeled targets are input into the initial target detection network, and the accuracy of the classification results of the training candidate targets is increased through the initial probability heat map head module in the initial target detection network; the training positioning parameters are determined through the initial positioning information head module in the initial target detection network, so as to reduce the work of pre-labeling the targets to be detected, thereby reducing the amount of manual work in the target detection process and increasing the degree of intelligence.

[0107] In one embodiment, before inputting a plurality of training images of labeled objects into an initial object detection network, the method includes:

[0108] Data augmentation preprocessing is performed on training images of multiple labeled targets; the data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

[0109] Specifically, the terminal uses the open source tool DOTA-toolkit to crop the input training image of the labeled target with the area where the target to be detected is located as the center and perform data augmentation to obtain a training image of a preset size.

[0110] Optionally, the size of the preset training image is 608*608.

[0111] In the above target detection method, data augmentation is used to expand the size of the training set and reduce the model's dependence on certain attributes.

[0112] In one embodiment, the HRSC2016 dataset includes 27 types of ships and 1061 pictures, including 2976 objects. The 1061 pictures are divided into three datasets including a training set, a validation set, and a test set, and the three datasets contain 436, 181, and 444 pictures respectively.

[0113] The images in the training set are used as training images to build the target detection network. The size of the processed training image is 608*608, the initial learning rate is 1.25*10-4, and the learning rate change function is an exponential function. The number of batches in the training phase is 16, and the number of batches in the test phase is 1. The secondary candidate grid point regions with the top 500 probability values ​​in the secondary candidate grid point regions are determined as candidate grid point regions.

[0114] The output result after inputting the test set into the target detection network is as follows Figure 7 shown.

[0115] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a target detection device for implementing the target detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target detection device embodiments provided below can refer to the limitations on the target detection method above, and will not be repeated here.

[0117] In one embodiment, Figure 8 As shown, a target detection device is provided, comprising: an image acquisition module 100 and an image detection module 200, wherein:

[0118] The image acquisition module 100 is used to acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0119] The image detection module 200 is used to input the image to be detected into a preset target detection network to obtain the detection results of each target to be detected; wherein, the target detection network includes a probability thermal map head module; the probability thermal map head module includes a background channel; the probability thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0120] In one embodiment, the target detection network further includes a feature map generation unit and a positioning information head module; accordingly, the image detection module 200 includes:

[0121] The image to be detected is input into the target detection network, and the feature map to be detected corresponding to the image to be detected is extracted by the feature map generation unit; the feature map to be detected is input into the probability heat map head module, and the classification result of the target to be detected in the feature map to be detected is determined from the grid area of ​​the feature map to be detected by the probability heat map head module; and the feature map to be detected is input into the positioning information head module, and the positioning information of the target to be detected in the feature map to be detected is determined from the feature map to be detected by the positioning information head module; according to the classification result and the positioning information, the detection result of each target to be detected is determined.

[0122] In one embodiment, the probability heat map header module further includes a non-background channel; accordingly, the image detection module 200 includes:

[0123] The maximum values ​​of all grid point areas in the feature map to be detected on all channels of the background channel and the non-background channel are used to correspond to the initial candidate grid point area; a preset number of grid point areas belonging to the non-background channel in the initial candidate grid point area are determined as candidate grid point areas; the candidate grid point areas corresponding to the same target to be detected in the feature map to be detected are merged to obtain the classification result of the target to be detected.

[0124] In one embodiment, the positioning information head module includes a position information head module and a size information head module; accordingly, the image detection module 200 includes:

[0125] The position information of the center point of the target to be detected is determined from the feature map to be detected through the position information head module; the size information of the target to be detected in the feature map to be detected is obtained through the size information head module; and the positioning information of the target to be detected is determined based on the position information and size information.

[0126] In one embodiment, the process of constructing the target detection network includes:

[0127] A training image acquisition module, used to acquire training images of multiple labeled targets;

[0128] A result and parameter acquisition module, used to input a plurality of training images of labeled targets into an initial target detection network, determine the classification results of the training candidate targets through an initial probability heat map head module in the initial target detection network, and determine the training positioning parameters through an initial positioning information head module in the initial target detection network;

[0129] The parameter substitution module is used to substitute the classification results and training positioning parameters of the training candidate targets into the preset loss function;

[0130] The detection network determination module is used to perform back propagation according to the calculation results of the loss function, iteratively optimize the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module until a preset number of iterations is reached to obtain the target detection network.

[0131] In one embodiment, before the result and parameter acquisition module, it includes:

[0132] The data preprocessing module is used to perform data augmentation preprocessing on the training images of multiple labeled targets; the data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

[0133] Each module in the above target detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0134] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store images to be detected. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection method is implemented.

[0135] Those skilled in the art will understand that Fig. 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0137] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0138] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0139] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0140] In one embodiment, the processor implements the following steps when executing a computer program: inputting an image to be detected into a target detection network, and extracting a feature map to be detected corresponding to the image to be detected through a feature map generation unit; inputting the feature map to be detected into a probability thermal map head module, and determining a classification result of the target to be detected in the feature map to be detected from a grid area of ​​the feature map to be detected through the probability thermal map head module; and inputting the feature map to be detected into a positioning information head module, and determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected through the positioning information head module; and determining the detection result of each target to be detected based on the classification result and the positioning information.

[0141] In one embodiment, the processor implements the following steps when executing a computer program: initial candidate grid point areas are determined corresponding to the maximum values ​​of all grid point areas in all channels of background channels and non-background channels in the feature map to be detected; a preset number of grid point areas belonging to non-background channels in the initial candidate grid point areas are determined as candidate grid point areas; and candidate grid point areas corresponding to the same target to be detected in the feature map to be detected are merged to obtain a classification result of the target to be detected.

[0142] In one embodiment, the processor implements the following steps when executing a computer program: determining the position information of the center point of the target to be detected from the feature map to be detected through a position information head module; obtaining the size information of the target to be detected in the feature map to be detected through a size information head module; and determining the positioning information of the target to be detected based on the position information and the size information.

[0143] In one embodiment, the processor implements the following steps when executing a computer program: acquiring a plurality of training images of labeled targets; inputting the plurality of training images of labeled targets into an initial target detection network, determining the classification results of the training candidate targets through an initial probability heat map head module in the initial target detection network, and determining the training positioning parameters through an initial positioning information head module in the initial target detection network; substituting the classification results and training positioning parameters of the training candidate targets into a preset loss function; performing back propagation according to the calculation result of the loss function, iteratively optimizing the initial feature map generation unit, the initial probability heat map head module, and the initial positioning information head module until a preset number of iterations is reached to obtain a target detection network.

[0144] In one embodiment, the processor implements the following steps when executing the computer program: performing data augmentation preprocessing on a plurality of training images of labeled targets; the data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

[0145] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0146] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0147] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0148] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0149] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: inputting the image to be detected into the target detection network, and extracting the feature map to be detected corresponding to the image to be detected through the feature map generation unit; inputting the feature map to be detected into the probability thermal map head module, and determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected through the probability thermal map head module; and inputting the feature map to be detected into the positioning information head module, and determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected through the positioning information head module; and determining the detection result of each target to be detected based on the classification result and the positioning information.

[0150] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the maximum values ​​of all grid point areas in all channels of the background channel and the non-background channel in the feature map to be detected are used to correspond to the initial candidate grid point areas; a preset number of grid point areas belonging to the non-background channel in the initial candidate grid point area are determined as candidate grid point areas; and the candidate grid point areas corresponding to the same target to be detected in the feature map to be detected are merged to obtain the classification result of the target to be detected.

[0151] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the position information of the center point of the target to be detected from the feature map to be detected through the position information head module; obtaining the size information of the target to be detected in the feature map to be detected through the size information head module; and determining the positioning information of the target to be detected based on the position information and the size information.

[0152] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining training images of multiple labeled targets; inputting the training images of multiple labeled targets into the initial target detection network, determining the classification results of the training candidate targets through the initial probability heat map head module in the initial target detection network, and determining the training positioning parameters through the initial positioning information head module in the initial target detection network; substituting the classification results and training positioning parameters of the training candidate targets into a preset loss function; performing back propagation according to the calculation result of the loss function, iteratively optimizing the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module until a preset number of iterations is reached to obtain the target detection network.

[0153] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: data augmentation preprocessing is performed on a plurality of training images of labeled targets; the data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

[0154] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0155] Acquire an image to be detected; the image to be detected includes at least one target to be detected;

[0156] Input the image to be detected into the preset target detection network to obtain the detection results of each target to be detected;

[0157] Among them, the target detection network includes a probabilistic thermal map head module; the probabilistic thermal map head module includes a background channel; the probabilistic thermal map head module is used to remove interference factors in the target to be detected through the background channel, and classify each target to be detected after removing the interference factors.

[0158] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: inputting the image to be detected into the target detection network, and extracting the feature map to be detected corresponding to the image to be detected through the feature map generation unit; inputting the feature map to be detected into the probability thermal map head module, and determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected through the probability thermal map head module; and inputting the feature map to be detected into the positioning information head module, and determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected through the positioning information head module; and determining the detection result of each target to be detected based on the classification result and the positioning information.

[0159] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the maximum values ​​of all grid point areas in all channels of the background channel and the non-background channel in the feature map to be detected are used to correspond to the initial candidate grid point areas; a preset number of grid point areas belonging to the non-background channel in the initial candidate grid point area are determined as candidate grid point areas; and the candidate grid point areas corresponding to the same target to be detected in the feature map to be detected are merged to obtain the classification result of the target to be detected.

[0160] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the position information of the center point of the target to be detected from the feature map to be detected through the position information head module; obtaining the size information of the target to be detected in the feature map to be detected through the size information head module; and determining the positioning information of the target to be detected based on the position information and the size information.

[0161] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining training images of multiple labeled targets; inputting the training images of multiple labeled targets into the initial target detection network, determining the classification results of the training candidate targets through the initial probability heat map head module in the initial target detection network, and determining the training positioning parameters through the initial positioning information head module in the initial target detection network; substituting the classification results and training positioning parameters of the training candidate targets into a preset loss function; performing back propagation according to the calculation result of the loss function, iteratively optimizing the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module until a preset number of iterations is reached to obtain the target detection network.

[0162] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: data augmentation preprocessing is performed on a plurality of training images of labeled targets; the data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0165] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire an image to be detected; the image to be detected includes at least one target to be detected; Inputting the image to be detected into a preset target detection network to obtain detection results of each target to be detected; Wherein, the target detection network includes a probability heat map head module; the probability heat map head module includes a background channel; the probability heat map head module is used to remove interference factors in the target to be detected through the background channel, and classify each of the targets to be detected after removing the interference factors; Wherein, the target detection network also includes a feature map generation unit and a positioning information head module; Correspondingly, the image to be detected is input into a preset target detection network to obtain the detection result of each target to be detected, including: Inputting the image to be detected into the target detection network, and extracting the feature map to be detected corresponding to the image to be detected by the feature map generation unit; Inputting the feature map to be detected into the probability heat map head module, and determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected by the probability heat map head module; and inputting the feature map to be detected into the positioning information head module, and determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected by the positioning information head module; Determining the detection result of each of the targets to be detected according to the classification result and the positioning information; Wherein, the probability heat map head module also includes a non-background channel; Accordingly, determining the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected by the probability heat map head module includes: The maximum values ​​of all grid point areas in the feature map to be detected on all channels of the background channel and the non-background channel are used to determine the initial candidate grid point area; Determine a preset number of grid point regions belonging to the non-background channel in the initial candidate grid point region as candidate grid point regions; The candidate grid point regions corresponding to the same target to be detected in the feature map to be detected are merged to obtain a classification result of the target to be detected.

2. According to the method of claim 1, the positioning information head module comprises a position information head module and a size information head module; Correspondingly, determining the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected by the positioning information head module includes: Determine the position information of the center point of the target to be detected from the feature map to be detected by the position information head module; Obtaining the size information of the target to be detected in the feature map to be detected by the size information head module; The positioning information of the target to be detected is determined according to the position information and the size information.

3. According to the method of claim 1 or 2, the process of constructing the target detection network comprises: Obtain multiple training images of labeled objects; Inputting the training images of the multiple labeled targets into an initial target detection network, determining the classification results of the training candidate targets through an initial probability heat map head module in the initial target detection network, and determining the training positioning parameters through an initial positioning information head module in the initial target detection network; Substituting the classification results of the training candidate targets and the training positioning parameters into a preset loss function; Back propagation is performed according to the calculation result of the loss function, and the initial feature map generation unit, the initial probability heat map head module and the initial positioning information head module are iteratively optimized until a preset number of iterations is reached to obtain the target detection network.

4. The method according to claim 3, before inputting the training images of the plurality of labeled targets into the initial target detection network, the method comprises: Performing data augmentation preprocessing on the training images of the multiple labeled objects; The data augmentation preprocessing at least includes image transformation processing, image cropping processing, and image flipping processing.

5. A target detection device, characterized in that: The device comprises: An image acquisition module, used to acquire an image to be detected; the image to be detected includes at least one marked target to be detected; An image detection module is used to input the image to be detected into a preset target detection network to obtain the classification result of each target to be detected; Wherein, the target detection network includes a probability heat map head module; the probability heat map head module includes a background channel; the probability heat map head module is used to classify each of the targets to be detected, and remove interference factors in the classified targets to be detected through the background channel; Wherein, the target detection network also includes a feature map generation unit and a positioning information head module; Correspondingly, the image detection module is specifically used to input the image to be detected into the target detection network, extract the feature map to be detected corresponding to the image to be detected through the feature map generation unit; input the feature map to be detected into the probability thermal map head module, and determine the classification result of the target to be detected in the feature map to be detected from the grid area of ​​the feature map to be detected through the probability thermal map head module; and input the feature map to be detected into the positioning information head module, and determine the positioning information of the target to be detected in the feature map to be detected from the feature map to be detected through the positioning information head module; determine the detection result of each target to be detected according to the classification result and the positioning information; Wherein, the probability heat map head module also includes a non-background channel; Correspondingly, the image detection module is further used to determine the initial candidate grid point area corresponding to the maximum value of all grid point areas in the feature map to be detected on all channels of the background channel and the non-background channel; Determine a preset number of grid point regions belonging to the non-background channel in the initial candidate grid point region as candidate grid point regions; The candidate grid point regions corresponding to the same target to be detected in the feature map to be detected are merged to obtain a classification result of the target to be detected.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images