A target detection method and system based on two-dimensional images

Abstract

The application discloses a target detection method and system based on two-dimensional images, and relates to the field of computer vision, and comprises the following steps: performing target position labeling on a two-dimensional image collected in a monitoring area to generate a training data set; then, training a backbone network and a target center prediction branch network of the deep learning target detection model by using the training set data, judging a minimum target branch network and a target scale prediction branch network, and detecting a target by using the trained deep learning target detection model, wherein different means are adopted for labeling and detecting targets with different image resolution sizes, even if the shooting range is expanded, some extremely small targets can also be detected by using a single pixel point, and thus the target detection range can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a target detection method and system based on two-dimensional images. Background Technology

[0002] With the development of transportation and the increase in the number of vehicles, intelligent traffic monitoring systems using video technology are also developing and playing an important role in traffic management. Among these systems, the use of aerial cameras for parking lot management is an indispensable part. As the number of parking lots and parking spaces increases, intelligent methods and devices are needed to automatically obtain parking space locations to further analyze vehicle parking behavior and serve parking operations. Therefore, object detection is a fundamental task for intelligent monitoring, behavior detection, event detection, and autonomous driving.

[0003] Currently, due to the different focal lengths of the camera and the target in the visual scene, the target scale in the image is different. As the shooting range expands, the boundaries of some extremely small targets cannot be distinguished in the image, which increases the difficulty for deep learning-based target detection models and causes a decrease in the accuracy of target recognition. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a target detection method and system based on two-dimensional images. This method can solve the problem that currently, some extremely small targets cannot have their boundaries distinguished in images, which increases the difficulty for deep learning-based target detection models and leads to a decrease in the accuracy of target recognition.

[0005] To achieve the above objectives, in one aspect, the present invention provides a target detection method based on two-dimensional images, the method comprising:

[0006] Vehicle targets are labeled in the two-dimensional images of the acquired monitoring area to generate a training dataset;

[0007] Feature maps are extracted from images in the training dataset using the backbone network of a deep learning object detection model;

[0008] The feature map is input into the target center prediction branch network of the deep learning target detection model, the minimum target prediction branch network and the target scale prediction branch network, respectively, to obtain the target center position, whether the target is a minimum target label and the target size. The target center prediction branch network, the minimum target prediction branch network and the target scale prediction branch network are trained based on the target center position, whether the target is a minimum target label and the target size.

[0009] Vehicle targets are detected and identified using a trained deep learning target detection model.

[0010] Furthermore, the step of annotating vehicle targets in the acquired two-dimensional images of the monitored area to generate a training dataset includes:

[0011] For non-minimal targets in the two-dimensional image of the monitoring area with pixels greater than or equal to a preset pixel size, mark the minimum bounding rectangle of the target;

[0012] For extremely small targets in a two-dimensional image within the monitored area whose pixels are smaller than a preset pixel size, the location and category of the target's center point are marked.

[0013] Furthermore, the step of training the target center prediction branch network based on the target center location includes:

[0014] Using the center of the target in the feature map as the center of the circle, and calculating the radius of the circle based on the target size;

[0015] A two-dimensional Gaussian function-filled heatmap is generated using the center and radius of the circle, and the target center prediction branch network is trained based on the regression method of the two-dimensional Gaussian function-filled heatmap.

[0016] Furthermore, the step of training the minimal target determination branch network based on whether the target is a minimal target label includes:

[0017] For the feature vector corresponding to the target center in the feature map, a binary classifier is used to determine the minimum target, thus completing the training of the minimum target determination branch network.

[0018] Furthermore, the step of training the target scale prediction branch network based on the target size includes:

[0019] The width and height of the target are generated by using the feature vector corresponding to the center of the non-minimal target in the feature image, and the width and height of the target are obtained by training through regression.

[0020] On the other hand, the present invention provides a target detection system based on two-dimensional images, the system comprising: a generation unit, used to annotate vehicle targets in two-dimensional images in a collected monitoring area and generate a training dataset;

[0021] The training unit is used to extract feature maps from images in the training dataset through the backbone network of the deep learning object detection model; input the feature maps into the target center prediction branch network, the minimum target prediction branch network, and the target scale prediction branch network of the deep learning object detection model to obtain the target center position, whether the target is a minimum target label, and the target size, respectively; and train the target center prediction branch network, the minimum target prediction branch network, and the target scale prediction branch network based on the target center position, whether the target is a minimum target label, and the target size.

[0022] The detection and recognition unit is used to detect and recognize vehicle targets using a trained deep learning target detection model.

[0023] Furthermore, the generating unit is specifically used to mark the minimum bounding rectangle of a non-minimal target in a two-dimensional image of the monitoring area whose pixel size is greater than or equal to a preset pixel size;

[0024] For extremely small targets in a two-dimensional image within the monitored area whose pixels are smaller than a preset pixel size, the location and category of the target's center point are marked.

[0025] Furthermore, the training unit is specifically used to take the target center in the feature map as the center of the circle and calculate the radius of the circle according to the target size; generate a two-dimensional Gaussian function-filled heatmap through the center and radius, and train the target center prediction branch network according to the regression method of the two-dimensional Gaussian function-filled heatmap.

[0026] Furthermore, the training unit is specifically used to perform minimum target judgment on the feature vector corresponding to the target center in the feature map through a binary classifier, thereby completing the training of the minimum target judgment branch network.

[0027] Furthermore, the training unit is specifically used to generate the width and height of the target through the feature vector corresponding to the non-minimal target center in the feature image, and to obtain the width and height of the target through regression training.

[0028] This invention provides a target detection method and system based on two-dimensional images. The method involves labeling vehicle targets in two-dimensional images of a monitored area to generate a training dataset. Then, the training dataset is used to train the target center prediction branch network, the minimum target prediction branch network, and the target scale prediction branch network of the deep learning target detection model. The trained deep learning target detection model is then used to detect and identify vehicle targets. Because this invention employs different methods for labeling and identifying targets of different image resolutions and sizes, even with an expanded shooting range, some extremely small targets can be identified and detected using a single pixel, thereby improving the accuracy of target recognition. Attached Figure Description

[0029] Figure 1 This is a flowchart of a target detection method based on two-dimensional images provided by the present invention;

[0030] Figure 2 This is a schematic diagram of the structure of a target detection system based on two-dimensional images provided by the present invention;

[0031] Figure 3 This is a schematic diagram of the labeling method for non-minimal targets and minimal targets provided by the present invention. Detailed Implementation

[0032] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0033] like Figure 1 As shown in the figure, an embodiment of the present invention provides a target detection method based on two-dimensional images, which includes the following steps:

[0034] 101. Label vehicle targets in the two-dimensional images of the collected monitoring area to generate a training dataset.

[0035] In this embodiment of the invention, step 101 may specifically include: marking the minimum bounding rectangle of non-minimal targets with pixels greater than or equal to a preset pixel size in the two-dimensional image of the monitoring area; and marking the position and category of the center point of the target for minimal targets with pixels smaller than the preset pixel size in the two-dimensional image of the monitoring area.

[0036] Specifically, for example, the data used in this embodiment of the invention are two-dimensional images collected in an intelligent traffic monitoring scenario, with vehicle detection as the target. The annotation of the training dataset consists of two parts: for non-minimal targets of 20x20 pixels or larger, the minimum bounding rectangle of the target is annotated, i.e., (x,y,w,h,1), where x,y,w,h represent the coordinates of the center point and the width and height of the bounding rectangle model of the target, and the label 1 indicates that the target is a non-minimal target; for minimal targets smaller than 20x20 pixels, only the target's center point position and category are annotated, i.e., (x,y,0), and the label 0 indicates that the target is a minimal target. The annotation method for minimal and non-minimal targets in the training set images can be as follows: Figure 3 As shown, multiple images are collected and obtained according to the annotation method to form a training dataset.

[0037] 102. Extract feature maps from the two-dimensional images of the input training dataset through the backbone network of the deep learning object detection model. Input the feature maps into the target center prediction branch network of the deep learning object detection model, and determine the minimum target branch network and the target scale prediction branch network to obtain the target center position, whether the target is a minimum target label, and the target size, respectively.

[0038] In this embodiment of the invention, the backbone network of the deep learning object detection model is a ResNet network composed of multiple convolutional layers, and the Feature Pyramid Module (FPN) is used to connect and fuse semantic features of different feature layers across layers to meet the needs of object detection at different scales.

[0039] 103. The training dataset is used to train the backbone network of the deep learning object detection model, the target center prediction branch network, the minimum target prediction branch network, and the target scale prediction branch network.

[0040] In this embodiment of the invention, step 103 may specifically include: after extracting the feature map through the backbone network, obtaining different parameters of the target position through three branches, the three branches including a target center prediction branch network, a minimum target determination branch network, and a target height prediction branch network, all of which are neural networks composed of multiple convolutional layers. Specifically, the target center prediction branch network predicts a heatmap of the target center point distribution, the minimum target determination branch network uses a classifier to determine whether the target at the corresponding center point position is a minimum target, and the target height prediction branch network predicts the width and height of non-minimum targets at the corresponding center point position.

[0041] Specifically, a normalized RGB image with dimensions of 512 pixels is used as input. The backbone network extracts a 3D feature map f with a scale of 64*128*128. The feature map f is fed into three branch networks for prediction. The heatmap size for the predicted target center is 1x128x128, and the predicted length and width are 2x128x128, where 2 represents the length and width. The branch size for judging the minimum target is 2x128x128, where 2 represents the binary classification of non-maximum targets and maximum targets.

[0042] For example, a deep learning object detection model is trained based on a training dataset. Specifically, the training dataset is used to train the object detection model. The labeled information in the training set includes the target center, whether the target is a minimal target, and the target size, which are used to supervise the training of the target center prediction branch network, the minimal target judgment branch network, and the target scale prediction branch network, respectively. For the target center point prediction, in this embodiment, it is trained using a center heatmap regression method. Based on the target position pairs labeled in the training set data, the target position is scaled to a 128x128 scale. The target center is used as the center, and the radius R of the circle is calculated according to the target size. The value of the two-dimensional Gaussian function filling the heatmap is generated using this center and radius. For minimal targets without scale labels, a fixed value r is taken as the radius value. In the target center point regression training process, the loss function used is...

[0043] Where α and β are hyperparameters used to balance easy and difficult samples and positive and negative samples, and N is the number of keypoints in the image. The subscript xy of the summation symbol represents all coordinate points on all heatmaps. Y is the predicted value. xy Generate real values ​​for the labeled information.

[0044] Furthermore, the step of determining the minimum target branch network includes: for the feature vector corresponding to the target center, determining the minimum target using a binary classifier, with the cross-entropy loss function being... Where N is the number of non-minimal targets. This indicates the predicted label at the branch where the target center point is located, where the target is judged as the smallest target. k This represents the actual label marked at the point where the coordinates of the target center point are used to determine the minimum target branch.

[0045] Furthermore, the target scale prediction branch network is trained by generating the target's width and height using the feature vectors corresponding to the non-minimal target centers, and then training the network using a regression method to obtain the target's width and height. The regression loss function is as follows: Where N is the number of non-minimal targets. w represents the predicted width and height at the target-scale prediction branch corresponding to the coordinates of the non-minimal target center point. k h k These represent the width and height of the non-minimal target annotation, respectively.

[0046] 104. Vehicle targets are detected and identified using a trained deep learning target detection model.

[0047] In this embodiment of the invention, step 104 may specifically include: inputting the image to be detected into the target detection model, and after feature extraction, outputting the target's center point, whether its type is a small target, and the target's width and height at the target terminal. For objects that are not small targets, the target's position is represented by a bounding box rectangle formed by the center point and the width. For extremely small target objects, only the midpoint position is used to represent the target object's position.

[0048] This invention provides a target detection method based on two-dimensional images. The method involves labeling vehicle targets in two-dimensional images of a monitored area to generate a training dataset. Then, the training dataset is used to train the target center prediction branch network, the minimum target prediction branch network, and the target scale prediction branch network of the deep learning target detection model. The trained deep learning target detection model is then used to detect and identify vehicle targets. Because this invention employs different methods for labeling and identifying targets of different image resolutions and sizes, even with an expanded shooting range, some extremely small targets can be identified and detected using a single pixel, thereby improving the accuracy of target recognition.

[0049] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide a target detection system based on two-dimensional images, such as... Figure 2 As shown, the system includes: a generation unit 21, a training unit 22, and a detection and recognition unit 23.

[0050] The generation unit 21 is used to annotate vehicle targets in the two-dimensional images of the acquired monitoring area to generate a training dataset.

[0051] The training unit is used to extract feature maps from the two-dimensional images of the input training dataset through the backbone network of the deep learning object detection model, and input the feature maps into the target center prediction branch network, the minimum target judgment branch network and the target scale prediction branch network of the deep learning object detection model to obtain the target center position, whether the target is a minimum target label and the target size, respectively. The training unit is used to train the backbone network of the deep learning object detection model and the target center prediction branch network, the minimum target judgment branch network and the target scale prediction branch network through the training dataset.

[0052] The detection and recognition unit 23 is used to detect and recognize vehicle targets using a trained deep learning target detection model.

[0053] Furthermore, the generation unit 21 is specifically used to mark the minimum bounding rectangle of non-minimal targets with pixels greater than or equal to a preset pixel size in the two-dimensional image of the monitoring area; and to mark the position and category of the center point of the target for minimal targets with pixels smaller than the preset pixel size in the two-dimensional image of the monitoring area.

[0054] Furthermore, the training unit 22 is specifically used to take the target center in the feature map as the center of the circle and calculate the radius of the circle according to the target size; generate a two-dimensional Gaussian function-filled heatmap through the center and radius, and train the target center prediction branch network according to the regression method of the two-dimensional Gaussian function-filled heatmap.

[0055] Furthermore, the training unit 22 is specifically used to perform minimum target judgment on the feature vector corresponding to the target center in the feature map through a binary classifier, thereby completing the training of the minimum target judgment branch network.

[0056] Furthermore, the training unit 22 is specifically used to generate the width and height of the target through the feature vector corresponding to the non-minimal target center in the feature image, and to obtain the width and height of the target through regression training.

[0057] This invention provides a target detection system based on two-dimensional images. The system annotates vehicle targets in two-dimensional images of a monitored area to generate a training dataset. Then, it uses the training dataset to train the target center prediction branch network, the minimum target prediction branch network, and the target scale prediction branch network of the deep learning target detection model. The trained deep learning target detection model is then used to detect and identify vehicle targets. Because this invention uses different methods for annotation and identification of targets with different image resolutions and sizes, even as the shooting range expands, some extremely small targets can be identified and detected using a single pixel, thereby improving the accuracy of target recognition.

[0058] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0059] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0060] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0061] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0062] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0063] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic system, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing systems, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0064] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0065] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage systems, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server, or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0066] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method based on two-dimensional images, characterized in that, The method includes: Vehicle targets are labeled in the two-dimensional images of the acquired monitoring area to generate a training dataset; The input 2D image is processed by the backbone network of the deep learning object detection model to extract feature maps. The feature maps are then input into the target center prediction branch network of the deep learning object detection model. The minimum target branch network and the target scale prediction branch network are used to determine the target center position, whether the target is a minimum target label, and the target size, respectively. The training dataset is used to train the backbone network of the deep learning object detection model and the target center prediction branch network, as well as the minimum target branch network and the target scale prediction branch network. Vehicle targets are detected and identified using a trained deep learning target detection model; The steps of annotating vehicle targets in the acquired two-dimensional images of the monitoring area to generate a training dataset include: For non-minimal targets in the two-dimensional image of the monitoring area with pixels greater than or equal to a preset pixel size, mark the minimum bounding rectangle and category of the target; For extremely small targets in a two-dimensional image within the monitored area whose pixels are smaller than a preset pixel size, mark the location and category of the target center; The steps for training the target center prediction branch network based on the target center location include: Using the center of the target in the feature map as the center of the circle, and calculating the radius of the circle based on the target size; A two-dimensional Gaussian function-filled heatmap is generated using the center and radius of the circle, and the target center prediction branch network is trained based on the regression method of the two-dimensional Gaussian function-filled heatmap. The steps for training the minimal target identification branch network based on whether the target is labeled as a minimal target include: The training of the branch network for judging the smallest target is completed by using the feature vector corresponding to the target center in the feature map as input to a binary classifier. The steps of training the target scale prediction branch network based on the target size include: The width and height of the target are generated by using the feature vector corresponding to the center of the non-minimal target in the feature image, and the width and height of the target are obtained by training through regression.

2. A target detection system based on two-dimensional images, characterized in that, The system includes: The generation unit is used to annotate vehicle targets in the two-dimensional images of the acquired monitoring area and generate a training dataset. The training unit is used to extract feature maps from the two-dimensional images of the input training dataset through the backbone network of the deep learning object detection model, input the feature maps into the target center prediction branch network of the deep learning object detection model, the minimum target prediction branch network and the target scale prediction branch network, and obtain the target center position, whether the target is a minimum target label and the target size, respectively. The training unit is used to train the backbone network of the deep learning object detection model and the target center prediction branch network, the minimum target prediction branch network and the target scale prediction branch network through the training dataset. The detection and recognition unit is used to detect and recognize targets using a trained deep learning target detection model. The generation unit is specifically used to mark the minimum bounding rectangle and category of non-minimal targets with pixels greater than or equal to a preset pixel size in a two-dimensional image of the monitoring area. For extremely small targets in a two-dimensional image within the monitored area whose pixels are smaller than a preset pixel size, mark the location and category of the target center; The training unit is specifically used to take the target center in the feature map as the center of the circle and calculate the radius of the circle according to the target size; generate a two-dimensional Gaussian function-filled heatmap through the center and radius, and train the target center prediction branch network according to the regression method of the two-dimensional Gaussian function-filled heatmap; The training unit uses the feature vector corresponding to the target center in the feature map as input to a binary classifier to complete the training of the branch network for judging the smallest target. The training unit is further configured to generate the width and height of the target using the feature vector corresponding to the non-minimal target center in the feature image, and to obtain the width and height of the target through regression training.

Images