A vehicle detection method and system based on grid annotation

Abstract

The present invention belongs to the field of computer technology, and more specifically relates to a vehicle detection method and system based on grid annotation. The embodiments of the present application provide a vehicle detection method and system based on grid annotation to address the issues of existing vehicle detection methods based on convolutional neural networks, which require a large amount of annotation and are susceptible to viewing angles.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a vehicle detection method and system based on grid annotation. Background Art

[0002] With the increasing number of vehicles in recent years, traffic congestion has become increasingly severe, leading to a growing demand from transportation authorities for technology that can detect the number of vehicles on each road in real time. Traditional manual detection methods are both labor-intensive and inaccurate. Furthermore, current mainstream machine vision-based vehicle detection methods only perform well in static images, but their performance in dynamic video falls short of the technical requirements. Existing vehicle detection technologies primarily address dynamic detection by training models on video images using convolutional neural networks. For example, one-stage methods employ the YOLO family of object detectors, while two-stage methods employ Faster R-CNN. However, convolutional neural network training relies on large amounts of annotated data. For example, vehicle detection requires adding a bounding box around the vehicle to the test image and extensively annotating the box. Furthermore, this annotation is affected by the vehicle's viewing angle, significantly increasing the amount of annotation required. Training an ideal vehicle detection model requires at least 10,000 images to be manually annotated. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to propose a vehicle detection method and system based on grid annotation to solve the problem that the vehicle detection method based on convolutional neural network in the prior art requires a large amount of annotation and is easily affected by viewing angle.

[0004] In a first aspect, an embodiment of the present application provides a vehicle detection method based on grid annotation, the method comprising the following steps:

[0005] S100, constructing a normalized geometric scene for the area to be measured, defining a normalized scene image of the area to be measured, capturing a fixed-viewing angle image with a fixed camera having a fixed viewing angle, and establishing a mapping relationship between the normalized scene image and the fixed-viewing angle image;

[0006] S200, transforming the captured actual test image by using the mapping relationship to obtain a normalized test image, then performing grid annotation on the normalized test image to obtain a grid-annotated image, and generating training samples based on possible location information of vehicles on the grid-annotated image;

[0007] S300: Train the training samples to obtain a vehicle detection model.

[0008] In some embodiments, the method steps for generating training samples in step S200 include: dividing the normalized test image into n test areas, each test area corresponds to a value of 0 or 1, setting the vehicle's position labeling information to an n-dimensional vector according to the number of test areas, setting the value of the test area where the vehicle is parked to 1, and setting the values ​​of the remaining test areas to 0, and marking all possible vehicle position labeling information to generate training samples.

[0009] This operation converts the traditional annotation of the three-dimensional rectangular box into the annotation of the vehicle's relative position to the scene, which can greatly simplify the annotation information and reduce the amount of annotation.

[0010] In some embodiments, the number of test areas occupied by each parked vehicle is 1, 2, ..., or n.

[0011] In real life, the sizes of vehicles of different models and brands vary greatly, and the occupancy of the test area by a single vehicle varies. This operation can make the applicability of this method more robust.

[0012] In some embodiments, the specific steps of transforming the captured actual test image by means of the mapping relationship in step S200 include:

[0013] Mark the coordinates of the normalized scene image;

[0014] The fixed-view image is annotated with coordinates, a mapping relationship between the fixed-view image and the normalized scene image is established, and the coordinate transformation matrix between the fixed-view image and the normalized scene image is obtained through training;

[0015] The actual test image captured is brought into the corresponding coordinate transformation matrix according to the shooting angle to obtain the normalized test image

[0016] This operation can convert images from different perspectives into fixed-perspective images, solving the problem that different perspectives affect the annotation information and thus cause a large amount of annotation.

[0017] In some embodiments, the specific method of training the training samples in step 300 is based on supervised neural network model training.

[0018] Supervised neural network technology can output the optimal model after training, making the test results in practical applications controllable.

[0019] In some embodiments, the supervised neural network model includes: YOLO algorithm, ResNet algorithm or convolutional neural network.

[0020] The above-mentioned YOLO algorithm, ResNet algorithm and convolutional neural network model are only preferred examples of supervised neural network models suitable for this method, and are not exhaustive. Other supervised neural network models can also be applied to this method.

[0021] In some embodiments, the training step in step 300 includes incorporating a three-dimensional model of a known vehicle into the training samples.

[0022] Incorporating the three-dimensional model of known vehicles into the training samples makes it easier to identify the positional relationship between the vehicle and the area to be tested, including the number of occupied vehicles, specific orientation, etc. There is no need to label the three-dimensional model frame like traditional neural network algorithms, which greatly saves the amount of labeling.

[0023] In a second aspect, an embodiment of the present application provides a vehicle detection system based on grid annotation, the system comprising:

[0024] a normalization module configured to construct a normalized geometric scene for the area to be measured, define a normalized scene image of the area to be measured, capture the fixed-view image with a fixed camera having a fixed view, and establish a mapping relationship between the normalized scene image and the fixed-view image;

[0025] a sample generation module configured to transform the captured actual test image by using the mapping relationship to obtain a normalized test image, then perform grid annotation on the normalized test image to obtain a grid-annotated image, and generate training samples based on possible location information of the vehicle on the grid-annotated image;

[0026] The sample training module is configured to train the training samples to obtain a vehicle detection model.

[0027] In some embodiments, the sample generation module includes:

[0028] An image transformation module, the image transformation module is configured to perform coordinate annotation on the normalized scene image;

[0029] The fixed-view image is annotated with coordinates, a mapping relationship between the fixed-view image and the normalized scene image is established, and the coordinate transformation matrix between the fixed-view image and the normalized scene image is obtained through training;

[0030] The actual test image captured is brought into the corresponding coordinate transformation matrix according to the shooting angle to obtain a normalized test image;

[0031] The gridding module is configured to divide the normalized test image into n test areas, each test area corresponds to a value of 0 or 1, and the vehicle position annotation information is set to an n-dimensional vector according to the number of test areas. The value of the test area where the vehicle is parked is set to 1, and the values ​​of the remaining test areas are set to 0. All possible vehicle position annotation information is annotated to generate training samples.

[0032] This module processes the training samples and converts the test images of different perspectives into normalized images of fixed perspective to solve the influence of different perspectives on the detection results. At the same time, the test images are annotated in a grid manner, which greatly reduces the required annotation amount and solves the problem of large annotation amount in traditional vehicle detection methods.

[0033] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the above-mentioned first aspects is implemented.

[0034] The embodiments of the present application provide a vehicle detection method and system based on grid annotation. By using a mapping method to convert test images of different perspectives into normalized images of a fixed perspective, the influence of different perspectives on the detection results is solved. At the same time, grid annotation of the test images greatly reduces the required annotation amount, thus solving the problem of large annotation amount in traditional vehicle detection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0036] Figure 1 is an exemplary basic flow chart according to an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of a fixed-viewing angle image in a vehicle detection method based on grid annotation according to an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of a normalized scene image in a vehicle detection method based on grid annotation according to an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of a gridding of a region to be detected in a vehicle detection method based on gridding annotation according to an embodiment of the present invention;

[0040] Figure 5 3 is a schematic structural diagram of a vehicle detection system based on grid annotation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0042] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] Figure 1 An exemplary basic process of the vehicle detection method based on grid annotation of the present invention is shown.

[0044] like Figure 1 As shown, the basic process includes:

[0045] Step S100: construct a normalized geometric scene for the area to be measured, define a normalized scene image of the area to be measured, capture the fixed-viewing angle image with a fixed camera having a fixed viewing angle, and establish a mapping relationship between the normalized scene image and the fixed-viewing angle image;

[0046] Step S200: transforming the captured actual test image using the mapping relationship to obtain a normalized test image, then performing grid annotation on the normalized test image to obtain a grid-annotated image, and generating training samples based on possible location information of the vehicle on the grid-annotated image;

[0047] Step S300: training the training samples to obtain a vehicle detection model.

[0048] In some embodiments, step S100 may be specifically implemented as follows:

[0049] 1) If Figure 2 As shown, in the area to be measured, a capture camera of the traffic scene and a ground sensing coil cooperate to capture and obtain a fixed-viewing angle image through the viewing angle of the capture camera.

[0050] 2) If Figure 2 As shown, areas A and B are randomly selected as annotation points on the fixed-view image, and the coordinates of the four vertices of area A and area B are manually marked, which are recorded as p1, p2...p8 respectively.

[0051] 3) If Figure 3 As shown, the normalized scene image of the region is defined, and the four vertex coordinates of region A and region B in the normalized scene image are manually marked and recorded as q1, q2...q8 respectively.

[0052] 4) Establish a mapping relationship between fixed-view images and normalized scene images.

[0053] Assume that regions A and B are 2m*2m in size and set pi = A*qi + b. Pi and qi are the image coordinates in the fixed-view image and the normalized scene image, respectively. A is a 2*2 matrix, and b is a 2*1 column vector. Since the coordinates of pi and qi are known from the manual annotation in steps 2) and 3), the goal here is to find A and b.

[0054] The specific solution method is as follows: First, construct an objective function about A and b, as shown below:

[0055]

[0056] Then, the solution is obtained using the Gauss-Newton method.

[0057] Here, we use a single traffic camera in a traffic scene as an example. In practice, traffic cameras have diverse viewpoints, resulting in different mapping relationships depending on the camera's position. Training can capture all possible mapping relationships. Images from different viewpoints can be converted into normalized scene images, resolving the issue of varying viewpoints affecting annotation information and resulting in a large amount of annotation.

[0058] In some embodiments, step S200 may be specifically implemented as follows:

[0059] 1) In the test area, actual test images are captured by using a traffic scene capture camera and a ground sensor coil. The capture cameras here are not limited to a fixed angle of view or a fixed number, so the actual test images obtained have diverse angles of view.

[0060] 2) The actual test image is normalized using all possible mapping relationships obtained in the above-mentioned implementation methods of step S100 to obtain a normalized test image.

[0061] 3) If Figure 4 As shown, the normalized test image is divided into n test regions, centered around regions A and B. Each test region corresponds to a value of 0 or 1. The vehicle position annotation information is then assigned an n-dimensional vector based on the number of test regions. The value of each region with a parked vehicle is set to 1, while the values ​​of all other regions are set to 0. This operation transforms the traditional annotation of 3D rectangular boxes into annotation of the vehicle's position relative to the scene, greatly simplifying the annotation information and reducing the amount of annotation required.

[0062] For example, if a vehicle is located in areas 2 and 3, the location information of the vehicle is an n-dimensional vector, where the second and third dimensions are 1 and the other dimensions are 0.

[0063] 4) Label all possible vehicle location information to generate training samples.

[0064] In some embodiments, step S300 may be specifically implemented as follows:

[0065] 1) Obtain the CAD model of the mainstream molding currently on the market.

[0066] 2) Using the normalized scene image as the base map, the vehicle CAD model is overlaid on top. By adding illumination and combining the positional annotation information of the training samples obtained in the aforementioned implementation of step S200, a final training sample is obtained using computer graphics 3D imaging methods. Incorporating the 3D model of the known vehicle into the training sample facilitates identification of the positional relationship between the vehicle and the area to be measured, including the number of occupied vehicles and their specific orientation. This eliminates the need to annotate the 3D model frame as with traditional neural network algorithms, significantly reducing the amount of annotation required.

[0067] 3) The final training samples, including their annotated information, are fed into a supervised neural network model to train a vehicle detection model. Supervised neural network models include the YOLO algorithm, ResNet algorithm, or convolutional neural network. Supervised neural network technology can output an optimal model through training, making test results controllable in practical applications.

[0068] 4) Finally, the newly captured actual traffic image is substituted into the vehicle detection model to obtain the vehicle position information in the image.

[0069] refer to Figure 5 , which shows a structural diagram of a vehicle detection system 400 based on grid annotation according to an embodiment of the present invention.

[0070] like Figure 5 As shown, the system 400 includes a normalization module 410, a sample generation module 420, and a sample training module 430, wherein the sample generation module 420 further includes an image transformation module 421 and a gridding module 422. The specific description is as follows:

[0071] Normalization module 410, which is configured to construct a normalized geometric scene for the area to be measured, define a normalized scene image of the area to be measured, capture a fixed-viewing angle image using a fixed camera with a fixed viewing angle, and establish a mapping relationship between the normalized scene image and the fixed-viewing angle image;

[0072] Sample generation module 420, which is configured to transform the captured actual test image using the mapping relationship to obtain a normalized test image, then perform grid annotation on the normalized test image to obtain a grid-annotated image, and generate training samples based on possible vehicle location information on the grid-annotated image;

[0073] The sample training module 430 is configured to train the training samples to obtain a vehicle detection model.

[0074] In some feasible embodiments of the vehicle detection system 400 based on grid annotation according to an embodiment of the present invention, the sample generation module 420 further includes:

[0075] An image transformation module 421 is configured to perform coordinate annotation on the normalized scene image;

[0076] The fixed-view image is annotated with coordinates, a mapping relationship between the fixed-view image and the normalized scene image is established, and the coordinate transformation matrix between the fixed-view image and the normalized scene image is obtained through training;

[0077] The actual test image captured is brought into the corresponding coordinate transformation matrix according to the shooting angle to obtain a normalized test image;

[0078] The gridding module 422 is configured to divide the image after the matrix transformation into n test areas, each test area corresponds to a value of 0 or 1, and the vehicle position labeling information is set to an n-dimensional vector according to the number of test areas. The value of the test area where the vehicle is parked is set to 1, and the values ​​of the remaining test areas are set to 0. All possible position labeling information of the vehicle is marked to generate the training sample.

[0079] The training samples are processed through the sample generation module, and the test images of different perspectives are converted into normalized scene images of fixed perspective to solve the impact of different perspectives on the detection results. At the same time, the test images are grid-annotated, which greatly reduces the required annotation amount and solves the problem of large annotation amount in traditional vehicle detection methods.

[0080] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. The computer-readable storage medium described in this application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, or any suitable combination thereof.

[0081] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0082] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the prescribed logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented by a dedicated hardware-based system that performs the prescribed function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0083] The modules described in the embodiments of this application may be implemented in software or hardware. The modules described may also be provided in a processor. For example, a processor may be described as including an acquisition module, an analysis module, and an output module. The names of these modules do not, in some cases, limit the modules themselves.

[0084] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A vehicle detection method based on grid annotation, characterized by: The steps of the method include: S100, constructing a normalized geometric scene for the area to be measured, and defining a normalized scene image of the area to be measured, shooting a fixed-angle image with a fixed camera having a fixed angle of view, and establishing a mapping relationship between the normalized scene image and the fixed-angle image, specifically including: arbitrarily selecting areas A and B on the fixed-angle image as annotation points, marking the four vertex coordinates of areas A and B, which are recorded as p1, p2...p8 respectively; defining a normalized scene image of the area, marking the four vertex coordinates of areas A and B in the normalized scene image, which are recorded as q1, q2...q8 respectively; setting pi=A*qi+b, where pi and qi are the image coordinates on the fixed-angle image and the normalized scene image respectively; constructing an objective function with respect to A and b, as shown below, Solving the mapping relationship between the normalized scene image and the fixed-viewing angle image by using the Gauss-Newton method; S200, transforming the captured actual test image by means of the mapping relationship to obtain a normalized test image, then grid-annotating the normalized test image to obtain a grid-annotated image, and generating training samples based on possible position information of the vehicle on the grid-annotated image, wherein the method specifically includes dividing the normalized test image into n test areas, each of the test areas corresponding to a value of 0 or 1, setting the position annotation information of the vehicle to an n-dimensional vector according to the number of the test areas, setting the value of the test area where the vehicle is parked to 1, and setting the values ​​of the remaining test areas to 0, and annotating all possible position annotation information of the vehicle to generate the training samples; S300, integrating the three-dimensional model of the known vehicle into the training sample, specifically including: using the normalized scene image as a base map, superimposing the three-dimensional model of the known vehicle on the base map, increasing lighting, combining the position annotation information of the training sample obtained in the above step S200, using the three-dimensional imaging method of computer graphics, obtaining the final training sample, and training the training sample to obtain a vehicle detection model.

2. The method according to claim 1, wherein: The number of the test areas occupied by each parked vehicle is 1, 2... or n.

3. The method according to claim 1, wherein: The specific steps of transforming the captured actual test image by means of the mapping relationship in step S200 include: Performing coordinate marking on the normalized scene image; Annotating the fixed-view image with coordinates, establishing a mapping relationship between the fixed-view image and the normalized scene image, and training to obtain a coordinate transformation matrix between the fixed-view image and the normalized scene image; The actual test image captured is brought into the corresponding coordinate transformation matrix according to the shooting angle of view to obtain the normalized test image.

4. The method according to claim 1, wherein: The specific method of training the training samples in step 300 is based on supervised neural network model training.

5. The method according to claim 4, characterized in that: The supervised neural network model includes: YOLO algorithm, ResNet algorithm or convolutional neural network.

6. A vehicle detection system based on grid annotation, characterized by: The system comprises: A normalization module is configured to construct a normalized geometric scene for the area to be measured, and define a normalized scene image of the area to be measured, shoot a fixed-angle image with a fixed camera with a fixed angle of view, and establish a mapping relationship between the normalized scene image and the fixed-angle image, specifically including: arbitrarily selecting areas A and B as annotation points on the fixed-angle image, marking the four vertex coordinates of areas A and B, respectively denoted as p1, p2...p8; defining the normalized scene image of the area, marking the four vertex coordinates of areas A and B in the normalized scene image, respectively denoted as q1, q2...q8; setting pi=A*qi+b, where pi and qi are the image coordinates on the fixed-angle image and the normalized scene image, respectively; constructing an objective function about A and b, as shown below, Solving the mapping relationship between the normalized scene image and the fixed-viewing angle image by using the Gauss-Newton method; A sample generation module, wherein the sample generation module is configured to transform the actual test image captured by means of the mapping relationship to obtain a normalized test image, and then grid-annotate the normalized test image to obtain a grid-annotated image, and generate training samples based on possible position information of the vehicle on the grid-annotated image. The sample generation module includes a gridding module, and the gridding module is configured to divide the normalized test image into n test areas, each of which corresponds to a value of 0 or 1, and set the vehicle's position annotation information to an n-dimensional vector according to the number of test areas, setting the value of the test area where the vehicle is parked to 1, and setting the values ​​of the remaining test areas to 0, and annotating all possible position annotation information of the vehicle to generate the training samples; A sample training module is configured to integrate the three-dimensional model of the known vehicle into the training sample, specifically including: using the normalized scene image as a base map, superimposing the three-dimensional model of the known vehicle on the base map, increasing lighting, combining the position annotation information of the training sample obtained by the above-mentioned sample generation module, using the three-dimensional imaging method of computer graphics to obtain the final training sample, and training the training sample to obtain a vehicle detection model.

7. The vehicle detection system based on grid annotation according to claim 6, characterized in that: The sample generation module includes: An image transformation module configured to perform coordinate annotation on the normalized scene image; Annotating the fixed-view image with coordinates, establishing a mapping relationship between the fixed-view image and the normalized scene image, and training to obtain a coordinate transformation matrix between the fixed-view image and the normalized scene image; The actual test image captured is brought into the corresponding coordinate transformation matrix according to the shooting angle of view to obtain the normalized test image.

8. A computer-readable storage medium, wherein a computer program is stored in the medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images