Training method of vehicle detection model, traffic state sensing method and system

Abstract

The invention discloses a training method of a vehicle detection model and a traffic state perception method and system.The training method comprises the steps that a panchromatic image in a single-temporal remote sensing image is obtained and preprocessed, and a panchromatic training image is obtained; performing image feature extraction on the panchromatic training image to obtain panchromatic image features; candidate region extraction is carried out on the panchromatic image features to obtain a target rotating anchor frame set, and the target rotating anchor frame set comprises a plurality of target rotating anchor frames corresponding to the target vehicle; performing rotation area pooling on the target rotation anchor frame set to obtain an anchor frame feature map; and according to the anchor frame feature map, performing parameter updating on the initialized vehicle detection model to obtain a trained vehicle detection model. According to the method, by providing the vehicle detection model, the detection precision and the detection efficiency of vehicle detection can be improved, the cost of traffic state perception is reduced, and the practicability of traffic state perception is improved. The invention relates to the technical field of vehicle detection.

Description

Technical Field

[0001] The present invention relates to the field of vehicle detection technology, and in particular to a vehicle detection model training method, a traffic state perception method and a system. Background Art

[0002] At present, the traditional traffic status perception method based on remote sensing images usually realizes vehicle detection based on multi-temporal remote sensing images, and then realizes traffic status perception based on the detected multi-temporal vehicle data. However, this method has poor detection accuracy and efficiency when performing vehicle detection, and due to the difficulty and high cost of obtaining multi-temporal remote sensing images, the practicality of this method for traffic status perception is unsatisfactory.

[0003] Therefore, the problems existing in the prior art still need to be solved and optimized. Summary of the invention

[0004] The purpose of the present invention is to solve one of the technical problems existing in the related art to at least a certain extent.

[0005] To this end, an object of an embodiment of the present invention is to provide a vehicle detection model training method, a traffic state perception method and a system, wherein the method can improve the detection accuracy and efficiency of vehicle detection, reduce the cost of traffic state perception, and improve the practicality of traffic state perception by providing a vehicle detection model.

[0006] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present application include:

[0007] In a first aspect, the embodiments of the present application provide a vehicle detection model training method, a traffic state perception method and a system, including:

[0008] Acquire a full-color image in a single-phase remote sensing image of a target vehicle, and preprocess the full-color image to obtain a full-color training image;

[0009] Performing image feature extraction on the full-color training image to obtain full-color image features;

[0010] Extracting candidate regions from the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different bounding box scale, aspect ratio, or rotation angle;

[0011] Performing rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map;

[0012] According to the anchor frame feature map, the parameters of the initialized vehicle detection model are updated to obtain a trained vehicle detection model.

[0013] In addition, the method according to the above embodiment of the present application may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present application, the visual scene recognition model includes a plurality of cascaded dynamic pooling network frameworks, and the local features are subjected to multi-layer dynamic pooling aggregation to obtain splicing features, including:

[0015] Inputting the local features into the plurality of cascaded dynamic pooling network frameworks to obtain a plurality of image features, each of the image features corresponding to one of the dynamic pooling network frameworks;

[0016] Performing feature flattening on all the image features to obtain flattened features corresponding to different feature levels;

[0017] Feature splicing is performed on all the flattened features to obtain the spliced ​​features.

[0018] Furthermore, in one embodiment of the present application, extracting a candidate region from the panchromatic image feature to obtain a target rotation anchor frame set corresponding to the panchromatic image feature includes:

[0019] Performing a rotation anchor frame generation process on the panchromatic image feature to obtain a first intermediate anchor frame set, wherein the first intermediate anchor frame set includes a plurality of intermediate rotation anchor frames corresponding to the panchromatic image feature, and each of the intermediate rotation anchor frames corresponds to a region scale, an aspect ratio, or a rotation angle that is different from each other;

[0020] Performing anchor frame feature mapping on the first intermediate anchor frame set to obtain a second intermediate anchor frame set;

[0021] Perform bounding box classification regression on the second intermediate anchor frame set to obtain the target rotation anchor frame set.

[0022] Further, in one embodiment of the present application, performing bounding box classification regression on the second intermediate anchor frame set to obtain the target rotation anchor frame set includes:

[0023] Performing anchor frame classification on the second intermediate anchor frame set to obtain a third intermediate anchor frame set, where the third intermediate anchor frame set is a set of intermediate rotation anchor frames corresponding to the target vehicle;

[0024] The third intermediate anchor frame set is updated by bounding box regression offset to obtain the target rotation anchor frame set.

[0025] Further, in one embodiment of the present application, performing bounding box regression offset update on the third intermediate anchor frame set to obtain the target rotation anchor frame set includes:

[0026] Performing border offset regression prediction on the third intermediate anchor frame set to obtain border offset prediction data;

[0027] The third intermediate anchor frame set is updated with a border offset according to the border offset prediction data to obtain the target rotation anchor frame set.

[0028] Further, in one embodiment of the present application, performing rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map includes:

[0029] Performing a rotation coordinate transformation on the target rotation anchor frame set to obtain a fourth rotation anchor frame set, wherein each target rotation anchor frame in the fourth rotation anchor frame set has the same rotation angle;

[0030] Performing regional scale cropping on the fourth rotated anchor frame set to obtain a fifth rotated anchor frame set;

[0031] Perform feature map pooling mapping on the fifth rotated anchor frame set to obtain the anchor frame feature map.

[0032] In a second aspect, an embodiment of the present application provides a traffic state perception method of a vehicle detection model, comprising:

[0033] Acquire traffic network data and target single-temporal remote sensing images, wherein the target single-temporal remote sensing images include target panchromatic images and target multispectral images;

[0034] Inputting the target full-color image into the trained vehicle detection model to perform vehicle target detection and obtain a vehicle target detection result;

[0035] According to the vehicle target detection result, the vehicle speed analysis processing is performed on the target multispectral image to obtain vehicle speed data;

[0036] The traffic state perception is performed on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain a traffic state perception result.

[0037] Further, in the embodiment of the present application, the vehicle speed analysis processing is performed on the target multispectral image according to the vehicle target detection result to obtain the vehicle speed data, including:

[0038] According to the target multispectral image, acquiring a first band spectral image and a second band spectral image;

[0039] According to the vehicle target detection result, extracting a first candidate region from the first band spectral image to obtain a first band candidate region, and according to the vehicle target detection result, extracting a second candidate region from the second band spectral image to obtain a second band candidate region;

[0040] According to the first band candidate area, regional displacement matching is performed on the second band candidate area to obtain the vehicle speed data.

[0041] Further, in the embodiment of the present application, performing regional displacement matching on the second band candidate area according to the first band candidate area to obtain the vehicle speed data includes:

[0042] Acquiring a spectral imaging time difference of the target multispectral image;

[0043] According to the first band candidate area, template matching is performed on the second band candidate area to obtain first vehicle pixel coordinate data and second vehicle pixel coordinate data, wherein the first vehicle pixel coordinate data is pixel coordinate data of the moving vehicle in the first band candidate area, and the second vehicle pixel coordinate data is pixel coordinate data of the moving vehicle in the second band candidate area;

[0044] According to the first vehicle pixel coordinate data, performing vehicle displacement calculation on the second vehicle pixel coordinate data to obtain vehicle displacement data;

[0045] The vehicle speed data is calculated based on the spectral imaging time difference to obtain the vehicle speed data.

[0046] Further, in the embodiment of the present application, the traffic state perception is performed on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain the traffic state perception result, including:

[0047] According to the vehicle target detection result, performing vehicle space matching on the traffic network data to obtain vehicle space matching data;

[0048] The traffic state is evaluated on the vehicle space matching data according to the vehicle speed data to obtain the traffic state perception result.

[0049] In a third aspect, an embodiment of the present application provides a vehicle detection model training system, including:

[0050] A first processing unit is used to obtain a full-color image in a single-phase remote sensing image of a target vehicle, and pre-process the full-color image to obtain a full-color training image;

[0051] A second processing unit is used to extract image features from the full-color training image to obtain full-color image features;

[0052] A third processing unit is used to extract a candidate region from the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different bounding box scale, aspect ratio or rotation angle;

[0053] a fourth processing unit, configured to perform rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map;

[0054] The fifth processing unit is used to update the parameters of the initialized vehicle detection model according to the anchor frame feature map to obtain a trained vehicle detection model.

[0055] In a fourth aspect, an embodiment of the present application provides a traffic state perception system of a vehicle detection model, including:

[0056] A sixth processing unit, used for acquiring traffic network data and target single-temporal remote sensing images, wherein the target single-temporal remote sensing images include target panchromatic images and target multispectral images;

[0057] A seventh processing unit, used for inputting the target full-color image into the above-mentioned trained vehicle detection model to perform vehicle target detection and obtain a vehicle target detection result;

[0058] an eighth processing unit, configured to perform vehicle speed analysis processing on the target multispectral image according to the vehicle target detection result to obtain vehicle speed data;

[0059] The ninth unit is used to perform traffic state perception on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain a traffic state perception result.

[0060] In a fifth aspect, an embodiment of the present application further provides an electronic device, including:

[0061] at least one processor;

[0062] at least one memory for storing at least one program;

[0063] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0064] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above method when executed by the processor.

[0065] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0066] The embodiments of the present application disclose a training method for a vehicle detection model, a traffic state perception method and a system, wherein the training method obtains a full-color image in a single-phase remote sensing image of a target vehicle, and preprocesses the full-color image to obtain a full-color training image; performs image feature extraction on the full-color training image to obtain full-color image features; performs candidate region extraction on the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different border scale, aspect ratio or rotation angle; performs rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map; and performs parameter update on an initialized vehicle detection model according to the anchor frame feature map to obtain a trained vehicle detection model. The training method extracts candidate regions from full-color image features in single-phase remote sensing images to extract target rotation anchor frames with at least one different bounding box scale, aspect ratio or rotation angle, and performs rotation region pooling on all extracted target rotation anchor frames. This allows the trained vehicle detection model to identify vehicles of any orientation, position and size, and can effectively improve the detection accuracy and efficiency of vehicle detection. In addition, the traffic state perception method uses single-phase remote sensing images to perform vehicle target detection through a vehicle detection model, and then performs vehicle speed analysis and traffic state perception based on the obtained vehicle target detection results, which can effectively reduce the cost of traffic state perception and improve the practicability of traffic state perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the embodiments of the present application or the drawings of the related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly expressing some embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A flowchart of a method for training a vehicle detection model provided in an embodiment of the present application;

[0069] Figure 2 A flow chart of a traffic state perception method of a vehicle detection model provided in an embodiment of the present application;

[0070] Figure 3 A schematic diagram of the structural framework of a vehicle detection model training system provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limitations on the present application. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0074] First, the technical terms involved in the embodiments of the present application are explained as follows:

[0075] Single-phase remote sensing image: refers to the remote sensing image acquired by a high-spatial-resolution satellite at a specific point in time.

[0076] Multi-temporal remote sensing imagery: refers to multiple remote sensing images acquired by high spatial resolution satellites at different time points.

[0077] Panchromatic image (PAN): A black-and-white image acquired only in the visible light band, with the highest spatial resolution.

[0078] Multispectral imagery (MS): Color images acquired in multiple spectral bands, usually with lower spatial resolution than panchromatic images.

[0079] Ghost phenomenon: refers to the significant difference in the imaging position of high-speed moving objects in images of different spectral bands, resulting in repeated or blurred images in the synthetic image. Specifically, since high-spatial-resolution satellites are often equipped with multiple sensors, the images of different spectral bands acquired by high-spatial-resolution satellites at the same time will cause the imaging position of high-speed moving vehicles in two different spectral bands to shift due to the slight imaging time difference of the sensors.

[0080] At present, the traditional traffic state perception method based on remote sensing images usually realizes vehicle detection based on multi-temporal remote sensing images, and then realizes traffic state perception based on the detected multi-temporal vehicle data. However, since this method uses multi-temporal remote sensing images, it needs to perform vehicle detection for remote sensing images of different phases respectively when performing vehicle detection, which requires more computing resources. The generated candidate area is often a horizontal target rectangular frame, which cannot meet the detection requirements of vehicles in any direction well, and the detection accuracy and efficiency of vehicle detection are not good; in addition, this method requires high spatial resolution satellites to collect remote sensing images at different time points when obtaining multi-temporal remote sensing images, and the satellite resources of high spatial resolution satellites are relatively scarce. It is difficult and costly to obtain multi-temporal remote sensing images, and the practicality of traffic state perception is not satisfactory.

[0081] In view of this, an embodiment of the present invention provides a training method for a vehicle detection model, a traffic status perception method and a system, wherein the training method extracts candidate regions from full-color image features in a single-phase remote sensing image to extract target rotation anchor frames with at least one different bounding box scale, aspect ratio or rotation angle, and performs rotation region pooling on all extracted target rotation anchor frames, which enables the trained vehicle detection model to generate vehicle target detection results of any orientation, position and size based on the single-phase remote sensing image, and can effectively improve the detection accuracy and detection efficiency of vehicle detection; in addition, the traffic status perception method uses the single-phase remote sensing image to perform vehicle target detection through the vehicle detection model, and then performs vehicle speed analysis and traffic status perception based on the obtained vehicle target detection results, which can effectively reduce the cost of traffic status perception and improve the practicality of traffic status perception.

[0082] Reference Figure 1 In an embodiment of the present application, a method for training a vehicle detection model includes:

[0083] Step 110: acquiring a full-color image in a single-phase remote sensing image of the target vehicle, and preprocessing the full-color image to obtain a full-color training image;

[0084] In the embodiment of the present application, a single-phase remote sensing image of the target vehicle for training can be obtained. The single-phase remote sensing image can be acquired by a high spatial resolution satellite, or can be obtained from a remote sensing image dataset that has been publicly available on the Internet (such as a WorldView-3 remote sensing image dataset, a ZY-3 remote sensing image dataset). In addition, the acquired single-phase remote sensing image includes a full-color image and a multispectral image. Step 110 can be to cut the full-color image according to a preset pixel size (such as a 512×512 pixel size), and then mark the rotation box of the vehicle target in each cut full-color image block, so as to obtain a full-color training image.

[0085] Step 120, extracting image features from the full-color training image to obtain full-color image features;

[0086] In the embodiment of the present application, image feature extraction is used to extract image features in the full-color training image and generate a corresponding feature map. Specifically, step 120 can be to input the full-color training image into a pre-trained deep convolutional neural network, such as a feature pyramid network ResNet50-FPN, to capture different vehicle target features in the full-color training image through multi-layer convolution and pooling operations, and to determine the generated high-level feature map as the full-color image feature.

[0087] Step 130: extract candidate regions from the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different bounding box scale, aspect ratio, or rotation angle.

[0088] In an embodiment of the present application, step 130 may be to extract a target rotation anchor frame related to the target vehicle in a feature map of the full-color image feature, each target rotation anchor frame corresponding to at least one different bounding box scale, aspect ratio, or rotation angle, and determine all obtained target rotation anchor frames as a target rotation anchor frame set.

[0089] In some embodiments, the step 130 of extracting candidate regions from the panchromatic image features to obtain a target rotation anchor frame set corresponding to the panchromatic image features includes:

[0090] A1. Performing a rotation anchor frame generation process on the panchromatic image feature to obtain a first intermediate anchor frame set, wherein the first intermediate anchor frame set includes a plurality of intermediate rotation anchor frames corresponding to the panchromatic image feature, and each of the intermediate rotation anchor frames corresponds to a region scale, aspect ratio or rotation angle that is different from each other;

[0091] A2. Perform anchor frame feature mapping on the first intermediate anchor frame set to obtain a second intermediate anchor frame set;

[0092] In the embodiment of the present application, step A1 may be to generate a series of intermediate rotation anchor frames (anchors) with different bounding box scales, aspect ratios and rotation angles for each feature position in the feature map of the full-color image feature. Specifically, for a certain feature position in the feature map of the full-color image feature, step A1 can generate several intermediate rotation anchor frames based on the feature position, wherein the frame scale of the intermediate rotation anchor frame can be any one of 32 pixels, 64 pixels, 128 pixels, 256 pixels, 512 pixels, etc., the aspect ratio can be any one of 1:1, 1:2, 2:1, etc., and the rotation angle can be any one of 0°, 30°, 60°, 90°, 120°, 150°, etc. By combining different frame scales, aspect ratios and rotation angles, several intermediate rotation anchor frames are obtained, and each intermediate rotation anchor frame has at least one different frame scale, aspect ratio or rotation angle. The examples of this application are only for illustration and do not limit the frame scale, aspect ratio and rotation angle. The specific rotation angle can also be any one of 15°, 45°, etc., and the frame scale and aspect ratio are the same. In addition, for the remaining feature positions in the feature map of the full-color image feature, it is similar to the above content and can be simply deduced by analogy, and this application will not repeat them here.

[0093] It can be understood that the anchor frame feature map in step A2 is used to improve the feature expression ability of each intermediate rotated anchor frame. Specifically, each intermediate rotated anchor frame in the first intermediate anchor frame set can be input into a 3×3 convolution layer, and then the features of the intermediate rotated anchor frame are transformed through a nonlinear activation function to obtain the second intermediate anchor frame set.

[0094] A3. Perform bounding box classification and regression on the second intermediate anchor frame set to obtain the target rotation anchor frame set.

[0095] Furthermore, the step A3 of performing bounding box classification regression on the second intermediate anchor frame set to obtain the target rotation anchor frame set includes:

[0096] A31, performing anchor frame classification on the second intermediate anchor frame set to obtain a third intermediate anchor frame set, where the third intermediate anchor frame set is a set of intermediate rotation anchor frames corresponding to the target vehicle;

[0097] In this embodiment of the present application, step A31 is used to determine whether the intermediate rotated anchor frame in the second intermediate anchor frame set is a foreground candidate area containing the target vehicle or a background candidate area not containing the target vehicle, and then all the intermediate rotated anchor frames containing the target vehicle are determined as the third intermediate anchor frame set.

[0098] Exemplarily, the anchor frame classification in step A31 can be to input the second intermediate anchor frame set into a classification layer for binary classification. Specifically, a 1×1 convolution operation can be used to process all intermediate rotated anchor frames in the second intermediate anchor frame set, and output the foreground candidate area probability and the background candidate area probability of each intermediate rotated anchor frame; then, the output is mapped to a probability value through the Softmax function, and filtered based on a preset probability threshold to filter out the intermediate rotated anchor frames that may contain the target vehicle, thereby obtaining a third intermediate anchor frame set.

[0099] A32. Perform bounding box regression offset update on the third intermediate anchor frame set to obtain the target rotation anchor frame set.

[0100] Furthermore, the step A32 of performing bounding box regression offset update on the third intermediate anchor frame set to obtain the target rotation anchor frame set includes:

[0101] A321, performing border offset regression prediction on the third intermediate anchor frame set to obtain border offset prediction data;

[0102] A322. Perform a bounding box offset update on the third intermediate anchor frame set according to the bounding box offset prediction data to obtain the target rotation anchor frame set.

[0103] In an embodiment of the present application, step A32 is used to generate a target rotation anchor frame with more accurate positioning based on the intermediate rotation anchor frame. Specifically, step A321 can be to perform regression prediction on all intermediate rotation anchor frames in the third intermediate anchor frame set through a 1×1 convolution operation, and output five offset parameters, which are the horizontal and vertical offsets (Δx, Δy) of the center coordinates of the intermediate rotation anchor frame, the width and height offsets (Δw, Δh) and the rotation angle offset Δθ, and the obtained five offset parameters are determined as the frame offset prediction data.

[0104] It can be understood that step A322 can be based on the obtained bounding box offset prediction data, respectively adjust the corresponding parameters of the intermediate rotation anchor frame to generate a precisely positioned target rotation anchor frame, and then determine all the generated target rotation anchor frames as a target rotation anchor frame set.

[0105] Step 140: performing rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map;

[0106] In an embodiment of the present application, rotation area pooling is used to extract feature blocks containing a target vehicle in each target rotation anchor frame of a target rotation anchor frame set, and the feature blocks containing the target vehicle are pooled and mapped to a feature map of a fixed size to obtain an anchor frame feature map.

[0107] In some embodiments, the step 140 of performing rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map includes:

[0108] B1. Performing a rotation coordinate transformation on the target rotation anchor frame set to obtain a fourth rotation anchor frame set, wherein each target rotation anchor frame in the fourth rotation anchor frame set has the same rotation angle;

[0109] B2. Performing regional scale cropping on the fourth rotated anchor frame set to obtain a fifth rotated anchor frame set;

[0110] B3. Perform feature map pooling mapping on the fifth rotated anchor frame set to obtain the anchor frame feature map.

[0111] In an embodiment of the present application, step B1 may be inputting the target rotation anchor frame set into a rotation coordinate transformation layer (Rotation Coordinate Transformation Layer), and aligning the rotation angle of each target rotation anchor frame in the target rotation anchor frame set to a unified direction through the rotation coordinate transformation layer. Specifically, it may be based on the rotation angle of each target rotation anchor frame, calculating the corresponding affine transformation matrix, and then unifying the rotation axis of each target rotation anchor frame based on the calculated affine transformation matrix, so as to obtain a fourth rotation anchor frame set.

[0112] It can be understood that step B2 can be to input the fourth rotated anchor frame set into the region cropping layer (RoICropping Layer), and the corresponding feature area is cropped out based on the border scale (such as border width and border height) of the target rotated anchor frame through the region cropping layer, so as to extract the feature block containing the target vehicle information from the feature map to obtain the fifth rotated anchor frame set.

[0113] It should be noted that step B3 can be to divide the feature area at the cropping location into fixed grid units (such as 7×7 or 14×14), and perform maximum pooling or average pooling operations in each grid unit to map target rotation anchor frames of different sizes to feature maps of fixed sizes to obtain anchor frame feature maps.

[0114] Step 150: Update the parameters of the initialized vehicle detection model according to the anchor frame feature map to obtain a trained vehicle detection model.

[0115] In the embodiment of the present application, step 150 may be to identify the target vehicle detection result corresponding to the anchor frame feature map through the target classifier, and then determine a target loss value based on the target candidate anchor frame and predicted category in the target vehicle detection result, and the real candidate frame and real category in the full-color training image, and then update the parameters of the model through the back propagation algorithm based on the target loss value, and iterate several rounds to obtain a trained vehicle detection model. The specific number of iterations can be preset, or the training is considered to be completed when the test set meets the accuracy requirements.

[0116] Specifically, an embodiment of the present application may first input the anchor frame feature map into a target classifier, which includes a fully connected layer, a target classification layer and a bounding box regression layer. Specifically, the fully connected layer may be used to perform further feature extraction and information fusion on the anchor frame feature map; then, the target classification layer may be used to perform multi-category classification on the features output by the fully connected layer to obtain the category of each target candidate anchor frame; then, the bounding box regression layer may be used to perform regression offset prediction on the target candidate anchor frame to obtain the target vehicle detection result. The regression offset prediction is similar to the bounding box regression offset update in the aforementioned step A32, and can be simply deduced.

[0117] Reference Figure 2 , a traffic state perception method of a vehicle detection model proposed in an embodiment of the present application includes:

[0118] Step 210: Acquire traffic network data and target single-temporal remote sensing images, wherein the target single-temporal remote sensing images include target panchromatic images and target multispectral images;

[0119] In an embodiment of the present application, the traffic network data may be data related to the road on which the vehicle travels, which may specifically include the road center, number of lanes, road type, speed limit information, etc.; the target single-phase remote sensing image may be a remote sensing image currently corresponding to the location area of ​​the traffic network data, which may include a target panchromatic image and a target multispectral image.

[0120] Step 220: input the target full-color image into the trained vehicle detection model to perform vehicle target detection and obtain a vehicle target detection result;

[0121] In an embodiment of the present application, after obtaining a target full-color image of a moving vehicle, the target full-color image can be input into a trained vehicle detection model for vehicle target detection, thereby obtaining a vehicle target detection result output by the vehicle detection model, which records the candidate bounding boxes and categories of several moving vehicles in the target full-color image.

[0122] Step 230: According to the vehicle target detection result, the target multispectral image is subjected to vehicle speed analysis to obtain vehicle speed data;

[0123] In an embodiment of the present application, based on the candidate bounding boxes in the vehicle target detection results, candidate areas about the moving vehicle in different spectral bands of the target multispectral image can be extracted, and the moving speed of the moving vehicle (i.e., vehicle speed data) can be determined based on the candidate areas in different spectral bands.

[0124] In some embodiments, the step 230, performing vehicle speed analysis processing on the target multispectral image according to the vehicle target detection result to obtain vehicle speed data, includes:

[0125] C1. acquiring a first band spectral image and a second band spectral image according to the target multispectral image;

[0126] C2. Extracting a first candidate region from the first band spectral image according to the vehicle target detection result to obtain a first band candidate region, and extracting a second candidate region from the second band spectral image according to the vehicle target detection result to obtain a second band candidate region;

[0127] In an embodiment of the present application, step C1 may be to obtain images of two different spectral bands of the target multispectral image, and record them as a first band spectral image and a second band spectral image respectively. Specifically, for the target single-phase remote sensing image collected by the WorldView-3 satellite, its first band spectral image may be the imaging result of the multispectral sensor in the blue light band (0.450 μm-0.510 μm), and the second band spectral image may be the imaging result of the multispectral sensor in the green light band (0.510 μm-0.580 μm). The examples of the present application are for illustration only, and the first band spectral image and the second band spectral image may also be images of other spectral bands, which does not limit the present application.

[0128] It can be understood that, for a certain moving vehicle, the first candidate area extraction in step C2 can be specifically based on the candidate bounding box of the moving vehicle in the vehicle target detection result, and the candidate area corresponding to the moving vehicle in the first band spectral image is extracted. The specific candidate area size can be determined by the vehicle size and possible displacement range of the moving vehicle, so as to obtain the first band candidate area. The content of the second band candidate area is similar to that of the aforementioned first band candidate area, and can be simply deduced by analogy.

[0129] C3. Performing regional displacement matching on the second band candidate area according to the first band candidate area to obtain the vehicle speed data.

[0130] Furthermore, the step C3, performing regional displacement matching on the second band candidate area according to the first band candidate area to obtain the vehicle speed data, includes:

[0131] C31, obtaining the spectral imaging time difference of the target multispectral image;

[0132] C32. Perform template matching on the second band candidate area according to the first band candidate area to obtain first vehicle pixel coordinate data and second vehicle pixel coordinate data, wherein the first vehicle pixel coordinate data is pixel coordinate data of the moving vehicle in the first band candidate area, and the second vehicle pixel coordinate data is pixel coordinate data of the moving vehicle in the second band candidate area;

[0133] C33. Calculate the vehicle displacement of the second vehicle pixel coordinate data according to the first vehicle pixel coordinate data to obtain vehicle displacement data;

[0134] C34. Calculate the vehicle speed of the vehicle displacement data according to the spectral imaging time difference to obtain the vehicle speed data.

[0135] In an embodiment of the present application, the spectral imaging time difference of the target multispectral image can be firstly obtained, specifically, the spectral imaging time difference between the first band spectral image and the second band spectral image can be obtained, and the spectral imaging time difference can be obtained based on the selected satellite parameters.

[0136] It is understandable that the embodiment of the present application can extract and match the features of the first band candidate area and the second band candidate area based on the template matching algorithm, so as to obtain the first vehicle pixel coordinate data and the second vehicle pixel coordinate data. Specifically, step C32 can be that the first band candidate area is used as a template image, the second band candidate area is used as the image to be matched, and the normalized cross-correlation coefficient (NCC) is used for template matching, and then the matching position with the largest NCC value is determined as the final second vehicle pixel coordinate data, and the pixel coordinate data corresponding to the first band candidate area is directly determined as the final first vehicle pixel coordinate data, and the first vehicle pixel coordinate data and the second vehicle pixel coordinate data correspond to the same moving vehicle.

[0137] It should be noted that step C33 can be based on the spatial resolution corresponding to the target multispectral image, converting the pixel displacement of the moving vehicle in the first band spectral image and the second spectral image into the actual displacement of the vehicle on the ground (i.e., vehicle displacement data), wherein the spatial resolution corresponding to the target multispectral image can be obtained from the parameters of the high spatial resolution satellite that collects the target multispectral image.

[0138] Specifically, the equivalent expression of the vehicle displacement data in the embodiment of the present application can be:

[0139]

[0140] Among them, d i is the vehicle displacement data; r is the spatial resolution corresponding to the target multispectral image; Δx i Δy is the pixel coordinate difference between the first vehicle pixel coordinate data and the second vehicle pixel coordinate data on the horizontal axis; i is the pixel coordinate difference between the first vehicle pixel coordinate data and the second vehicle pixel coordinate data on the vertical coordinate.

[0141] It is worth mentioning that step C34 can determine the vehicle speed data of the moving vehicle based on the spectral imaging time difference between the first band spectral image and the second band spectral image and the vehicle displacement data. Specifically, it can be calculated based on the ratio of the vehicle displacement data to the spectral imaging time difference.

[0142] It should be added that due to the tiny imaging time difference of single-phase remote sensing images, high-speed moving targets (such as vehicles) will have position offsets (i.e., ghosting) in different spectral bands of multi-spectral images. In order to avoid this ghosting phenomenon, traditional traffic status perception methods based on remote sensing images usually realize vehicle detection based on multi-phase remote sensing images, and then realize traffic status perception based on the detected multi-phase vehicle data. The embodiment of the present application realizes the transient speed analysis of moving vehicles based on the position offset caused by the spectral imaging time difference, which can reduce the cost of traffic status perception and improve the practicality of traffic status perception.

[0143] Step 240: Perform traffic state perception on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain a traffic state perception result.

[0144] In an embodiment of the present application, the traffic state of the traffic network can be determined based on the vehicle position of each moving vehicle and the vehicle speed data of each moving vehicle in the vehicle target detection results, thereby obtaining a traffic state perception result.

[0145] In some embodiments, the step 240, performing traffic state perception on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain a traffic state perception result, includes:

[0146] D1. Performing vehicle space matching on the traffic network data according to the vehicle target detection result to obtain vehicle space matching data;

[0147] D2. Performing a traffic state evaluation on the vehicle space matching data according to the vehicle speed data to obtain the traffic state perception result.

[0148] In an embodiment of the present application, step D1 can first generate a surface layer for each lane based on traffic network information such as the road centerline, number of lanes, road type, speed limit information, etc. in the traffic network data, thereby forming a lane surface layer; then, based on GIS spatial analysis technology, the vehicle position of each moving vehicle in the vehicle target detection result is superimposed on the corresponding spatial position of the lane surface layer, so as to determine the lane to which each moving vehicle belongs in the traffic network and obtain vehicle spatial matching data.

[0149] It can be understood that, for a lane in the vehicle space matching data, step D2 can first calculate the average speed on the lane based on the vehicle speed data corresponding to each vehicle traveling on the lane. The equivalent expression of the average speed on the lane can be:

[0150]

[0151] in, is the average speed of vehicles traveling on the jth lane of the vehicle space matching data; n j is the total number of vehicles traveling on the jth lane of the vehicle space matching data; v i The vehicle space matching data is the vehicle speed data corresponding to the i-th vehicle on the j-th lane.

[0152] It is worth mentioning that the average speed of the remaining lanes is similar. After obtaining the average speed of each lane of the vehicle space matching data, the congestion level of each lane of the vehicle space matching data can be determined based on the speed limit of each lane and the preset congestion threshold, thereby obtaining the traffic state perception result. Specifically, for a certain lane, the congestion coefficient of the lane can be determined based on the average speed of the lane and the corresponding speed limit. The equivalent expression of the congestion coefficient can be:

[0153]

[0154] Among them, C j is the congestion coefficient of the jth lane; v limit,j is the speed limit of the jth lane.

[0155] It should be added that after determining the congestion coefficient of each lane, the size relationship between each lane coefficient and the preset congestion threshold can be compared respectively, so as to obtain the congestion level of each lane, thereby obtaining the traffic state perception result. The congestion threshold in the embodiment of the present application may include 0.25, 0.5, 0.75, etc., and the specific value of the congestion threshold can be set according to the actual situation. Specifically, for a certain lane, if its congestion coefficient is less than 0.25, it can be considered that the congestion level of the lane is unblocked; or, if its congestion coefficient is greater than or equal to 0.25 and less than 0.5, it can be considered that the congestion level of the lane is basically unblocked; or, if its congestion coefficient is greater than or equal to 0.25 and less than 0.75, it can be considered that the congestion level of the lane is congested; or, if its congestion coefficient is greater than or equal to 0.75, it can be considered that the congestion level of the lane is severely congested. The same is true for the remaining lanes. By integrating the congestion level corresponding to each lane, the traffic state perception result can be obtained.

[0156] A vehicle detection model training system proposed according to an embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0157] Reference Figure 3 , a vehicle detection model training system proposed in an embodiment of the present application includes:

[0158] The first processing unit 101 is used to obtain a full-color image in a single-phase remote sensing image of a target vehicle, and pre-process the full-color image to obtain a full-color training image;

[0159] The second processing unit 102 is used to extract image features from the full-color training image to obtain full-color image features;

[0160] The third processing unit 103 is used to extract a candidate region from the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different frame scale, aspect ratio or rotation angle.

[0161] The fourth processing unit 104 is used to perform rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map;

[0162] The fifth processing unit 105 is used to update the parameters of the initialized vehicle detection model according to the anchor frame feature map to obtain a trained vehicle detection model.

[0163] Reference Figure 4 , the embodiment of the present application further provides an electronic device, including:

[0164] at least one processor 201;

[0165] At least one memory 202, used to store at least one program;

[0166] When the at least one program is executed by the at least one processor 201, the at least one processor 201 implements the above method embodiment.

[0167] Similarly, it can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0168] The embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the processor 201 is stored. The program executable by the processor 201 is used to implement the above-mentioned method embodiment when executed by the processor 201.

[0169] Similarly, the contents of the above method embodiments are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0170] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0171] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0172] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method of the embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0173] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0174] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0175] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0176] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0177] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0178] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for training a vehicle detection model, characterized in that: include: Acquire a full-color image in a single-phase remote sensing image of a target vehicle, and preprocess the full-color image to obtain a full-color training image; Performing image feature extraction on the full-color training image to obtain full-color image features; Extracting candidate regions from the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different bounding box scale, aspect ratio, or rotation angle; Performing rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map; According to the anchor frame feature map, the parameters of the initialized vehicle detection model are updated to obtain a trained vehicle detection model.

2. The method according to claim 1, characterized in that The step of extracting a candidate region from the panchromatic image feature to obtain a target rotation anchor frame set corresponding to the panchromatic image feature includes: Performing a rotation anchor frame generation process on the panchromatic image feature to obtain a first intermediate anchor frame set, wherein the first intermediate anchor frame set includes a plurality of intermediate rotation anchor frames corresponding to the panchromatic image feature, and each of the intermediate rotation anchor frames corresponds to a region scale, an aspect ratio, or a rotation angle that is different from each other; Performing anchor frame feature mapping on the first intermediate anchor frame set to obtain a second intermediate anchor frame set; Perform bounding box classification regression on the second intermediate anchor frame set to obtain the target rotation anchor frame set.

3. The method according to claim 2, characterized in that The performing bounding box classification regression on the second intermediate anchor frame set to obtain the target rotation anchor frame set includes: Performing anchor frame classification on the second intermediate anchor frame set to obtain a third intermediate anchor frame set, where the third intermediate anchor frame set is a set of intermediate rotation anchor frames corresponding to the target vehicle; The third intermediate anchor frame set is updated by bounding box regression offset to obtain the target rotation anchor frame set.

4. The method according to claim 3, characterized in that The performing bounding box regression offset update on the third intermediate anchor frame set to obtain the target rotation anchor frame set includes: Performing border offset regression prediction on the third intermediate anchor frame set to obtain border offset prediction data; The third intermediate anchor frame set is updated with a border offset according to the border offset prediction data to obtain the target rotation anchor frame set.

5. The method according to claim 1, characterized in that: The performing rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map includes: Performing a rotation coordinate transformation on the target rotation anchor frame set to obtain a fourth rotation anchor frame set, wherein each target rotation anchor frame in the fourth rotation anchor frame set has the same rotation angle; Performing regional scale cropping on the fourth rotated anchor frame set to obtain a fifth rotated anchor frame set; Perform feature map pooling mapping on the fifth rotated anchor frame set to obtain the anchor frame feature map.

6. A traffic state perception method of a vehicle detection model, characterized in that: include: Acquire traffic network data and target single-temporal remote sensing images, wherein the target single-temporal remote sensing images include target panchromatic images and target multispectral images; Inputting the target full-color image into the trained vehicle detection model according to any one of claims 1 to 5 to perform vehicle target detection, and obtaining a vehicle target detection result; According to the vehicle target detection result, the vehicle speed analysis processing is performed on the target multispectral image to obtain vehicle speed data; The traffic state perception is performed on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain a traffic state perception result.

7. The method according to claim 6, characterized in that The step of performing vehicle speed analysis on the target multispectral image according to the vehicle target detection result to obtain vehicle speed data includes: According to the target multispectral image, acquiring a first band spectral image and a second band spectral image; According to the vehicle target detection result, extracting a first candidate region from the first band spectral image to obtain a first band candidate region, and according to the vehicle target detection result, extracting a second candidate region from the second band spectral image to obtain a second band candidate region; According to the first band candidate area, regional displacement matching is performed on the second band candidate area to obtain the vehicle speed data.

8. The method according to claim 7, characterized in that The performing regional displacement matching on the second band candidate area according to the first band candidate area to obtain the vehicle speed data includes: Acquiring a spectral imaging time difference of the target multispectral image; According to the first band candidate area, template matching is performed on the second band candidate area to obtain first vehicle pixel coordinate data and second vehicle pixel coordinate data, wherein the first vehicle pixel coordinate data is pixel coordinate data of the moving vehicle in the first band candidate area, and the second vehicle pixel coordinate data is pixel coordinate data of the moving vehicle in the second band candidate area; According to the first vehicle pixel coordinate data, performing vehicle displacement calculation on the second vehicle pixel coordinate data to obtain vehicle displacement data; The vehicle speed data is calculated based on the spectral imaging time difference to obtain the vehicle speed data.

9. The method according to claim 6, characterized in that The performing traffic state perception on the traffic network data according to the vehicle speed data and the vehicle target detection result to obtain a traffic state perception result includes: According to the vehicle target detection result, performing vehicle space matching on the traffic network data to obtain vehicle space matching data; The traffic state is evaluated on the vehicle space matching data according to the vehicle speed data to obtain the traffic state perception result.

10. A vehicle detection model training system, characterized in that: include: A first processing unit is used to obtain a full-color image in a single-phase remote sensing image of a target vehicle, and pre-process the full-color image to obtain a full-color training image; A second processing unit is used to extract image features from the full-color training image to obtain full-color image features; A third processing unit is used to extract a candidate region from the full-color image features to obtain a target rotation anchor frame set, wherein the target rotation anchor frame set includes a plurality of target rotation anchor frames corresponding to the target vehicle, and each of the target rotation anchor frames has at least one different bounding box scale, aspect ratio or rotation angle; a fourth processing unit, configured to perform rotation region pooling on the target rotation anchor frame set to obtain an anchor frame feature map; The fifth processing unit is used to update the parameters of the initialized vehicle detection model according to the anchor frame feature map to obtain a trained vehicle detection model.

Images