Vehicle detection model training method, device, vehicle detection method and electronic equipment

Abstract

The embodiments of the present application provide a vehicle detection model training method, device, vehicle detection method and electronic equipment, which relate to the field of target detection. The method includes obtaining a training data set including multiple training samples, each training sample including a sample image, a vehicle label representing whether the sample image contains a vehicle, and an information label representing the vehicle position information in the sample image. The position information of at least two sub-images corresponding to each sample image is obtained, and the vehicle label of each sub-image is determined. By iteratively training a neural network model including a feature extraction network and a target recognition network cascaded in sequence based on each training sample and each sub-image corresponding to each sample image, until the training loss value of the neural network model meets the training end condition, the vehicle detection model with higher accuracy trained according to this method can perform more accurate vehicle detection on the vehicle to be detected.

Description

Technical Field

[0001] The present application relates to the field of target detection. Specifically, the present application relates to a vehicle detection model training method, device, vehicle detection method, electronic equipment and computer-readable storage medium. Background Art

[0002] With the increasing number of vehicle violations, rapidly detecting vehicles in traffic images or videos to better assist urban traffic management has become an important task. Following its widespread success in image classification tasks, deep learning has also been widely used in vehicle detection tasks.

[0003] In a large number of vehicle detection tasks, how to detect the vehicle to be detected more accurately has become a challenging task. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle detection model training method, device, vehicle detection method, electronic device and computer-readable storage medium. The vehicle detection model with higher accuracy trained according to this method can perform more accurate vehicle detection on the vehicle to be detected.

[0005] According to one aspect of an embodiment of the present application, a vehicle detection method is provided, the method comprising:

[0006] Acquire a training data set including a plurality of training samples, each training sample including a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating position information of the vehicle in the sample image;

[0007] Obtaining position information of at least two sub-images corresponding to each sample image in the sample image, and determining a vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image;

[0008] Based on each training sample and each sub-image corresponding to each sample image, a neural network model including a cascaded feature extraction network and a target recognition network is iteratively trained until the training loss value of the neural network model meets the training end condition, thereby obtaining a vehicle detection model;

[0009] The training end condition includes at least one of the following:

[0010] The loss function of the neural network model converges, the training loss value of the neural network model is less than the set value, and the number of training times of the neural network model reaches the set number.

[0011] According to another aspect of an embodiment of the present application, a vehicle detection method is provided, comprising:

[0012] Obtain the image to be detected;

[0013] Based on the trained vehicle detection model, whether the image to be detected includes a vehicle and the position information of the vehicle in the image to be detected are determined. The trained vehicle detection model is obtained by training according to the above method.

[0014] According to another aspect of an embodiment of the present application, a vehicle detection device is provided, the device comprising an acquisition module, a sub-image determination module, and a training module.

[0015] An acquisition module is used to acquire a training data set including a plurality of training samples, each training sample including a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating the location information of the vehicle in the sample image;

[0016] a sub-image determination module, configured to obtain position information of at least two sub-images corresponding to each sample image in the sample image, and determine a vehicle label of each sub-image corresponding to each sample image based on an information label of each sample image;

[0017] a training module for iteratively training a neural network model comprising a cascaded feature extraction network and a target recognition network based on each training sample and each sub-image corresponding to each sample image, until a training loss value of the neural network model satisfies a training termination condition, thereby obtaining a vehicle detection model;

[0018] The training end condition includes at least one of the following:

[0019] The loss function of the neural network model converges, the training loss value of the neural network model is less than the set value, and the number of training times of the neural network model reaches the set number.

[0020] According to another aspect of an embodiment of the present application, a vehicle detection device is provided, comprising an acquisition module and a detection module.

[0021] An acquisition module, used for acquiring an image to be detected;

[0022] The detection module is used to determine whether the image to be detected includes a vehicle and the position information of the vehicle in the image to be detected based on a trained vehicle detection model, wherein the trained vehicle detection model is obtained by training according to the above method.

[0023] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0024] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0025] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0026] The present application relates to a vehicle detection method, device, electronic device, and computer-readable storage medium. The method obtains a training data set comprising a plurality of training samples, each training sample comprising a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating the vehicle position information in the sample image. The method obtains the position information of at least two sub-images corresponding to each sample image, and determines the vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image. During the training process, a neural network model comprising a sequentially cascaded feature extraction network and a target recognition network is iteratively trained based on each training sample and each sub-image corresponding to each sample image until the training loss value of the neural network model meets the training end condition. The vehicle detection model with higher accuracy obtained by training according to the method can perform more accurate vehicle detection on the vehicle to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0028] Figure 1 A flow chart of a vehicle detection model training method provided in an embodiment of the present application is shown;

[0029] Figure 2 A schematic diagram of the training process of the neural network model in an embodiment of the present application is shown;

[0030] Figure 3 A schematic diagram showing location information of the area where a vehicle is located in a sample image in an embodiment of the present application;

[0031] Figure 4a and Figure 4b shows a schematic diagram of a sample image before and after embedding processing;

[0032] Figure 5 A schematic diagram of a vehicle detection model training method according to an embodiment of the present application is shown;

[0033] Figure 6 A flow chart of a vehicle detection method provided in an embodiment of the present application is shown;

[0034] Figure 7 A schematic diagram of a vehicle detection model training device provided in an embodiment of the present application is shown;

[0035] Figure 8 A schematic diagram of a vehicle detection device provided in an embodiment of the present application is shown;

[0036] Figure 9 A schematic diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0038] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates implementation as "A," or implementation as "A," or implementation as "A and B."

[0039] With the increasing number of vehicle violations, rapidly detecting vehicles in traffic images or videos has become a critical task in urban traffic management. Following its widespread success in image classification tasks, deep learning has also been applied to vehicle detection tasks. These methods are categorized into two main groups: region-based methods and regression-based methods. To address this, this application proposes a vehicle detection method, apparatus, electronic device, and computer-readable storage medium. This solution can quickly and accurately train a vehicle detection model to better perform vehicle detection and meet practical needs.

[0040] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0041] An embodiment of the present application provides a vehicle detection model training method. Figure 1 FIG. 1 is a flow chart of a vehicle detection model training method provided by an embodiment of the present application. Figure 1 As shown, the method includes steps S110 to S130.

[0042] Step S110: Acquire a training data set including a plurality of training samples, each training sample including a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating vehicle position information in the sample image.

[0043] The multiple training samples in the training data set may include sample images containing vehicles and sample images not containing vehicles. The number of sample images containing vehicles (i.e., positive samples) and sample images not containing vehicles (i.e., negative samples) in the multiple training samples can be determined based on actual conditions, or the proportion of sample images containing vehicles and sample images not containing vehicles in the multiple training samples can be determined.

[0044] The embodiments of the present application do not limit the representation of the vehicle label, which can be in the form of text, numbers, letters, etc. For example, the vehicle label can be represented as "yes" or "no", where "yes" indicates that the sample image contains a vehicle, and "no" indicates that the sample image does not contain a vehicle. The vehicle label can also be represented as "1" or "0", where "1" indicates that the sample image contains a vehicle, and "0" indicates that the sample image does not contain a vehicle.

[0045] It is understood that if the vehicle label indicates that the sample image does not contain a vehicle, the information label can be set to 0, or the sample image can be annotated with no information label. That is, if the vehicle label indicates that the sample image does not contain a vehicle, the annotated label can only include the vehicle label. If the vehicle label indicates that the sample image does contain a vehicle, the annotated label also includes an information label, which contains the location information of the area where the vehicle is located in the sample image.

[0046] Optionally, any point in each sample image can be used as the coordinate origin to establish a coordinate system to determine the vehicle position information of the vehicle in each sample image, wherein the vehicle position information of a vehicle in a sample image is, that is, the position information of the area where the vehicle is located in the sample image. This application does not limit the representation method of the position information of the area where the vehicle is located in the sample image. The upper left corner of a sample image can be used as the coordinate origin, and based on the coordinate information (x1, y1) of the upper left corner and the coordinate information (x2, y2) of the lower right corner of the area where the vehicle is located in the sample image, the representation method of the position information of the area where the vehicle is located in the sample image is determined to be (x1, y1, x2, y2). The coordinate information (x1, y1) of the center position of the area where the vehicle is located in the sample image can also be used. center ,y center), the width w and height h of the area where the vehicle is located in the sample image, and the representation of the position information of the area where the vehicle is located in the sample image is determined as (x center ,y center , w, h). Figure 3 A schematic diagram showing the location information of the area where the vehicle is located in the sample image in the embodiment of the present application is shown. Figure 3 As shown, the larger rectangle represents the sample image, and the smaller rectangle represents the area where the vehicle is located in the sample image. Taking the upper left corner of the sample image as the coordinate origin, the position information of the area where the vehicle is located in the sample image can be represented as (16, 16, 58, 58) based on the coordinate information (16, 16) of the upper left corner and the coordinate information (58, 58) of the lower right corner. Of course, the coordinate information (37, 37) of the center position of the area where the vehicle is located in the sample image and the width 144 and height 72 of the area where the vehicle is located in the sample image can also be used to determine that the position information of the area where the vehicle is located in the sample image is represented as (37, 37, 144, 72). This application does not limit the representation method of the position information of the area where the vehicle is located in the sample image.

[0047] Step S120: obtaining position information of at least two sub-images corresponding to each sample image in the sample image, and determining a vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image.

[0048] Optionally, a selective search algorithm may be used to obtain position information of at least two sub-images (hereinafter, "sub-images" are also referred to as "candidate frames") corresponding to each sample image within the sample image. Of course, the position information of at least two sub-images corresponding to each sample image within the sample image may also be obtained using other methods, and this application does not impose any restrictions thereto.

[0049] This application does not impose any restrictions on the size of each candidate frame corresponding to each sample image, nor does it impose any restrictions on the number of candidate frames included in each sample image. The sizes of the candidate frames corresponding to each sample image can be the same or different. This application does not impose any restrictions on this. The number of candidate frames included in each sample image can be determined based on actual conditions. For example, the number of candidate frames corresponding to each sample image can be set to 2000.

[0050] The representation of the position information of each candidate frame can be the same as the representation of the position information of the area where the vehicle is located in the sample image above. The position information of a candidate frame can be represented as (n, X center , Y center, W, H), can also be expressed as (n, X1, Y1, X2, Y2). Among them, n represents the sequence number of the sample image corresponding to the candidate box in the training sample set, (X center , Y center ) represents the coordinates of the center position of the image region of the sample image corresponding to the candidate frame, W represents the width of the image region of the sample image corresponding to the candidate frame, and H represents the height of the image region of the sample image corresponding to the candidate frame. (X1, Y1) represents the coordinates of the upper left corner of the image region of the sample image corresponding to the candidate frame, and (X2, Y2) represents the coordinates of the lower right corner of the image region of the sample image corresponding to the candidate frame.

[0051] Step S130: Based on each training sample and each sub-image corresponding to each sample image, iteratively train a neural network model including a cascaded feature extraction network and a target recognition network until the training loss value of the neural network model meets the training end condition, thereby obtaining a vehicle detection model;

[0052] The training end condition includes at least one of the following:

[0053] The loss function of the neural network model converges, the training loss value of the neural network model is less than the set value, and the number of training times of the neural network model reaches the set number.

[0054] The value of the above setting value is negatively correlated with the accuracy of the trained vehicle detection model, that is, the smaller the value of the above setting value is, the higher the accuracy of the trained vehicle detection model is.

[0055] The value of the set number of times is positively correlated with the accuracy of the trained vehicle detection model. That is, the larger the value of the set number, the higher the accuracy of the trained vehicle detection model. It should be noted that as the number of training times increases, the computational effort required for the training process also increases. The specific value of the set number of times can be determined based on actual needs.

[0056] This method obtains a training dataset comprising multiple training samples, each of which includes a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating the vehicle's location information within the sample image. The method then obtains the location information of at least two sub-images corresponding to each sample image and, based on the information label of each sample image, determines the vehicle label for each sub-image corresponding to each sample image. During the training process, a neural network model comprising a cascaded feature extraction network and an object recognition network is iteratively trained based on each training sample and each sub-image corresponding to each sample image until the training loss of the neural network model meets a training termination condition. This allows the vehicle detection model trained using this method to more accurately detect vehicles to be detected.

[0057] Figure 2 The figure shows a schematic diagram of the training process of the neural network model in the embodiment of the present application. The loss function of the neural network model includes a first loss function and a second loss function. Figure 2 As shown, for each sample image, the training process includes steps S131 to S134.

[0058] Step S131: extract features from the sample image through a feature extraction network to obtain a first feature map corresponding to the sample image.

[0059] Step S132: determining a second feature map corresponding to each sub-image based on the first feature map and position information of each sub-image corresponding to the sample image in the sample image.

[0060] Step S133: For each sub-image, based on the second feature map corresponding to the sub-image and the position information corresponding to the sub-image, the target recognition network is used to obtain the recognition result of the sub-image. The recognition result includes the predicted classification result and the predicted vehicle position information. The predicted classification result represents whether the sub-image includes a vehicle.

[0061] Step S134: Determine a first training loss value using a first loss function based on the predicted classification results of each sub-image corresponding to the sample image and the vehicle label of the corresponding sub-image; determine a second training loss value using a second loss function based on the predicted vehicle position information of each sub-image corresponding to the sample image and the information label of the sample image; and obtain a training loss value of the neural network model based on the first training loss value and the second training loss value.

[0062] If the training loss value of the neural network model does not meet the training end condition, the model parameters of the neural network model are adjusted, and the adjusted model is continued to be trained based on each training sample and each sub-image corresponding to each sample image.

[0063] The vehicle detection method provided in the embodiments of the present application obtains the position information of at least two sub-images corresponding to each sample image and, based on the information label of each sample image, determines the vehicle label of each sub-image corresponding to each sample image. During training, a feature extraction network extracts features from the sample image to obtain a first feature map for the sample image. Based on the first feature map and the position information of each sub-image corresponding to the sample image within the sample image, a second feature map for each sub-image is determined. This allows each sub-image corresponding to the sample image to share the same first feature map, thus sharing computation and memory, reducing computational overhead and improving training efficiency.

[0064] Furthermore, for each sub-image, the object recognition network uses the corresponding second feature map and the sub-image's corresponding location information to obtain a recognition result for that sub-image. The recognition result includes a predicted classification result and predicted vehicle location information, which indicates whether the sub-image contains a vehicle. In other words, the object recognition network can simultaneously obtain the predicted classification result and predicted vehicle location information for each sub-image, improving training efficiency and speeding up training.

[0065] A first training loss value is determined based on the predicted classification results of each sub-image corresponding to the sample image and the vehicle label of the corresponding sub-image. A second training loss value is determined based on the predicted vehicle position information of each sub-image corresponding to the sample image and the information label of the sample image. A training loss value of the neural network model is obtained based on the first training loss value and the second training loss value. The neural network model is trained based on the training loss value until the training loss value of the neural network model meets the training end condition, thereby obtaining a vehicle detection model.

[0066] As can be seen from the above, the first feature map of each sample image in the training dataset can be shared, enabling shared computing and memory, reducing computational overhead and improving training efficiency. Furthermore, the object recognition network can simultaneously obtain the predicted classification results for each sub-image and the predicted vehicle position information, further improving training efficiency and speeding up training.

[0067] In the above method, the feature extraction network and the target recognition network can be cascaded through a fully connected network, that is, a fully connected network is added between the feature extraction network and the target recognition network.

[0068] This method uses SVD (singular value decomposition) to accelerate fully connected networks. Specifically, a fully connected network is split into two fully connected networks, the first without bias and the second with bias. This splitting of the fully connected network can improve training speed.

[0069] The specific composition of the feature extraction network can be determined according to actual conditions, as long as the feature extraction network can be used to perform feature extraction on the sample image to obtain the first feature map of the sample image. For example, the feature extraction network may include a convolution module and a pooling module, or may only include a convolution module, or may only include a pooling module, and this application does not impose any restrictions on this. In addition, there is no restriction on the number of convolution modules and / or the number of pooling modules included in the feature extraction network. This application does not impose any restrictions on the convolution kernel size of each convolution module and the pooling kernel size of each pooling module. The size of the convolution kernel corresponding to each convolution module may be the same or different. The size of the pooling kernel corresponding to each pooling module may be the same or different.

[0070] Without considering the depth information (i.e., the number of channels) of the sample image, when the sample image is convolved by the convolution module, the size of the first feature map of the sample image is the same as the size of the sample image. When the sample image is pooled by the pooling module, the size of the first feature map of the sample image is smaller than the size of the sample image.

[0071] When a sample image is pooled through a pooling module network, the size of the first feature map corresponding to the sample image depends on the size of the sample image and the size of the pooling kernel of the pooling module. For example, if the size of the sample image (in pixels) is 256*256, and the feature extraction network includes two pooling modules, each with a pooling kernel size of 2*2, then after the first pooling module, the size of the feature map corresponding to the sample image is 128*128, and after the second pooling module, the size of the first feature map of the sample image is 64*64.

[0072] Among them, when pooling the sample image based on the pooling module, the pooling method adopted can be maximum pooling or average pooling. This application does not impose any restrictions on this, and the specific pooling method can be determined according to actual needs.

[0073] When pooling a sample image based on a pooling module, if the ratio between the width of the sample image and the width of the pooling kernel is an integer, and the ratio between the height of the sample image and the height of the pooling kernel is an integer, then the sample image can be directly pooled based on the pooling module. Otherwise, the sample image can be first embedded so that the ratio between the width of the sample image after the embedding process and the width of the pooling kernel is an integer, and the ratio between the height of the sample image after the embedding process and the height of the pooling kernel is an integer, and then the sample image can be pooled based on the pooling module. The embedding process can include padding the pixel matrix corresponding to the sample image with zeros.

[0074] As an example, Figure 4a and Figure 4b Figure 2 shows a schematic diagram of the sample image before and after embedding. Figure 4a As shown in , the size of the sample image is 3*3, and the size of the pooling kernel is 2*2, so the sample image needs to be embedded. Figure 4b As shown, the size of the sample image after embedding processing is 4*4, and the sample image after embedding processing can be pooled based on the pooling module.

[0075] For each sample image, a first training loss value for each sub-image corresponding to the sample image can be determined using a first loss function based on the predicted classification results of each sub-image corresponding to the sample image and the vehicle label of each sub-image corresponding to the sample image, and a first training loss value for the sample image can be determined based on the first training loss values ​​of each sub-image corresponding to the sample image;

[0076] Determine, using a second loss function, a second training loss value for each sub-image corresponding to the sample image based on the predicted position information of each sub-image corresponding to the sample image and the information label of the sample image, and determine a second training loss value for the sample image based on the second training loss values ​​of each sub-image corresponding to the sample image;

[0077] Determining a total loss value of the sample image based on the first training loss value of the sample image and the second training loss value of the sample image;

[0078] When the total loss value of the sample image does not meet the training end condition, the model parameters of the neural network model are adjusted until the neural network model converges to obtain a vehicle detection model.

[0079] Among them, the specific function forms of the first loss function and the second loss function are not limited in the embodiments of this application, and can be selected according to actual needs. Optionally, the first loss function can be a target recognition loss function, which characterizes the accuracy of recognition. For example, the first loss function can include any one of the following: softmax (normalization) loss, centerloss (center loss), etc., and this application does not limit this. The second loss function can be a multi-classification prediction loss function, and the second loss function can include any one of the following: softmax, center loss, etc., and this application does not limit this. Among them, the first loss function and the second loss function can be the same loss function or different loss functions.

[0080] Specifically, for each sample image, after determining the first training loss value of each sub-image corresponding to the sample image, the first training loss value of the sample image can be determined based on the cross-entropy loss function. After determining the second training loss value of each sub-image corresponding to the sample image, the second training loss value of the sample image can be determined based on the mean absolute error loss function, that is, by taking the average of the second training loss values ​​of each candidate box.

[0081] In this method, the neural network model can be trained using mini-batch training. As an example, experiments have shown that when the number of training samples in each training sample set is 2, the training speed is faster and the vehicle detection model obtained after training is more accurate.

[0082] Optionally, the feature extraction network includes at least one feature extraction module and a pooling module cascaded with the at least one feature extraction module. For each sample image, feature extraction is performed on the sample image through the feature extraction network to obtain a first feature map corresponding to the sample image, including:

[0083] Performing feature extraction on the sample image by at least one feature extraction module to obtain a feature extraction result corresponding to the sample image;

[0084] Pooling the feature extraction results through a pooling module to obtain a first feature map corresponding to the sample image;

[0085] Determining, according to the first feature map and position information of each sub-image corresponding to the sample image in the sample image, a second feature map corresponding to each sub-image, including:

[0086] Determining, according to the size of the pooling kernel of the pooling network, a mapping relationship between each eigenvalue of the first feature map and the image area in the sample image;

[0087] For each sub-image, determining the image area corresponding to the sub-image in the sample image according to the position information corresponding to the sub-image;

[0088] For each sub-image, according to the mapping relationship and the first feature map, the eigenvalues ​​corresponding to the image area corresponding to the sub-image in the first feature map are determined, and the second feature map corresponding to the sub-image is obtained based on the determined eigenvalues ​​corresponding to the sub-images.

[0089] In the case of at least one feature extraction module and a pooling module cascaded with the at least one feature extraction module, the mapping relationship between each eigenvalue of the first feature map and the image area in the sample image can be determined according to the size of the pooling kernel of the pooling module, thereby determining the eigenvalues ​​corresponding to the image area corresponding to the sub-image in the first feature map based on the mapping relationship and the first feature map, and the second feature map corresponding to the sub-image can be accurately determined based on the determined eigenvalues ​​corresponding to each sub-image.

[0090] As described above, when the sample image is convolved by the convolution module, the size of the first feature map corresponding to the sample image is the same as the size of the sample image. When the sample image is pooled by the pooling module, the size of the first feature map corresponding to the sample image is smaller than the size of the sample image. Therefore, if the feature extraction network does not include a pooling module, the pixel values ​​included in the image area corresponding to each sub-image in the sample image should be the same as the feature values ​​included in the image area corresponding to each sub-image in the first feature map corresponding to the sample image, and the values ​​of each element in the position information corresponding to each sub-image in the sample image are the same as the values ​​of the corresponding elements in the position information corresponding to each sub-image in the first feature map corresponding to the sample image, that is, the mapping relationship between the first feature map and the sample image is 1:1. There is no need to scale the position information corresponding to each sub-image in the sample image, and each sub-image in the sample image can be directly mapped to the first feature map of the sample image.

[0091] If the feature extraction network includes a pooling module, the pixel values ​​included in the image area corresponding to each sub-image in the sample image should be greater than the feature values ​​included in the image area corresponding to each sub-image in the first feature map corresponding to the sample image, and the value of each element in the position information corresponding to each sub-image in the sample image should be s times the value of the corresponding element in the position information corresponding to each sub-image in the first feature map of the sample image, that is, the mapping relationship between the first feature map and the sample image is 1:s, where the value of s depends on the size of the pooling kernel. The position information corresponding to each sub-image in the sample image needs to be reduced before the mapping of each sub-image in the sample image to the first feature map of the sample image can be completed.

[0092] For example, if the position information corresponding to a sub-image in a sample image is (32,32,64,64), and the pooling kernel is 2*2, then the position information of the sub-image in the feature map of the sample image is (16,16,32,32). It is understandable that when determining the position information of each sub-image in the sample image, if the value of each element in the position information is a non-integer, it is necessary to quantize the value of each element in the position information to an integer. For example, the value of the non-integer element in the position information can be replaced with the integer closest to the value of the non-integer element.

[0093] Among them, the second feature map of a sub-image corresponding to the sample image, that is, after mapping each sub-image in the sample image to the first feature map of the sample image, the feature matrix composed of all eigenvalues ​​in the area of ​​the sub-image corresponding to the first feature map.

[0094] Optionally, after determining the second feature map corresponding to each sub-image, the method further includes:

[0095] For each sub-image, dividing the second feature map corresponding to the sub-image into a set number of third feature maps, performing feature extraction on each of the third feature maps corresponding to the sub-image, and obtaining a feature vector corresponding to the sub-image;

[0096] For each sub-image, based on the second feature map corresponding to the sub-image and the position information corresponding to the sub-image, a recognition result of the sub-image is obtained through the object recognition network, including:

[0097] According to the feature vector corresponding to the sub-image and the position information corresponding to the sub-image, the recognition result of the sub-image is obtained through the target recognition network.

[0098] In this implementation, for each sub-image, the second feature map corresponding to the sub-image is divided into a set number of third feature maps, and feature extraction is performed on each third feature map corresponding to the sub-image to obtain a feature vector corresponding to the sub-image, thereby obtaining a feature vector of the same size corresponding to each sub-image. In other words, regardless of whether the sizes of the sample images in the training dataset are the same or whether the sizes of the sub-images are the same, feature vectors of the same size corresponding to each sub-image can be obtained. This avoids the problem in related arts of scaling the sample images before training, which results in deformation during feature extraction of the sample images and errors in the subsequent recognition process. This improves training efficiency and training accuracy, and also improves the detection accuracy of the resulting vehicle detection model.

[0099] Optionally, for each sub-image, feature extraction is performed on each third feature map corresponding to the sub-image to obtain a feature vector corresponding to the sub-image, including:

[0100] For each third feature map, performing a pooling operation on each eigenvalue of the third feature map;

[0101] Based on the results of the pooling operation of all third feature maps corresponding to the sub-image, a feature vector corresponding to the sub-image is obtained.

[0102] Among them, when the pooling operation is performed on each eigenvalue of the third feature map, the pooling method adopted can be maximum pooling or average pooling. This application does not impose any restrictions on this, and the specific pooling method can be determined according to actual needs.

[0103] In this implementation, for each sub-image, the second feature map corresponding to the sub-image is divided into a set number of third feature maps, and the eigenvalues ​​of each third feature map are pooled to obtain the feature vector corresponding to the sub-image, thereby improving the training efficiency and training accuracy, and also improving the detection accuracy of the obtained vehicle detection model.

[0104] For a sub-image, the second feature map corresponding to the sub-image can be divided into a set number of third feature maps without considering depth information. The set number can be a*b, that is, the total number of third feature maps is a*b. Then, the eigenvalues ​​in each third feature map are max-pooled to obtain the maximum value of each eigenvalue in each third feature map. Finally, the feature vector of the sub-image is obtained, and the size of the feature vector in the sub-image is a*b.

[0105] The value of the set number can be determined based on actual conditions and is not limited in this application. The value of the set number is inversely proportional to the number of eigenvalues ​​included in each third characteristic graph. That is, the larger the set number, the fewer eigenvalues ​​each third characteristic graph includes, and the smaller the set number, the more eigenvalues ​​each third characteristic graph includes.

[0106] For example, if the size of the second feature map corresponding to a sub-image is 128*128, and the number is set to 2*2, the width and height of the second feature map of the sub-image can be divided into 64 blocks respectively. After dividing the second feature map corresponding to the sub-image, the total number of third feature maps obtained is 64*64, and the number of eigenvalues ​​included in each third feature map is 2*2. If the number is set to 16*16, the width and height of the second feature map of the sub-image can be divided into 16 blocks respectively. After dividing the second feature map corresponding to the sub-image, the total number of third feature maps obtained is 16*16, and the number of eigenvalues ​​included in each third feature map is 16*16. It can be seen that according to the former division method, the total number of third feature maps obtained after the second feature map of the sub-image is divided is larger, but the number of eigenvalues ​​included in each third feature map is smaller. The eigenvalues ​​used in pooling each third feature map are relatively small, and the resulting feature vector of the sub-image cannot clearly express the feature information of the sub-image. According to the latter division method, the total number of third feature maps obtained after the second feature map of the sub-image is divided is small, but the number of eigenvalues ​​included in each third feature map is large. When pooling each third feature map, more eigenvalues ​​are used, and the corresponding feature vector of the sub-image obtained can clearly express the feature information of the sub-image. Therefore, in the embodiment of the present application, it is preferred to divide the second feature map of each sub-image of the sample image according to the set number of 16*16, so as to obtain the feature vector corresponding to each sub-image.

[0107] Through the above processing, for a sample image, regardless of whether the sub-images corresponding to the sample image are of the same size, feature vectors of the same size can be obtained for each sub-image, and the feature vectors corresponding to each sub-image can well reflect the characteristics of the sample image. This can solve the technical problem that due to the inconsistent sizes of the sample images in the training sample set, it is necessary to first scale each sample image, resulting in deformation of the scaled sample images, and causing deviations between the features obtained after feature extraction from each deformed image and the features of the sample images themselves.

[0108] Optionally, determining the vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image includes:

[0109] For each sample image, determining an image area corresponding to vehicle position information of a vehicle in the sample image based on the information tag of the sample image;

[0110] For each sub-image, determining a degree of overlap between an image region corresponding to the sub-image in the sample image and an image region corresponding to the vehicle position information of the vehicle in the sample image;

[0111] For each sub-image, if the overlap is greater than or equal to a preset value, the vehicle label of the sub-image is determined to be that the sub-image contains a vehicle; otherwise, the vehicle label of the sub-image is determined to be that the sub-image does not contain a vehicle.

[0112] For a sub-image in a sample image, the vehicle label of the candidate box can be determined based on the degree of overlap (Intersection over Union, IoU) between the image area corresponding to the sub-image in the sample image and the area where the vehicle is located in the sample image. When IoU ≥ m, the sub-image is determined to be a positive sample of the sample image, and the vehicle label of the sub-image is determined to be that the image area corresponding to the sub-image in the sample image contains a vehicle (hereinafter referred to as, the vehicle label of the candidate box is contained vehicle). When IoU < m, the sub-image is determined to be a negative sample of the sample image, and the vehicle label of the sub-image is determined to be that the image area corresponding to the sub-image in the sample image does not contain a vehicle (hereinafter referred to as, the vehicle label of the candidate box is not contained vehicle). The value of m can be determined according to actual needs, and this application does not impose any restrictions on this. For example, m = 0.5 can be set.

[0113] Specifically, the first area of ​​the image area corresponding to the sub-image in the sample image can be used to determine the second area of ​​the area where the vehicle is located in the sample image according to the information label of the sample image. The third area of ​​the intersection area of ​​the two (the image area corresponding to the sub-image in the sample image and the area where the vehicle is located in the sample image) can be determined according to the image area corresponding to the sub-image in the sample image and the information label of the sample image, and IoU = third area / (first area+second area) can be determined.

[0114] By determining the degree of overlap between the image area corresponding to the sub-image in the sample image and the image area corresponding to the vehicle position information of the vehicle in the sample image, it is possible to determine whether each sub-image is a positive sample relative to its corresponding sample image. Thus, if the sub-image is a positive sample relative to its corresponding sample image, the vehicle label of the sub-image is determined to be that the sub-image contains a vehicle. Otherwise, the vehicle label of the sub-image is determined to be that the sub-image does not contain a vehicle. The vehicle label of each sub-image can be accurately determined.

[0115] Optionally, the target recognition network includes a classification module and a bounding box regression module, which are respectively cascaded with the feature extraction network. For each sub-image, based on the feature vector corresponding to the sub-image and the position information corresponding to the sub-image, the target recognition network obtains a recognition result of the sub-image, including:

[0116] According to the feature vector corresponding to the sub-image, a prediction classification result of the sub-image is obtained through a classification module;

[0117] According to the position information corresponding to the sub-image, the predicted vehicle position information of the sub-image is obtained through the bounding box regression module.

[0118] By adopting a multi-task learning approach, after the corresponding feature vectors of each sub-image and the corresponding position information of each sub-image are input into the target recognition network, the classification module and the bounding box regression module can share the features extracted by the feature extraction network, so as to simultaneously execute the classification task and the bounding box regression task of each sub-image corresponding to the sample image, and at the same time determine the classification results and predicted position information of each sub-image corresponding to each sample image, thereby improving the training efficiency.

[0119] In this implementation, the feature extraction network and the classification module can be cascaded through a fully connected network, and the feature extraction network and the bounding box regression module can be cascaded.

[0120] Specifically, the classification module can be a softmax classification module, which obtains the predicted classification result of each sub-image based on the feature vector corresponding to each sub-image. The predicted classification result of each sub-image can be whether the sub-image contains a vehicle, or can be the first confidence level corresponding to the sub-image containing a vehicle and the second confidence level corresponding to the sub-image not containing a vehicle, and this application does not impose any restrictions on this. When the predicted classification result of each sub-image is the first confidence level corresponding to the sub-image containing a vehicle and the second confidence level corresponding to the sub-image not containing a vehicle, the classification information corresponding to the larger confidence level between the first confidence level and the second confidence level can be determined as the predicted classification result of the sub-image. For example, when the first confidence level is 0.2 and the second confidence level is 0.8, it can be determined that the predicted classification result of the sub-image is that it does not contain a vehicle.

[0121] Based on the position information corresponding to the sub-image, the predicted vehicle position information of the sub-image is obtained through the bounding box regression module, including:

[0122] According to the position information corresponding to the sub-image, a correction offset of the sub-image is determined through a border regression module;

[0123] The position information of the sub-image is corrected based on the corrected offset of the sub-image to obtain predicted vehicle information of the sub-image.

[0124] Based on the bounding box regression module, a correction offset can be determined for each sub-image. Each sub-image can then be corrected based on the correction offset to obtain predicted position information for each sub-image. The correction offset for each sub-image is the correction offset corresponding to each element in the sub-image's position information. The predicted position information for a sub-image can be obtained by adding the corresponding correction offset to each element in the sub-image's position information.

[0125] As an example, in a sub-image, the position information corresponding to the sample image is (n, X center , Y center , W, H), the offset of the sub-image is (n, Δ1X center , Δ2Y center , Δ3W, Δ4H), where (Δ1X center , Δ2Y center ) represents the correction offset of the coordinates of the center position of the sub-image corresponding to the image area in the sample image, Δ3W represents the correction offset of the width of the sub-image corresponding to the image area in the sample image, and Δ4H represents the correction offset of the height of the sub-image corresponding to the image area in the sample image. After correction, the predicted position information of the sub-image can be (n, X center +Δ1X center , Y center +Δ2Y center , W+Δ3W, H+Δ4H).

[0126] In this implementation, when the vehicle label of a sub-image indicates that it does not contain a vehicle, the correction offset of the position information of each sub-image may be determined without using a bounding box regression network.

[0127] For a sample image, since the number of sub-images corresponding to the sample image is large through the selective search algorithm, there may be overlap between the sub-images. Therefore, after obtaining the predicted position information of each sub-image, the predicted position information of each sub-image can be screened based on non-maximum suppression (NMS), and redundant sub-images in each sub-image can be deleted to obtain the best target sub-image in each sub-image.

[0128] Figure 5 FIG. 1 shows a schematic diagram of a vehicle detection model training method proposed in an embodiment of the present application. Figure 5 As shown, during the training process of the vehicle detection model, the following steps S10 to S10 may be included.

[0129] Step S10: Obtain an image of any size. Specifically, obtain a training dataset comprising multiple training samples, each of which includes a sample image and a label for the sample image. The label includes a vehicle label, which indicates whether the sample image contains a vehicle. If the vehicle label indicates that the sample image contains a vehicle, the label also includes an information label, which indicates the location of the vehicle in the sample image. The sample images in the training sample may vary in size.

[0130] Step S20: Region Proposal. This involves using a selective search algorithm to obtain the location information of multiple candidate boxes corresponding to each sample image. Based on the information labels of each sample image, the vehicle labels of the multiple candidate boxes corresponding to each sample image are determined.

[0131] Based on the training data set and multiple candidate boxes corresponding to each sample image in the training sample set, the neural network model is iteratively trained until the training loss value of the neural network model meets the training end condition, thereby obtaining a vehicle detection model; the neural network model includes a sequentially cascaded convolution layer (i.e., the above-mentioned convolution module), a pooling layer (i.e., the above-mentioned pooling module), a ROI pooling layer (i.e., the above-mentioned module for determining the second feature map, the third feature map, and the feature vector), a first fully connected layer (i.e., the above-mentioned fully connected network), a second fully connected layer (i.e., the above-mentioned fully connected network), a classification layer (i.e., the above-mentioned classification module), a third fully connected layer (i.e., the above-mentioned fully connected network), and a bbox regression layer (i.e., the above-mentioned bounding box regression module). The neural network model also includes a second fully connected layer, a classification layer, and a bbox regression layer sequentially cascaded with the first fully connected layer. The neural network model also includes a third fully connected layer cascaded with the first fully connected layer. The convolution layer, the pooling layer, the ROI pooling layer, and the first fully connected layer can constitute the above-mentioned feature extraction network. The second fully connected layer, the classification layer, the third fully connected layer, and the bbox regression layer can constitute the above-mentioned target recognition network.

[0132] Taking a sample image as an example, the specific training process includes:

[0133] Step S30: performing feature extraction on the sample image based on the convolution layer and the pooling layer in sequence to obtain a feature map of the sample image.

[0134] Step S40: Based on the ROI pooling layer and the position information of multiple candidate boxes corresponding to the sample image, each candidate box corresponding to the sample image is mapped to the feature map of the sample image to obtain the feature map of each candidate box corresponding to the sample image;

[0135] The feature map of each candidate frame in the sample image is divided into a set number of third feature maps, and each third feature map obtained after the feature map of each candidate frame is divided is subjected to maximum pooling to obtain a feature vector of each candidate frame.

[0136] Step S50: The feature vector of each candidate box is sequentially input into the first fully connected layer, the second fully connected layer, and the classification layer to obtain the classification score corresponding to the candidate box (i.e., the above-mentioned confidence level). The position information of each candidate box is sequentially input into the first fully connected layer, the third fully connected layer, and the bbox regression layer to obtain the correction offset corresponding to the candidate box, so that each candidate box can be corrected based on the correction offset of each candidate box to obtain the predicted position information of each candidate box.

[0137] For each sample image, based on the predicted classification results of each candidate frame in the sample image and the vehicle label of each candidate frame, determine the first training loss value of each candidate frame, and determine the first training loss value of the sample image according to the first training loss value of each candidate frame;

[0138] Determine a second training loss value for each candidate frame based on the predicted position information of each candidate frame and the information label of the sample image corresponding to each candidate frame, and determine a second training loss value for the sample image according to the second training loss value of each candidate frame;

[0139] Determining a total loss value of the sample image based on the first training loss value of the sample image and the second training loss value of the sample image;

[0140] When the total loss value of the sample image does not meet the training end condition, the model parameters of the neural network model are adjusted until the neural network model converges to obtain a vehicle detection model.

[0141] An embodiment of the present application provides a vehicle detection method. Figure 6 A flow chart of the vehicle detection method provided in an embodiment of the present application is shown. Figure 6 As shown, the method includes steps S210 to S220.

[0142] Step S210: Acquire the image to be detected.

[0143] The image to be detected may be an image captured in real time by an image acquisition device, or may be any image, for example, any image obtained via the Internet, and this application does not impose any limitation thereto. It is understood that when the image to be detected is an image captured in real time by an image acquisition device, the image to be detected may also be an image obtained by intercepting an image of a video captured in real time by the image acquisition device.

[0144] As an example, an image capture device can capture a video of a vehicle and extract multiple frames from the video without frame drops, thereby performing vehicle detection on these images. Frame drops are defined as a pause in the video caused by a low frame rate, where the subsequent frame in the video is identical to the previous one. In this example, the multiple frames without frame drops are distinct images.

[0145] Step S220: Based on the trained vehicle detection model, determine whether the image to be detected includes a vehicle and the position information of the vehicle in the image to be detected. The trained vehicle detection model is obtained by training according to the above method.

[0146] It is understandable that when performing vehicle detection using the vehicle detection model trained using the above method, if it is determined that the image to be detected does not include a vehicle, it is not necessary to determine the position information of the vehicle in the image to be detected.

[0147] The vehicle detection model trained by the above method can be used for vehicle detection to quickly and accurately determine whether the image to be detected includes a vehicle, and, if the image to be detected includes a vehicle, obtain the location information of the vehicle in the image to be detected.

[0148] An embodiment of the present application also provides a vehicle detection model training device. Figure 7 A schematic diagram of a vehicle detection model training device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the apparatus 70 may include an acquisition module 701, a sub-image determination module 702, and a training module 703.

[0149] An acquisition module 701 is configured to acquire a training data set including a plurality of training samples, each training sample including a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating the location information of the vehicle in the sample image;

[0150] The sub-image determination module 702 is configured to obtain position information of at least two sub-images corresponding to each sample image in the sample image, and determine a vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image;

[0151] A training module 703 is configured to iteratively train a neural network model comprising a cascaded feature extraction network and an object recognition network based on each training sample and each sub-image corresponding to each sample image, until a training loss value of the neural network model satisfies a training termination condition, thereby obtaining a vehicle detection model.

[0152] The training end condition includes at least one of the following:

[0153] The loss function of the neural network model converges, the training loss value of the neural network model is less than the set value, and the number of training times of the neural network model reaches the set number.

[0154] Optionally, the loss function of the neural network model includes a first loss function and a second loss function. For each sample image, the training process includes:

[0155] Perform feature extraction on the sample image through a feature extraction network to obtain a first feature map corresponding to the sample image;

[0156] Determining a second feature map corresponding to each sub-image based on the first feature map and position information of each sub-image corresponding to the sample image in the sample image;

[0157] For each sub-image, based on the second feature map corresponding to the sub-image and the location information corresponding to the sub-image, a recognition result of the sub-image is obtained through the object recognition network. The recognition result includes a predicted classification result and predicted vehicle location information. The predicted classification result indicates whether the sub-image contains a vehicle.

[0158] Determining a first training loss value using a first loss function based on the predicted classification results of each sub-image corresponding to the sample image and the vehicle label of the corresponding sub-image; determining a second training loss value using a second loss function based on the predicted vehicle position information of each sub-image corresponding to the sample image and the information label of the sample image; and obtaining a training loss value of the neural network model based on the first training loss value and the second training loss value;

[0159] If the training loss value of the neural network model does not meet the training end condition, the model parameters of the neural network model are adjusted, and the adjusted model is continued to be trained based on each training sample and each sub-image corresponding to each sample image.

[0160] Optionally, the feature extraction network includes at least one feature extraction module and a pooling module cascaded with the at least one feature extraction module. For each sample image, the training module 703 performs feature extraction on the sample image through the feature extraction network to obtain a first feature map corresponding to the sample image, specifically for:

[0161] Performing feature extraction on the sample image by at least one feature extraction module to obtain a feature extraction result corresponding to the sample image;

[0162] Pooling the feature extraction results through a pooling module to obtain a first feature map corresponding to the sample image;

[0163] When determining the second feature map corresponding to each sub-image based on the first feature map and the position information of each sub-image corresponding to the sample image in the sample image, the training module 703 is specifically configured to:

[0164] Determining, according to the size of the pooling kernel of the pooling network, a mapping relationship between each eigenvalue of the first feature map and the image area in the sample image;

[0165] For each sub-image, determining the image area corresponding to the sub-image in the sample image according to the position information corresponding to the sub-image;

[0166] For each sub-image, according to the mapping relationship and the first feature map, the eigenvalues ​​corresponding to the image area corresponding to the sub-image in the first feature map are determined, and the second feature map corresponding to the sub-image is obtained based on the determined eigenvalues ​​corresponding to the sub-images.

[0167] After determining the second feature map corresponding to each sub-image, the training module 703 is further configured to:

[0168] For each sub-image, dividing the second feature map corresponding to the sub-image into a set number of third feature maps, performing feature extraction on each of the third feature maps corresponding to the sub-image, and obtaining a feature vector corresponding to the sub-image;

[0169] For each sub-image, the training module 703 is specifically configured to:

[0170] According to the feature vector corresponding to the sub-image and the position information corresponding to the sub-image, the recognition result of the sub-image is obtained through the target recognition network.

[0171] Optionally, for each sub-image, when the training module 703 performs feature extraction on each third feature map corresponding to the sub-image to obtain a feature vector corresponding to the sub-image, the training module 703 is specifically configured to:

[0172] For each third feature map, performing a pooling operation on each eigenvalue of the third feature map;

[0173] Based on the results of the pooling operation of all third feature maps corresponding to the sub-image, a feature vector corresponding to the sub-image is obtained.

[0174] Optionally, the target recognition network includes a classification module and a bounding box regression module, which are respectively cascaded with the feature extraction network. For each sub-image, the training module 703 obtains the recognition result of the sub-image through the target recognition network based on the feature vector corresponding to the sub-image and the position information corresponding to the sub-image, and is specifically configured to:

[0175] According to the feature vector corresponding to the sub-image, a prediction classification result of the sub-image is obtained through a classification module;

[0176] According to the position information corresponding to the sub-image, the predicted vehicle position information of the sub-image is obtained through the bounding box regression module.

[0177] Optionally, when determining the vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image, the training module 703 is specifically configured to:

[0178] For each sample image, determining an image area corresponding to vehicle position information of a vehicle in the sample image based on the information tag of the sample image;

[0179] For each sub-image, determining a degree of overlap between an image region corresponding to the sub-image in the sample image and an image region corresponding to the vehicle position information of the vehicle in the sample image;

[0180] For each sub-image, if the overlap is greater than or equal to a preset value, the vehicle label of the sub-image is determined to be that the sub-image contains a vehicle; otherwise, the vehicle label of the sub-image is determined to be that the sub-image does not contain a vehicle.

[0181] An embodiment of the present application also provides a vehicle detection device. Figure 8 A schematic diagram of a vehicle detection device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the device 80 includes an acquisition module 801 and a detection module 802.

[0182] An acquisition module 801 is used to acquire an image to be detected;

[0183] The detection module 802 is used to determine whether the image to be detected includes a vehicle and the position information of the vehicle in the image to be detected based on a trained vehicle detection model, and the trained vehicle detection model is obtained by training according to the above device.

[0184] The device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, and will not be repeated here.

[0185] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps and corresponding contents of the above method.

[0186] In an alternative embodiment, an electronic device is provided, Figure 9 FIG. 1 shows a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 9 As shown, Figure 9 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0187] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0188] Bus 4002 may include a path for transmitting information between the above components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0189] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0190] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0191] Based on the same principles as the methods provided in the embodiments of the present application, the embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in any of the above-mentioned optional embodiments of the present application.

[0192] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the above method are implemented.

[0193] An embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0194] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the order of implementation of these steps is not limited to the order indicated by the arrows. Unless otherwise clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage in these sub-steps or stages can also be executed at different times respectively. Under different scenarios at the execution time, the execution order of these sub-steps or stages can be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0195] The above description is only an optional implementation method for some implementation scenarios of this application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of this application, the use of other similar implementation methods based on the technical ideas of this application also falls within the protection scope of the embodiments of this application.

Claims

1. A vehicle detection model training method, characterized in that: include: Acquire a training data set including a plurality of training samples, each training sample including a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating vehicle position information in the sample image; Obtaining position information of at least two sub-images corresponding to each sample image in the sample image, and determining a vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image; Based on each of the training samples and each sub-image corresponding to each of the sample images, iteratively training a neural network model including a feature extraction network and a target recognition network cascaded in sequence until the neural network model satisfies a training termination condition, thereby obtaining a vehicle detection model; wherein the feature extraction network and the target recognition network are connected via a fully connected network; during the iterative training of the neural network model, singular value decomposition is used to accelerate the fully connected network, dividing the fully connected network into two sub-fully connected networks, wherein the first sub-fully connected network does not contain a bias and the second sub-fully connected network contains a bias; The training end condition includes at least one of the following: The loss function of the neural network model converges, the training loss value of the neural network model is less than a set value, and the number of training times of the neural network model reaches a set number; The loss function of the neural network model includes a first loss function and a second loss function. For each sample image, the training process includes: Performing feature extraction on the sample image through the feature extraction network to obtain a first feature map corresponding to the sample image; determining a second feature map corresponding to each sub-image according to the first feature map and position information of each sub-image corresponding to the sample image in the sample image; For each of the sub-images, obtaining a predicted classification result for the sub-image and predicted vehicle position information based on the second feature map corresponding to the sub-image and the position information corresponding to the sub-image through the object recognition network, wherein the predicted classification result indicates whether the sub-image includes a vehicle; Determining a first training loss value using the first loss function based on the predicted classification results of each sub-image corresponding to the sample image and the vehicle label of the corresponding sub-image; determining a second training loss value using the second loss function based on the predicted vehicle position information of each sub-image corresponding to the sample image and the information label of the sample image; and obtaining a training loss value of the neural network model based on the first training loss value and the second training loss value; If the training loss value of the neural network model does not meet the training end condition, the model parameters of the neural network model are adjusted, and the adjusted model is continued to be trained based on the sub-images corresponding to the training samples and the sample images.

2. The method according to claim 1, characterized in that The feature extraction network includes at least one feature extraction module and a pooling module cascaded with the at least one feature extraction module. For each of the sample images, extracting features of the sample image through the feature extraction network to obtain a first feature map corresponding to the sample image includes: Performing feature extraction on the sample image by the at least one feature extraction module to obtain a feature extraction result corresponding to the sample image; Pooling the feature extraction results by the pooling module to obtain a first feature map corresponding to the sample image; The determining, based on the first feature map and position information of each sub-image corresponding to the sample image in the sample image, a second feature map corresponding to each sub-image includes: Determining, according to the size of the pooling kernel of the pooling module, a mapping relationship between each eigenvalue of the first feature map and an image area in the sample image; For each of the sub-images, determining an image region corresponding to the sub-image in the sample image according to position information corresponding to the sub-image; For each of the sub-images, according to the mapping relationship and the first feature map, the eigenvalues ​​corresponding to the image area corresponding to the sub-image in the first feature map are determined, and the second feature map corresponding to the sub-image is obtained based on the determined eigenvalues ​​corresponding to the sub-images.

3. The method according to claim 1, characterized in that After determining the second feature map corresponding to each sub-image, the method further includes: For each sub-image, dividing the second feature map corresponding to the sub-image into a set number of third feature maps, performing feature extraction on each of the third feature maps corresponding to the sub-image, and obtaining a feature vector corresponding to the sub-image; For each of the sub-images, obtaining a recognition result of the sub-image through the object recognition network according to the second feature map corresponding to the sub-image and the position information corresponding to the sub-image includes: According to the feature vector corresponding to the sub-image and the position information corresponding to the sub-image, a recognition result of the sub-image is obtained through the target recognition network.

4. The method according to claim 3, characterized in that For each of the sub-images, performing feature extraction on each third feature map corresponding to the sub-image to obtain a feature vector corresponding to the sub-image includes: For each third feature map, performing a pooling operation on each eigenvalue of the third feature map; Based on the results of the pooling operation of all third feature maps corresponding to the sub-image, a feature vector corresponding to the sub-image is obtained.

5. The method according to claim 1, wherein The determining of the vehicle label of each sub-image corresponding to each sample image based on the information label of each sample image includes: For each of the sample images, determining an image area corresponding to the vehicle position information of the vehicle in the sample image based on the information tag of the sample image; For each sub-image, determining a degree of overlap between an image region corresponding to the sub-image in the sample image and an image region corresponding to the vehicle position information of the vehicle in the sample image; For each sub-image, if the degree of overlap is greater than or equal to a preset value, the vehicle label of the sub-image is determined to be that the sub-image contains a vehicle; otherwise, the vehicle label of the sub-image is determined to be that the sub-image does not contain a vehicle.

6. A vehicle detection method, characterized in that: include: Obtain the image to be detected; Based on a trained vehicle detection model, it is determined whether the image to be detected includes a vehicle and the position information of the vehicle in the image to be detected, wherein the trained vehicle detection model is trained according to the method according to any one of claims 1 to 5.

7. A vehicle detection model training device, characterized in that: The device includes an acquisition module, a sub-image determination module and a training module. an acquisition module, configured to acquire a training data set comprising a plurality of training samples, each training sample comprising a sample image, a vehicle label indicating whether the sample image contains a vehicle, and an information label indicating vehicle position information in the sample image; a sub-image determination module, configured to obtain position information of at least two sub-images corresponding to each sample image in the sample image, and determine a vehicle label of each sub-image corresponding to each sample image based on an information label of each sample image; A training module for iteratively training a neural network model comprising a cascaded feature extraction network and a target recognition network based on each of the training samples and each of the sub-images corresponding to each of the sample images, until a training loss value of the neural network model satisfies a training termination condition, thereby obtaining a vehicle detection model; wherein the feature extraction network and the target recognition network are connected via a fully connected network; during the iterative training of the neural network model, singular value decomposition is employed to accelerate the fully connected network, dividing the fully connected network into two sub-fully connected networks, wherein the first sub-fully connected network does not contain a bias and the second sub-fully connected network contains a bias; The training end condition includes at least one of the following: The loss function of the neural network model converges, the training loss value of the neural network model is less than a set value, and the number of training times of the neural network model reaches a set number; The loss function of the neural network model includes a first loss function and a second loss function. For each sample image, the training process includes: Performing feature extraction on the sample image through the feature extraction network to obtain a first feature map corresponding to the sample image; determining a second feature map corresponding to each sub-image according to the first feature map and position information of each sub-image corresponding to the sample image in the sample image; For each of the sub-images, obtaining a predicted classification result for the sub-image and predicted vehicle position information based on the second feature map corresponding to the sub-image and the position information corresponding to the sub-image through the object recognition network, wherein the predicted classification result indicates whether the sub-image includes a vehicle; Determining a first training loss value using the first loss function based on the predicted classification results of each sub-image corresponding to the sample image and the vehicle label of the corresponding sub-image; determining a second training loss value using the second loss function based on the predicted vehicle position information of each sub-image corresponding to the sample image and the information label of the sample image; and obtaining a training loss value of the neural network model based on the first training loss value and the second training loss value; If the training loss value of the neural network model does not meet the training end condition, the model parameters of the neural network model are adjusted, and the adjusted model is continued to be trained based on the sub-images corresponding to the training samples and the sample images.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5, or to implement the steps of the method according to claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 5, or implements the steps of the method according to claim 6.

Images