Vehicle information extraction method

Abstract

The application discloses a vehicle information extraction method and relates to the technical field of computation, which comprises the following steps: S1, constructing an information extraction model; S2, obtaining a training sample set; S3, importing the training sample set into the information extraction model and training and optimizing the same; S4, placing the optimized information extraction model as a static resource in a front-end project folder; S5, obtaining a vehicle image to be extracted; S6, calling the optimized information extraction model to extract information of the vehicle image to be extracted through a front-end webpage or a mobile terminal application, and returning the information to a user interface; the method can bypass the cumbersome process of a back end, directly extract information in a vehicle picture and perform classification processing, has higher flexibility and universality, the information extraction model can extract features and solve an overfitting problem, and GPU acceleration calculation is used to reduce training time; through a feature fusion module, output features of refined feature information in each stage are used to easily aggregate global context, only a small amount of calculation is generated, and an upsampling operation is not needed.

Description

Technical Field

[0001] This invention relates to the field of computing technology, and in particular to a method for extracting vehicle information. Background Technology

[0002] Vehicle information extraction is a technology that uses computers to analyze user-uploaded vehicle images. By processing and analyzing images of specific areas, it detects and classifies vehicle information. This technology is easy to use and maintain; the trained neural network model is simply packaged as a static resource during front-end development and then deployed to the server. Typically, neural network models are large (tens to hundreds of megabytes) and cannot be directly applied to the front-end. Instead, image data is passed to the neural network model for processing via back-end APIs, and the extracted information is returned to the front-end. However, lightweight neural network models can be directly processed by the front-end, analyzing and extracting the required vehicle information in real time. Vehicle information extraction methods have broad application prospects in traffic management, vehicle insurance, and other fields, comprehensively improving user experience and optimizing system processes, playing an increasingly important role in intelligent extraction systems. There are many commonly used methods for vehicle recognition algorithms, such as module matching and statistical patterns.

[0003] Speed ​​and accuracy are two key performance indicators (KPIs) for image-based vehicle information extraction systems. In such systems, extracting vehicle features is a crucial factor influencing these KPIs and is key to the entire recognition process. However, vehicle feature extraction is affected by various factors, such as the large variety of vehicle types without obvious features, image clarity, and weather and lighting conditions. These factors pose significant challenges to vehicle information extraction. Summary of the Invention

[0004] The purpose of this invention is to design a vehicle information extraction method to solve the above-mentioned problems.

[0005] The present invention achieves the above objectives through the following technical solutions:

[0006] Vehicle information extraction methods include:

[0007] S1. Construct an information extraction model. The information extraction model, from input to output, includes a first downsampling layer, a first convolutional module, a second downsampling layer, a second convolutional module, a third downsampling layer, a third convolutional module, a feature fusion module, and a classifier. The information extraction model also includes a first feature extraction module and a second feature extraction module. The output of the second convolutional module serves as the input of the first feature extraction module, and the output of the third convolutional module serves as the input of the second feature extraction module. The outputs of both the first and second feature extraction modules serve as the input of the feature fusion module.

[0008] S2. Obtain the training sample set and label the basic information of the samples;

[0009] S3. Import the training sample set into the information extraction model and train and optimize it to obtain the optimized information extraction model;

[0010] S4. Place the optimized information extraction model as a static resource in the front-end project folder and package and deploy it.

[0011] S5. The front-end webpage or mobile application obtains the vehicle image to be extracted.

[0012] S6. The front-end webpage or mobile application calls the optimized information extraction model to extract information from the vehicle image to be extracted and returns the information to the user interface.

[0013] The beneficial effects of this invention are as follows: This method bypasses the cumbersome backend process, directly extracting information from vehicle images and performing classification processing, thus possessing greater flexibility and universality. Feature extraction using this information extraction model can solve the overfitting problem, and GPU acceleration reduces training time. Furthermore, the feature fusion module aggregates input features, refines output features based on each stage, easily aggregates the global context, and generates only a minimal computational load, requiring no upsampling. Moreover, this feature fusion module employs multi-feature fusion, capable of extracting and fusing rich feature information, and customizing the classifier according to the characteristics of the dataset to improve the accuracy of the entire neural network model. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the vehicle information extraction method of the present invention;

[0015] Figure 2 This is a schematic diagram of the information extraction model in the vehicle information extraction method of the present invention;

[0016] Figure 3 This is a schematic diagram of the feature fusion module in the vehicle information extraction method of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0018] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0020] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0021] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0022] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0023] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] like Figure 1 , Figure 2 , Figure 3 As shown, the vehicle information extraction method includes:

[0025] S1. Construct an information extraction model. The information extraction model, from input to output, includes a first downsampling layer, a first convolutional module, a second downsampling layer, a second convolutional module, a third downsampling layer, a third convolutional module, a feature fusion module, and a classifier. The information extraction model also includes a first feature extraction module and a second feature extraction module. The output of the second convolutional module serves as the input of the first feature extraction module, and the output of the third convolutional module serves as the input of the second feature extraction module. The outputs of both the first and second feature extraction modules serve as the input of the feature fusion module.

[0026] S2. Obtain the training sample set and label the basic information of the samples;

[0027] S3. Import the training sample set into the information extraction model and train and optimize it to obtain the optimized information extraction model;

[0028] S4. Place the optimized information extraction model as a static resource in the front-end project folder and package and deploy it.

[0029] S5. The front-end webpage or mobile application obtains the vehicle image to be extracted.

[0030] S6. The front-end webpage or mobile application calls the optimized information extraction model to extract information from the vehicle image to be extracted and returns the information to the user interface.

[0031] The first convolutional module consists of an x-layer first convolutional layer, the second convolutional module consists of a y-layer second convolutional layer, and the third convolutional module consists of a z-layer third convolutional layer, where x=2, y=3, and z=8.

[0032] The convolution kernels of the first, second, and third convolutional layers all use a hybrid mode of 1×3 and 3×1, with a convolution stride of 3.

[0033] The first and second feature extraction modules split the input feature map into two parts and extract features from each part separately. Then, they perform dot product combination and finally process the data using batch normalization and the ReLU activation function, as shown below:

[0034] F(t) =

[0035] Where F / 2 represents the feature map after the number of input feature map channels is halved, Conv 1*3 and Conv 3*1 These represent convolutional blocks with kernels of 1*3 and 3*1, respectively. represents dot product calculation, BatchNorm represents batch normalization, and ReLU represents the activation function.

[0036] The feature fusion module consists of a fourth convolutional layer, a fifth convolutional layer, an attention module, and a feature fusion layer, from input to output. The output of the seventh convolutional layer is also used as the input of the feature fusion layer. The convolutional kernels of the fourth and fifth convolutional layers both adopt a hybrid mode of 1×3 and 3×1.

[0037] The attention module consists of a global average pooling layer, a sixth convolutional layer, a batch normalization layer, and a sigmoid layer, from input to output. The convolutional kernel of the sixth convolutional layer uses a 1×1 convolutional block.

[0038] The classifier uses a weighted allocation scheme to classify the feature vectors output by the feature fusion module, obtaining the final classification result; the weighted allocation scheme is expressed as:

[0039]

[0040] Where N represents the number of categories in the dataset; This represents the number of pixels in the output feature multiplied by the number of channels. This represents the minimum value in the set of weights for all categories. This represents the maximum value in the set of weights for all categories. This represents the weight of each category after calculation.

[0041] The working principle of the vehicle information extraction method of this invention is as follows:

[0042] A: Preprocessing the training samples: It's important to understand that each image is represented by pixels ranging from 0 to 255. Therefore, a normalization formula is needed to convert images with pixels in the 0-255 range into images with pixels in the 0-1 range.

[0043] B: Determine the number of layers in the information extraction model: The information extraction model consists of a 16-layer backbone network and an output feature fusion module. The backbone network comprises 3 downsampling layers and 13 specific residual layers, ultimately outputting three feature information values ​​to the feature fusion module. Finally, the feature fusion module uses the end-to-end characteristics of the convolutional neural network to perform classification processing on the fused feature information.

[0044] C: Determine the network structure of the backbone network: When each image is input into the information extraction model, it is compressed to a size of 512*512 by the downsampling layer and divided into three color dimensions: red, green, and blue. The input image will go through 3 downsampling layers and 13 convolutional layers for feature information abstraction. The convolution kernels use a mixed mode of 1*3 and 3*1, and the convolution stride is 3. After calculation, it can be found that the output feature map size of the backbone network is 12*12, and the number of feature maps is 512.

[0045] To improve training speed while reducing model size, the residual layer is designed using a hybrid mode of 1×3 and 3×1. The input feature map is split into two parts, and features are extracted separately for each part. Finally, a dot product is performed to combine the extracted features. Existing techniques such as ReLU activation and batch normalization are also used, as shown below:

[0046] F(t) =

[0047] Where F / 2 represents the feature map after the number of input feature map channels is halved, Conv 1*3 and Conv 3*1 These represent convolutional blocks with kernels of 1*3 and 3*1, respectively. This represents dot product calculation, BatchNorm represents batch normalization, and ReLU represents the activation function. The purpose of this is to reduce the computational cost by half after splitting the number of channels, and to extract features using convolutional blocks with 1*3 and 3*1 kernels. This improves the model's speed and reduces its size without sacrificing accuracy.

[0048] D: Determine the network structure of the feature fusion module: The 24*24*512 feature map output from the backbone network is input to the feature fusion module. Two sets of 1*3 and 3*1 group convolutions are used to merge and aggregate the spatial feature information of the input. Then, an attention vector is calculated to guide feature learning. The purpose of this design is to refine the output features at each stage after aggregating the input features. It can easily aggregate the global context without any upsampling operations, thus the computational cost is negligible.

[0049] The attention vector is calculated as follows:

[0050] F(T)=Sigmoid(BatchNorm(Conv(AvgPool(F))))

[0051] Where F represents the input feature information, AvgPool represents global average pooling, Conv represents a 1*1 convolutional layer, BatchNorm represents batch normalization, and Sigmoid represents a sigmoid activation function.

[0052] The calculation method for the feature fusion module is as follows:

[0053]

[0054] in, Indicates output features, This represents the formula for calculating the attention vector, Conv. 1*3 and Conv 3*1These represent convolutional blocks with kernels of 1*3 and 3*1, respectively. This represents the input features.

[0055] Finally, after processing by the backbone network and feature fusion module of the neural network, the size of the feature map is 12*12, with a total of 512 feature maps.

[0056] E: Obtain the feature vector: Based on the output of D, calculate the number of nodes in the fully connected layer output as 12*12*512=73728; arrange the 12*12 feature maps obtained from D into a column vector, and obtain the final feature vector.

[0057] F: Feature Classification: We utilize the end-to-end nature of convolutional neural networks and employ a specific weight allocation scheme to classify the feature vectors obtained from E, yielding the final classification result. The allocation formula is as follows:

[0058]

[0059] Where N represents the number of categories in the dataset; This represents the number of pixels in the output feature multiplied by the number of channels. This represents the minimum value in the set of weights for all categories. This represents the maximum value in the set of weights for all categories. This represents the weight of each category after calculation.

[0060] Compared to traditional methods, this method's information extraction model is smaller, only about 7MB, and can be packaged and deployed as a front-end static resource to front-end pages or mobile applications. Furthermore, this method bypasses cumbersome back-end processes, directly extracting information from vehicle images and performing classification, offering greater flexibility and versatility. Feature extraction using this model addresses overfitting issues, and GPU acceleration improves training time. Additionally, the feature fusion module aggregates input features, refining output features based on each stage, easily aggregating global context with minimal computational cost and no upsampling or downsampling operations. This method also combines a multi-feature fusion module with support vector machines, enabling the extraction and fusion of rich feature information, and allowing for customized classifiers based on dataset characteristics to improve the accuracy of the entire neural network model.

[0061] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A method for extracting vehicle information, characterized in that, include: S1. Construct an information extraction model. From input to output, the information extraction model includes a first downsampling layer, a first convolutional module, a second downsampling layer, a second convolutional module, a third downsampling layer, a third convolutional module, a feature fusion module, and a classifier. The model also includes a first feature extraction module and a second feature extraction module. The output of the second convolutional module serves as the input to the first feature extraction module, and the output of the third convolutional module serves as the input to the second feature extraction module. The outputs of both the first and second feature extraction modules serve as the input to the feature fusion module. The first and second feature extraction modules split the input feature map into two parts, extract features from each part separately, then perform dot product combination, and finally process the data using batch normalization and the ReLU activation function. This can be represented as: Where F / 2 represents the feature map after the number of input feature map channels is halved, Conv 1*3 and Conv 3*1 These represent convolutional blocks with kernels of 1*3 and 3*1, respectively. represents dot product, BatchNorm represents batch normalization, and ReLU represents the activation function; the classifier uses a weighted allocation scheme to classify the feature vectors output by the feature fusion module, obtaining the final classification result; the weighted allocation scheme is expressed as: Where N represents the number of categories in the dataset; This represents the number of pixels in the output feature multiplied by the number of channels. This represents the minimum value in the set of weights for all categories. This represents the maximum value in the set of weights for all categories. This represents the calculated weight of each category; S2. Obtain the training sample set and label the basic information of the samples; S3. Import the training sample set into the information extraction model and train and optimize it to obtain the optimized information extraction model; S4. Place the optimized information extraction model as a static resource in the front-end project folder and package and deploy it. S5. The front-end webpage or mobile application obtains the vehicle image to be extracted. S6. The front-end webpage or mobile application calls the optimized information extraction model to extract information from the vehicle image to be extracted and returns the information to the user interface.

2. The vehicle information extraction method according to claim 1, characterized in that, The first convolutional module includes an x-layer first convolutional layer, the second convolutional module includes a y-layer second convolutional layer, and the third convolutional module includes a z-layer third convolutional layer.

3. The vehicle information extraction method according to claim 2, characterized in that, x=2, y=3, z=8.

4. The vehicle information extraction method according to claim 2, characterized in that, The convolution kernels of the first, second, and third convolutional layers all use a hybrid mode of 1×3 and 3×1, with a convolution stride of 3.

5. The vehicle information extraction method according to claim 1, characterized in that, The feature fusion module consists of a fourth convolutional layer, a fifth convolutional layer, an attention module, and a feature fusion layer, from input to output. The output of the seventh convolutional layer is also used as the input of the feature fusion layer. The convolutional kernels of the fourth and fifth convolutional layers both adopt a hybrid mode of 1×3 and 3×1.

6. The vehicle information extraction method according to claim 5, characterized in that, The attention module consists of a global average pooling layer, a sixth convolutional layer, a batch normalization layer, and a sigmoid layer, from input to output. The convolutional kernel of the sixth convolutional layer uses a 1×1 convolutional block.

Images