Method and device of online car-hailing vehicle detection and classification system based on deep learning

Abstract

The invention discloses a method and a device of an online car-hailing vehicle detection and classification system based on deep learning, and particularly relates to the technical field of image processing. Online car-hailing image data including vehicle images under different vehicle types, different angles and different illumination conditions are randomly selected from an online database, and corresponding vehicle types are labeled; preprocessing and feature engineering processing are carried out on the collected data to generate a training set, a test set and a verification set, the feature engineering processing is to extract pixel values, color histograms and size features of vehicle images from the preprocessed images, and according to the existing vehicle image data, the vehicle images are processed to generate a training set, a test set and a verification set; the method comprises the following steps of: rotating, overturning, scaling, changing brightness and contrast of existing vehicle image data by adopting a data enhancement technology to generate a plurality of groups of vehicle image samples, and carrying out model training by adopting a convolutional neural network and a recurrent neural network algorithm to optimize a classification effect.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a method and device for a taxi vehicle detection and classification system based on deep learning. Background Art

[0002] With the rapid development of the mobile Internet and the sharing economy, taxi services have become an important choice for people to travel. However, the subsequent problems of vehicle management and operation efficiency have become increasingly prominent. How to improve the vehicle detection and classification ability has become a key technical problem that taxi companies need to solve.

[0003] Traditional vehicle detection and classification technologies usually rely on manual feature extraction and classical machine learning algorithms, such as support vector machines, random forests, etc. These methods have achieved certain results in their early applications, but due to their dependence on manually designed features, they often struggle to withstand the challenges in complex environments. Specifically, traditional methods are prone to problems such as low recognition accuracy and insufficient real-time performance when dealing with vehicle images at different angles, under different lighting conditions, and in complex scenarios. In addition, the design of artificial features lacks universality and is difficult to adapt to the changing vehicle appearance and form. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method and device for a taxi vehicle detection and classification system based on deep learning to solve the problems raised in the above background art.

[0005] To achieve the above object, the present invention provides the following technical solution, a method for a taxi vehicle detection and classification system based on deep learning, specifically including the following steps:

[0006] Step S1: Randomly select taxi vehicle image data from an online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types;

[0007] Step S2: Preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set;

[0008] Step S3: According to the existing vehicle image data, use data augmentation technology to generate multiple sets of vehicle image data, and use convolutional neural network and recurrent neural network algorithms for model training to optimize the classification effect.

[0009] In a preferred embodiment, in step S1, randomly select taxi vehicle image data from an online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types. The specific steps are as follows:

[0010] Step A1, Random Sampling: Use the method of random sampling to select data from the online database to ensure that the selected samples are representative. Utilize the uniform sampling algorithm S = {x 1 , x 2 ,..., x k}, where x i ∈ D, i = 1, 2,..., k, D represents all vehicle images in the database, and S represents the selected sample set;

[0011] Step A2, Label Assignment: For each selected image, classify it according to its appearance and features, and assign a label to each image, which represents the type of vehicle in the image. When the image x has the vehicle type y, the mathematical expression is: f: x → y, where f is the function used to map the image to the corresponding vehicle type;

[0012] Step A3, Data Recording: Store the selected images and their corresponding labels in the database, denoted as T = {(x i , f(x i ))|x i ∈ S, i = 1, 2,..., k}, where T represents the set of stored images and their labels, and f(x i ) represents the vehicle type label of the image x i , and verify the labeled data to ensure the accuracy of the labels and remove the incorrect data.

[0013] In a preferred embodiment, in step S2, preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set. The specific steps are as follows:

[0014] Step B1, Image Preprocessing: Include operations such as image normalization, denoising, and resizing. Check for duplicate images and invalid images in the image dataset and remove them. Resize all images to the same size. Denote the original image size as (H o , W o ), and the target image size as (H t , W t ). Use the interpolation method to resize the original image to the specified height and width: x resized = Resize(x, H t , W t ), where x resized represents the image after the resizing operation, x is the original input image, (H o , W o ) is the height and width of the original image, and (H t , W t ) is the height and width of the target image;

[0015] Step B2. Feature engineering processing: Extract the pixel values, color histograms, and size features of the vehicle images from the preprocessed images, and divide the processed data into a training set, a validation set, and a test set in chronological order. This further includes the following steps:

[0016] Step B201. Extract the pixel intensity values of each color channel to form a feature vector, denoted as: f pixels =[p R ,p G ,p B , where p C represents the average pixel intensity of color channel C, and f pixels is the pixel intensity value feature; x processed (i,j,C) represents the pixel value of color channel C at position (i,j) in the preprocessed image, C represents the color channel, and its values are R, G, B, H, and W are the height and width of the image;

[0017] Step B202. Calculate the color histogram of the image on each color channel to capture the color distribution feature, denoted as: where h C (m) represents the pixel count of color channel C in the m-th color bin, N is the number of bins of the color histogram, 1 is a marker function that returns 1 if the pixel value falls within the current bin range and 0 otherwise;

[0018] Step B203. Calculate the bounding box of the vehicle and its area to obtain the size feature: Area = (x max -x min )×(y max -y min ), where Area is the area of the bounding box, (x min ,y min ) is the upper left coordinate of the bounding box, and (x max ,y max ) is the lower right coordinate of the bounding box. Combine all the extracted features into a feature vector: where f pixels is the pixel intensity value feature, h C is the color distribution feature, and Area is the area of the bounding box, representing the size feature.

[0019] In a preferred embodiment, in step S3, according to the existing vehicle image data, use data augmentation techniques to generate multiple sets of vehicle image data, and use convolutional neural network and recurrent neural network algorithms for model training to optimize the classification effect. The specific steps are as follows:

[0020] Step C1, Data Augmentation: Perform operations such as rotation, flipping, scaling, and changing brightness and contrast on the existing vehicle image data to generate multiple sets of vehicle image samples, so as to increase the diversity of training data;

[0021] Step C2, Model Training: Use the training set to train the convolutional neural network model. Input the feature vector X into the convolutional neural network, and perform feature extraction and classification through multiple convolutional, pooling, and fully connected layers. Use the cross-entropy loss function to optimize the model: where L is the loss function, y is the true label, is the predicted label, λ is the number of classes, y q is the label of the q-th class, is the predicted probability of the q-th class. When the input image is sequential data, use the recurrent neural network to capture the temporal dependencies: h t = f(W h h t-1 + W x X), where h t and h t-1 are the current and previous hidden states respectively, W h is the weight matrix from the hidden state to the hidden state, W x is the weight matrix from the current input feature vector to the hidden state, X is the current input feature vector, t represents the time step, and evaluate the performance of the model on the test set.

[0022] This application also provides a device for an online car-hailing vehicle detection and classification system based on deep learning, specifically including a data acquisition module, a data processing module, and a model training module;

[0023] Data Acquisition Module: Randomly select online car-hailing image data from the online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types;

[0024] Data Processing Module: Preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set;

[0025] Model Training Module: According to the existing vehicle image data, use data augmentation technology to generate multiple sets of vehicle image data, and use convolutional neural network and recurrent neural network algorithms for model training to optimize the classification effect.

[0026] The beneficial effects of the present invention are as follows: randomly select online car-hailing image data from an online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types. Preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set. The feature engineering process extracts the pixel values, color histograms, and size features of the vehicle images from the preprocessed images. According to the existing vehicle image data, use data augmentation techniques to perform operations such as rotation, flipping, scaling, changing brightness, and contrast on the existing vehicle image data to generate multiple sets of vehicle image samples. Use convolutional neural network and recurrent neural network algorithms for model training to optimize the classification effect. Through deep learning for vehicle detection, different types of online car-hailing can be accurately identified and classified, improving the accuracy of detection and reducing the false alarm rate. Description of the Drawings

[0027] Figure 1 It is a flowchart of the method of the present invention. Detailed Embodiments

[0028] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

[0029] In the description of the present application, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more, unless otherwise specifically defined.

[0030] In the description of the present application, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present application is not necessarily construed as being more preferred or having more advantages than other embodiments. In order for any person skilled in the art to implement and use the present invention, the following description is given. In the following description, details are set forth for the purpose of explanation. It should be understood that those skilled in the art can recognize that the present invention can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessary details from obscuring the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope that conforms to the principles and features disclosed in the present application.

[0031] Example 1

[0032] This example provides a method for a car-hailing vehicle detection and classification system based on deep learning as shown in Figure 1 , which specifically includes the following steps:

[0033] Step S1: Randomly select car-hailing vehicle image data from the online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types;

[0034] Step S2: Preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set;

[0035] Step S3: According to the existing vehicle image data, use data augmentation techniques to generate multiple sets of vehicle image data, and use convolutional neural network and recurrent neural network algorithms for model training to optimize the classification effect.

[0036] Preferably, in step S1, randomly select car-hailing vehicle image data from the online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types, which can obtain more accurate supervision signals during model training and improve the classification accuracy. The specific steps are as follows:

[0037] Step A1: Random sampling: Use the method of random sampling to select data from the online database to ensure that the selected samples are representative. Use the uniform sampling algorithm S = {x 1 , x 2 ,..., x k}, where x i ∈ D, i = 1, 2,..., k, D represents all vehicle images in the database, and S represents the selected sample set;

[0038] Step A2: Label assignment: For each selected image, classify it according to its appearance and features, and assign a label to each image, representing the type of vehicle in the image. When the image x has the vehicle type y, the mathematical expression is: f: x → y, where f is a function used to map the image to the corresponding vehicle type;

[0039] Step A3: Data recording: Store the selected images and their corresponding labels in the database, represented as T = {(x i , f(x i ))|x i ∈ S, i = 1, 2,..., k}, where T represents the set of stored images and their labels, and f(x i ) represents the vehicle type corresponding to the image x iVehicle type labels, and verify the labeled data to ensure the accuracy of the labels and remove inconsistent data.

[0040] Preferably, in step S2, preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set, ensuring that the data for subsequent analysis and training is more accurate and improving the overall quality of the data set. The specific steps are as follows:

[0041] Step B1, Image preprocessing: including image normalization, denoising, and resizing operations. Check for duplicate and invalid images in the image data set and remove them. Resize all images to the same size. Represent the original image size as (H o , W o ), and the target image size as (H t , W t ). Use interpolation to resize the original image to the specified height and width: x resized = Resize(x, H t , W t ), where x resized represents the image after the resizing operation, x is the original input image, (H o , W o ) are the height and width of the original image, and (H t , W t ) are the height and width of the target image;

[0042] Step B2, Feature engineering processing: Extract the pixel values, color histograms, and size features of the vehicle images from the preprocessed images, and divide the processed data into a training set, a validation set, and a test set in chronological order. It further includes the following steps:

[0043] Step B201, Extract the pixel intensity values of each color channel to form a feature vector, denoted as: f pixels = [p R , p G , p B , where p C represents the average pixel intensity of color channel C, and f pixels is the pixel intensity value feature; x processed (i, j, C) represents the pixel value of color channel C at position (i, j) in the preprocessed image, C represents the color channel, and the values are R, G, B, H, and W are the height and width of the image;

[0044] Step B202, Calculate the color histogram of the image on each color channel to capture the color distribution feature, denoted as: where h C(m) represents the pixel count of color channel C in the m-th color group, N is the number of groups of the color histogram, 1 is a marking function that returns 1 if the pixel value falls within the current group range and 0 otherwise;

[0045] Step B203: Calculate the bounding box of the vehicle and its area, and obtain the size feature as: Area = (x max - x min ) × (y max - y min ), where Area is the area of the bounding box, (x min , y min ) is the upper left coordinate of the bounding box, and (x max , y max ) is the lower right coordinate of the bounding box. Combine all the extracted features into a feature vector: Among them, f pixels is the pixel intensity value feature, h C is the color distribution feature, and Area is the area of the bounding box, representing the size feature.

[0046] Preferably, in step S3, according to the existing vehicle image data, data augmentation technology is used to generate multiple sets of vehicle image data, which can increase the diversity of samples and reduce the risk of overfitting. The convolutional neural network and recurrent neural network algorithms are used for model training to optimize the classification effect. The specific steps are as follows:

[0047] Step C1: Data augmentation: Perform operations such as rotation, flipping, scaling, changing brightness and contrast on the existing vehicle image data to generate multiple sets of vehicle image samples to increase the diversity of training data;

[0048] Step C2: Model training: Use the training set to train the convolutional neural network model. Input the feature vector X into the convolutional neural network, and perform feature extraction and classification through multiple convolutional, pooling, and fully connected layers. Use the cross-entropy loss function to optimize the model: Among them, L is the loss function, y is the true label, is the predicted label, λ is the number of categories, y q is the label of the q-th category, is the predicted probability of the q-th category. When the input image is sequence data, use the recurrent neural network to capture the temporal dependence: h t = f(W h h t-1 + W x X), where h t and h t-1 are the current and previous hidden states respectively, W h is the weight matrix from the hidden state to the hidden state, and W xis the weight matrix from the current input feature vector to the hidden state, X is the current input feature vector, t represents the time step, and the performance of the model is evaluated on the test set.

[0049] Example 2

[0050] This embodiment provides a device for an online car-hailing vehicle detection and classification system based on deep learning, specifically including a data acquisition module, a data processing module, and a model training module;

[0051] Data acquisition module: Randomly select online car-hailing vehicle image data from the online database, including vehicle images under different vehicle models, different angles, and different lighting conditions, and label the corresponding vehicle types;

[0052] Data processing module: Preprocess and perform feature engineering on the collected data to generate a training set, a test set, and a validation set;

[0053] Model training module: According to the existing vehicle image data, use data augmentation techniques to generate multiple sets of vehicle image data, and use convolutional neural network and recurrent neural network algorithms for model training to optimize the classification effect.

[0054] It should be noted that in the above embodiments, the descriptions of each embodiment have their own emphases. For parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0055] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the function specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in one or more of the blocks or blocks.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in one or more of the blocks or blocks.

[0059] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0060] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for a vehicle detection and classification system for online ride-hailing vehicles based on deep learning, characterized in that: The specific steps include: Step S1: randomly select online car-hailing image data from the online database, including vehicle images of different models, different angles and different lighting conditions, and label the corresponding vehicle types; Step S2: preprocessing and feature engineering the collected data to generate training sets, test sets, and validation sets; Step S3: Based on the existing vehicle image data, multiple groups of vehicle image data are generated using data enhancement technology, and model training is performed using convolutional neural network and recurrent neural network algorithms to optimize the classification effect.

2. The method of a deep learning-based online car-hailing vehicle detection and classification system according to claim 1, characterized in that: In step S1, online car-hailing image data is randomly selected from the online database, including vehicle images of different models, different angles and different lighting conditions, and the corresponding vehicle types are labeled. The specific steps are as follows: Step A1, random sampling: Use random sampling method to select data from the online database to ensure that the selected samples are representative. Use uniform sampling algorithm S = {x1, x2, ..., x k }, where x i ∈D,i=1,2,...,k, D represents all vehicle images in the database, S represents the selected sample set; Step A2, label assignment: for each selected image, classify it according to its appearance and features, and assign a label to each image, which is represented by the type of vehicle in the image. When image x has vehicle type y, the mathematical expression is: f:x→y, where f is the function used to map the image to the corresponding vehicle type; Step A3, data recording: the selected images and their corresponding labels are stored in the database, represented as T = {(x i ,f(x i ))|x i ∈S,i=1,2,...,k}, where T represents the set of stored images and their labels, f(x i ) represents the image x i The vehicle type label is added and the annotated data is verified to remove the erroneous data.

3. The method of a deep learning-based online car-hailing vehicle detection and classification system according to claim 1, characterized in that: In step S2, the collected data is preprocessed and feature engineered to generate a training set, a test set, and a validation set. The specific steps are as follows: Step B1, image preprocessing: including image standardization, denoising, and resizing operations, checking duplicate images and invalid images in the image data set and removing them, adjusting all images to the same size, and representing the original image size as (H o ,W o ), the target image size is (H t ,W t ), resizes the original image to the specified height and width using interpolation: x resized =Resize(x,H t ,W t ), where x resized represents the image after resizing operation, x is the original input image, (H o ,W o ) is the height and width of the original image, (H t ,W t ) are the height and width of the target image; Step B2, feature engineering processing: extract the pixel value, color histogram and size features of the vehicle image from the preprocessed image, and divide the processed data into a training set, a validation set and a test set in chronological order.

4. The method of a deep learning-based online car-hailing vehicle detection and classification system according to claim 3, characterized in that: In the feature engineering process of step B2, the pixel value, color histogram and size features of the vehicle image are extracted from the preprocessed image, further comprising the following steps: Step B201: extract the pixel intensity value of each color channel to form a feature vector, expressed as: pixels =[p R ,p G ,p B ],in, p C represents the average pixel intensity of color channel C, f pixels is the pixel intensity value feature; x processed (i, j, C) represents the pixel value of the color channel C of the preprocessed image at position (i, j), C represents the color channel, and its value is R, G, B, H and W are the height and width of the image; Step B202: Calculate the color histogram of the image in each color channel to capture the color distribution characteristics, expressed as: Among them, h C (m) represents the pixel count of color channel C in the mth color group, N is the number of groups in the color histogram, 1 is the marking function, and returns 1 if the pixel value falls within the current group range, otherwise returns 0; Step B203: Calculate the bounding box and area of ​​the vehicle, and obtain the size feature: Area = (x max -x min )×(y max -y min ), where Area is the area of ​​the bounding box, (x min ,y min ) is the coordinate of the upper left corner of the bounding box, (x max ,y max ) is the lower right corner coordinate of the bounding box, and all extracted features are combined into a feature vector: Among them, f pixels is the pixel intensity value feature, h C is the color distribution feature, Area is the area of ​​the bounding box, and represents the size feature.

5. The method of a deep learning-based online car-hailing vehicle detection and classification system according to claim 1, characterized in that: In step S3, multiple groups of vehicle image data are generated using data enhancement technology based on the existing vehicle image data, and model training is performed using convolutional neural network and recurrent neural network algorithms to optimize the classification effect. The specific steps are as follows: Step C1, data enhancement: rotating, flipping, scaling, changing brightness and contrast of existing vehicle image data to generate multiple groups of vehicle image samples; Step C2, model training: Use the training set to train the convolutional neural network model, input the feature vector X into the convolutional neural network, extract and classify features through multiple layers of convolution, pooling and fully connected layers, and use the cross entropy loss function to optimize the model. When the input image is sequence data, use the recurrent neural network to capture the temporal dependency: h t =f(W h h t-1 +W x X), where h t and h t-1 are the current and previous hidden states, W h is the weight matrix from hidden state to hidden state, W x is the weight matrix from the current input feature vector to the hidden state, X is the current input feature vector, t represents the time step, and the performance of the model is evaluated on the test set.

6. The method of a deep learning-based online car-hailing vehicle detection and classification system according to claim 5, characterized in that: In step C2, the mathematical expression of the model optimized using the cross entropy loss function is: Among them, L is the loss function, y is the true label, is the predicted label, λ is the number of categories, y q is the label of the qth category, is the predicted probability of the qth category.

7. A device for a vehicle detection and classification system for online ride-hailing vehicles based on deep learning is applied to a method for a vehicle detection and classification system for online ride-hailing vehicles based on deep learning as described in any one of claims 1-6, characterized in that: Including data acquisition module, data processing module, and model training module; Data collection module: randomly selects online car-hailing image data from the online database, including vehicle images of different models, different angles and different lighting conditions, and labels the corresponding vehicle types; Data processing module: preprocesses and performs feature engineering on the collected data to generate training sets, test sets, and validation sets; Model training module: Based on the existing vehicle image data, data enhancement technology is used to generate multiple groups of vehicle image data, and convolutional neural network and recurrent neural network algorithms are used for model training to optimize the classification effect.

Images