Vehicle type identification method and terminal

Abstract

The present invention discloses a vehicle type recognition method and terminal, which recognizes the vehicle type of a vehicle to be recognized in an image to be recognized based on a trained vehicle type recognition model, obtains the vehicle type recognition results with the top N similarities from the vehicle type recognition model as a preliminary vehicle type set when the first vehicle type does not exceed a first threshold, obtains the 3D vehicle body model corresponding to each vehicle type in the preliminary vehicle type set, and restores the real scene to obtain a three-dimensional vehicle type simulation scene corresponding to each vehicle type in the preliminary vehicle type set; then controls the simulation camera in the three-dimensional vehicle type simulation scene corresponding to each vehicle type to shoot according to the shooting direction information to obtain a vehicle scene simulation image of each vehicle type; then retrains a vehicle type recognition sub-model for secondary recognition and outputs it. The present invention can improve the accuracy of vehicle type recognition results.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle identification, and in particular to a vehicle type identification method and a terminal. Background Art

[0002] Vehicle type recognition is a fine-grained classification problem in computer vision and is also an important research direction for intelligent traffic monitoring systems. It requires automatic analysis and recognition of vehicles in the monitoring screen, including vehicle manufacturers, vehicle series, and other fine categories. The most reliable and effective method is to include a clear and complete vehicle body image in the vehicle image. For example, many vehicles have important vehicle information such as the vehicle manufacturer's mark, vehicle series mark, and license plate at the rear. If such important vehicle information can be obtained, the vehicle model can be quickly and accurately identified.

[0003] At present, the algorithm for realizing vehicle model recognition is basically a deep convolutional neural network based on deep learning. However, the accuracy of deep learning is often closely related to the amount of training data and the accuracy of annotation. Not to mention that there are tens of thousands of automobile manufacturers and car series, and there are countless pictures of vehicles of different models in different application scenarios. As training data, it is impossible to cover and there is no time and energy to cover the vehicle images of all models in different application scenarios. Therefore, the existing deep learning vehicle model recognition often makes improvements on the specific implementation of the algorithm model in the hope of achieving higher recognition accuracy.

[0004] However, in actual application scenarios, the complexity of the application scenarios cannot be fully estimated, and the vehicle images are often not clear and complete body images. For example, there is no important vehicle information, and even the body parts are incomplete. This brings great difficulties to vehicle model recognition and affects the accuracy of vehicle model recognition. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a vehicle type recognition method and a terminal to improve the accuracy of vehicle type recognition.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] The vehicle type identification method comprises the following steps:

[0008] S1, based on the trained vehicle type recognition model, the vehicle to be recognized in the image to be recognized is recognized, and a first vehicle type is obtained, and it is determined whether the similarity of the first vehicle type in the vehicle type recognition model exceeds a first threshold value. If so, the first vehicle type is output as the vehicle type result of the image to be recognized. Otherwise, step S2 is executed, and the vehicle type recognition model is a deep convolutional neural network model.

[0009] S2. Obtaining the top N vehicle model recognition results in similarity from the vehicle model recognition model as a preliminary vehicle model set, and obtaining a 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set, where N is a positive integer greater than or equal to 2;

[0010] S3, obtaining a three-dimensional initial simulation scene and current environmental parameters corresponding to the real-time shooting scene, determining the specific position of the vehicle to be identified in the real-time shooting scene according to the image to be identified, and obtaining the vehicle position, wherein the real-time shooting scene is the actual scene where the vehicle to be identified is located, and the current environmental parameters include current light intensity, current light position and current visibility;

[0011] S4, simulating an environment corresponding to the real-time shooting scene in the three-dimensional initial simulation scene according to the current environmental parameters and the image environmental information other than the vehicle to be identified in the image to be identified, to obtain a three-dimensional environmental simulation scene, and placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environmental simulation scene, to obtain a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set;

[0012] S5, obtaining shooting position information and shooting direction information of the first camera that shoots the image to be identified in the real-time shooting scene, generating a simulated camera on the shooting position information corresponding to the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, controlling the simulated camera in the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set to shoot according to the shooting direction information, and obtaining a vehicle scene simulation image of each vehicle model in the preliminary vehicle model set;

[0013] S6. Use the original training image of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation image as new training data to train and generate a vehicle model recognition sub-model, perform vehicle model recognition on the vehicle to be recognized in the image to be recognized based on the trained vehicle model recognition sub-model to obtain a second vehicle model, and output the second vehicle model as the vehicle model result of the vehicle image to be recognized, where the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

[0014] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0015] The vehicle type identification terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vehicle type identification method described above is implemented.

[0016] The beneficial effects of the present invention are: the vehicle model recognition method and terminal, based on the trained vehicle model recognition model, perform vehicle model recognition on the vehicle to be recognized in the recognition image to output the vehicle model recognition effect. At this time, if the similarity of the output vehicle model result does not exceed the first threshold, it is considered that it is not very certain to be the vehicle model. Therefore, the vehicle model recognition results ranked in the top N in similarity are selected to establish a 3D vehicle body model, and a three-dimensional environment simulation scene is obtained according to the three-dimensional initial simulation scene and the current environmental parameters, and the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set is placed on the vehicle position in the three-dimensional environment simulation scene, and the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set is obtained, and then the similar vehicle models are simulated and photographed to obtain the vehicle scene simulation image of each vehicle model under the same scene and the same shooting conditions, and then used as new training data to train and generate a vehicle model recognition sub-model to obtain the final vehicle model recognition result. Therefore, the accuracy of the vehicle model results exceeding the first threshold is high, while the vehicle model results with low accuracy are subject to secondary recognition. Since a large number of vehicle models are narrowed down to a few models, the secondary recognition model is only trained for these models. It is highly targeted and requires a small amount of training, so that a relatively accurate vehicle model recognition sub-model that distinguishes these models can be quickly obtained to ensure the accuracy of the secondary recognition, thereby further improving the accuracy of the vehicle model recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of a vehicle type identification method according to an embodiment of the present invention;

[0018] Figure 2 Schematic diagram of the structure of a vehicle type identification terminal according to an embodiment of the present invention.

[0019] Description of labels:

[0020] 1. Vehicle type identification terminal; 2. Processor; 3. Memory. DETAILED DESCRIPTION

[0021] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0022] Please refer to Figure 1 , the vehicle type recognition method includes the following steps:

[0023] S1, based on the trained vehicle type recognition model, the vehicle to be recognized in the image to be recognized is recognized, and a first vehicle type is obtained, and it is determined whether the similarity of the first vehicle type in the vehicle type recognition model exceeds a first threshold value. If so, the first vehicle type is output as the vehicle type result of the image to be recognized. Otherwise, step S2 is executed, and the vehicle type recognition model is a deep convolutional neural network model.

[0024] S2. Obtaining the top N vehicle model recognition results in similarity from the vehicle model recognition model as a preliminary vehicle model set, and obtaining a 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set, where N is a positive integer greater than or equal to 2;

[0025] S3, obtaining a three-dimensional initial simulation scene and current environmental parameters corresponding to the real-time shooting scene, determining the specific position of the vehicle to be identified in the real-time shooting scene according to the image to be identified, and obtaining the vehicle position, wherein the real-time shooting scene is the actual scene where the vehicle to be identified is located, and the current environmental parameters include current light intensity, current light position and current visibility;

[0026] S4, simulating an environment corresponding to the real-time shooting scene in the three-dimensional initial simulation scene according to the current environmental parameters and the image environmental information other than the vehicle to be identified in the image to be identified, to obtain a three-dimensional environmental simulation scene, and placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environmental simulation scene, to obtain a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set;

[0027] S5, obtaining shooting position information and shooting direction information of the first camera that shoots the image to be identified in the real-time shooting scene, generating a simulated camera on the shooting position information corresponding to the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, controlling the simulated camera in the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set to shoot according to the shooting direction information, and obtaining a vehicle scene simulation image of each vehicle model in the preliminary vehicle model set;

[0028] S6. Use the original training image of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation image as new training data to train and generate a vehicle model recognition sub-model, perform vehicle model recognition on the vehicle to be recognized in the image to be recognized based on the trained vehicle model recognition sub-model to obtain a second vehicle model, and output the second vehicle model as the vehicle model result of the vehicle image to be recognized, where the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

[0029] From the above description, it can be seen that the beneficial effect of the present invention is that: based on the trained vehicle model recognition model, the vehicle to be identified in the image to be identified is identified to output the vehicle model recognition effect. At this time, if the similarity of the output vehicle model result does not exceed the first threshold, it is considered that it is not very certain to be the vehicle model. Therefore, the vehicle model recognition results ranked in the top N in similarity are selected to establish a 3D body model, and a three-dimensional environment simulation scene is obtained according to the three-dimensional initial simulation scene and the current environmental parameters, and the 3D body model corresponding to each vehicle model in the preliminary vehicle model set is placed on the vehicle position in the three-dimensional environment simulation scene, and the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set is obtained, and then the similar vehicle models are simulated and photographed to obtain the vehicle scene simulation image of each vehicle model under the same scene and the same shooting conditions, and then used as new training data to train and generate a vehicle model recognition sub-model to obtain the final vehicle model recognition result. Therefore, the accuracy of the vehicle model results exceeding the first threshold is high, while the vehicle model results with low accuracy are subject to secondary recognition. Since a large number of vehicle models are narrowed down to a few models, the secondary recognition model is only trained for these models. It is highly targeted and requires a small amount of training, so that a relatively accurate vehicle model recognition sub-model that distinguishes these models can be quickly obtained to ensure the accuracy of the secondary recognition, thereby further improving the accuracy of the vehicle model recognition results.

[0030] Furthermore, the step S4 specifically includes the following steps:

[0031] S41, acquiring a plurality of real-time shooting images taken by a plurality of cameras in the real-time shooting scene within a second preset time interval at intervals of a first preset time, obtaining differentiated static object information of the actual scene according to the plurality of real-time shooting images, and updating the three-dimensional initial simulation scene in real time according to the differentiated static object information to obtain an updated three-dimensional initial simulation scene;

[0032] S42, simulating an environment corresponding to the real-time shooting scene in the updated three-dimensional initial simulation scene according to the current environment parameters and the image environment information other than the vehicle to be identified in the image to be identified, to obtain a three-dimensional environment simulation scene, placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environment simulation scene, to obtain a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, wherein the image environment information includes living body information;

[0033] The step S5 is replaced by the following steps:

[0034] Generate multiple simulated cameras aimed at the 3D vehicle body model at different angles in the three-dimensional vehicle body simulation scene corresponding to each vehicle body in the preliminary vehicle body set, so as to obtain vehicle scene simulation images taken by the multiple simulated cameras at different angles on the 3D vehicle body model, and mark each of the obtained vehicle scene simulation images with a corresponding vehicle model mark;

[0035] The step S6 is replaced by the following steps:

[0036] The simulated vehicle scene images taken by the multiple simulated cameras at different angles for each vehicle model in the preliminary vehicle model set are used as new training data to train and generate a vehicle model recognition sub-model, and the vehicle to be identified in the image to be identified is identified based on the trained vehicle model recognition sub-model to obtain a second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified, and the second vehicle model is one of the models in the preliminary vehicle model set.

[0037] From the above description, it can be seen that for the vehicle model recognition sub-model of the deep convolutional neural network model, the accuracy of its training data directly affects the accuracy of the vehicle model recognition sub-model. At present, the training data is basically pictures obtained from the Internet, and the backgrounds of different vehicles are very different, which cannot be matched with the actual application scene. Through the above implementation, 3D body models of multiple similar vehicle models are placed in a three-dimensional initial simulation scene under the same real scene, and multiple vehicle scene simulation images of each vehicle model are obtained by shooting with cameras at different angles, so as to effectively increase the training picture data corresponding to a vehicle model, so that the pictures used as training data are more in line with the real scene and have a sufficient number of training, so that the vehicle model recognition accuracy of the obtained vehicle model recognition sub-model is higher.

[0038] Furthermore, before step S42, the following steps are also included:

[0039] Analyzing the body local information of the to-be-recognized vehicle appearing in the to-be-recognized image, and performing three-dimensional processing on the 3D body model corresponding to each vehicle model in the preliminary vehicle model set to obtain a 3D body local model with only the body local information remaining;

[0040] The 3D vehicle body model in step S42 and step S5 is a partial model of the 3D vehicle body.

[0041] From the above description, it can be seen that only the parts of the car body that appear in the image to be identified are included. This is because only the parts of the car body that appear in the image to be identified are identified. Therefore, the remaining parts of the car body that are not in the image to be identified are omitted to ensure that the feature data of the trained vehicle model identification sub-model is more concentrated on the vehicle body part. On the one hand, the amount of calculation is reduced, and on the other hand, the accuracy is improved, so that the vehicle model identification sub-model that distinguishes these types of vehicles can be quickly and accurately obtained.

[0042] Further, the step S6 is replaced by the following steps:

[0043] The vehicle scene simulation image of each vehicle model in the preliminary vehicle model set is input into the vehicle model recognition model as new training data to obtain a new vehicle model recognition model, and the vehicle model to be identified in the image to be identified is identified based on the new vehicle model recognition model to obtain a second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified, and the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

[0044] From the above description, it can be seen that the vehicle scene simulation images are used as new training data to update the vehicle model recognition model. Since the training data includes vehicle scene simulation images of different vehicle models in real scenes, the entire recognition model will correct its internal parameters according to the new training data to ensure that the same basic vehicle model recognition model has targeted model parameter adjustments when applied to different scenes. Compared with the homogenized vehicle model recognition model, the updated vehicle model recognition model has higher accuracy in the corresponding real scenes because it is further trained for real scenes.

[0045] Furthermore, the step S6 specifically includes the following steps:

[0046] Using the original training image of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation image as new training data to train and generate a vehicle model recognition sub-model;

[0047] Based on the trained vehicle type recognition sub-model, the vehicle to be recognized in the image to be recognized is recognized by the vehicle type recognition, and the similarity of each vehicle type in the preliminary vehicle type set is obtained; and the final similarity of each vehicle type in the preliminary vehicle type set is obtained according to the similarity of each vehicle type in the preliminary vehicle type set obtained in the vehicle type recognition model, the similarity obtained in the preliminary vehicle type set, and the preset weight values ​​corresponding to the two similarities;

[0048] The vehicle model with the highest final similarity is taken as the second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified, and the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

[0049] From the above description, it can be seen that compared with directly using the recognition effect of the vehicle model recognition sub-model as the final vehicle model recognition result, since the training data of the vehicle model recognition sub-model is much smaller than the vehicle model recognition model, its stability is relatively low. Therefore, by neutralizing the results of the two models according to the preset weights, the stability of the vehicle model recognition sub-model can be corrected, thereby ensuring that the final vehicle model recognition result has both higher accuracy and stability.

[0050] Furthermore, N is a preset number or the number of all vehicle type recognition results in the vehicle type recognition model whose similarity is greater than a second threshold.

[0051] The N is the number of all vehicle type recognition results in the vehicle type recognition model whose similarity is greater than the highest similarity*preset percentage and does not exceed the preset number.

[0052] From the above description, it can be seen that a variety of settings for N are provided, including preset numbers or preset thresholds. Relatively speaking, the threshold is set with the highest similarity * preset percentage to consider whether the differences between different models are within a certain range, and to continue the next step of comparison. At the same time, a certain preset number is limited to limit the subsequent calculation amount. Therefore, the preliminary model set provided can achieve a relative balance between accuracy and calculation amount.

[0053] Furthermore, obtaining the three-dimensional environment simulation scene in step S4 specifically includes the following steps:

[0054] A simulated light source is generated at the current lighting position of the three-dimensional initial simulation scene, and the simulated light source is made to emit light according to the current lighting intensity, and the visibility of the three-dimensional initial simulation scene is made to be the current visibility, so as to obtain a three-dimensional environment simulation scene.

[0055] From the above description, we can know that the training data of the neural network model is based on the distinction of feature data within a certain range. If it is a complete car body, the entire complete car body is considered. In real scenes, the effective information of the car body will be greatly weakened by the influence of visibility and lighting. At this time, if the model trained with the original clear and complete image is still used for recognition, the distinction between different models based on different effective information may be affected, resulting in inaccurate output model recognition results. The model that takes visibility and lighting into consideration and is trained as training data focuses on visible effective information to distinguish different models. It has more effective information, making the obtained model recognition results more accurate and reliable.

[0056] Furthermore, after placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environment simulation scene in step S5, the following steps are also included:

[0057] The 3D vehicle body model is subjected to light processing according to the light emission in the three-dimensional environment simulation scene.

[0058] From the above description, we can know that light processing is refraction, reflection and reflection. Sometimes the subtle differences in vehicle models will have very different effects under the same lighting conditions, such as occlusions, shadows, etc., so as to ensure that the vehicle scene simulation image used for training can be closer to the image to be recognized, so that the vehicle model recognition sub-model trained by this method has higher accuracy.

[0059] Please refer to Figure 2 The vehicle type identification terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned vehicle type identification method is implemented.

[0060] From the above description, it can be seen that the beneficial effect of the present invention is that: based on the trained vehicle model recognition model, the vehicle to be identified in the image to be identified is identified to output the vehicle model recognition effect. At this time, if the similarity of the output vehicle model result does not exceed the first threshold, it is considered that it is not very certain to be the vehicle model. Therefore, the vehicle model recognition results ranked in the top N in similarity are selected to establish a 3D body model, and a three-dimensional environment simulation scene is obtained according to the three-dimensional initial simulation scene and the current environmental parameters, and the 3D body model corresponding to each vehicle model in the preliminary vehicle model set is placed on the vehicle position in the three-dimensional environment simulation scene, and the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set is obtained, and then the similar vehicle models are simulated and photographed to obtain the vehicle scene simulation image of each vehicle model under the same scene and the same shooting conditions, and then used as new training data to train and generate a vehicle model recognition sub-model to obtain the final vehicle model recognition result. Therefore, the accuracy of the vehicle model results exceeding the first threshold is high, while the vehicle model results with low accuracy are subject to secondary recognition. Since a large number of vehicle models are narrowed down to a few models, the secondary recognition model is only trained for these models. It is highly targeted and requires a small amount of training, so that a relatively accurate vehicle model recognition sub-model that distinguishes these models can be quickly obtained to ensure the accuracy of the secondary recognition, thereby further improving the accuracy of the vehicle model recognition results.

[0061] Please refer to Figure 1 , Embodiment 1 of the present invention is:

[0062] The vehicle type identification method comprises the following steps:

[0063] S1, based on the trained vehicle model recognition model, the vehicle to be recognized in the image to be recognized is recognized, and a first vehicle model is obtained, and it is determined whether the similarity of the first vehicle model obtained in the vehicle model recognition model exceeds a first threshold value. If so, the first vehicle model is output as the vehicle model result of the image to be recognized. Otherwise, step S2 is executed, and the vehicle model recognition model is a deep convolutional neural network model;

[0064] The existing model can be used based on the basic model in step S1, that is, the initial vehicle model recognition model. The first threshold is 98%. In other equivalent embodiments, the first threshold is in the range of [95%, 99%] to ensure accuracy.

[0065] S2. Obtain the top N vehicle model recognition results with the highest similarity from the vehicle model recognition model as a preliminary vehicle model set, and obtain the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set, where N is a positive integer greater than or equal to 2;

[0066] In this embodiment, N is the number of all vehicle identification results in the vehicle identification model whose similarity is greater than the highest similarity * 90% and does not exceed five. For example, if the highest similarity is only 95%, it is less than the first threshold value of 98%. At this time, if there are six vehicle models in the vehicle identification model whose similarity is greater than 95% * 90% = 85.5%, then the first five can be taken. If there are only four, then four can be taken, and so on. In other equivalent embodiments, the interval of the preset percentage is [85%, 95%], and the interval of the preset number is [3, 10]. Of course, in other equivalent embodiments, N is the preset number or the number of all vehicle identification results in the vehicle identification model whose similarity is greater than the second threshold value. For example, if N is the preset number, it is set to 5, and the first five vehicle models will be obtained. For the second threshold value, for example, the second threshold value is 80%, then the number of all vehicle models with a similarity greater than 80% may be three or many.

[0067] Of course, if the number of similarities greater than the highest similarity * preset percentage is 0, then when there is only the highest similarity model result, the second similarity can be obtained as a model in the preliminary model set to ensure that there are at least two models for subsequent secondary identification.

[0068] S3, obtaining a three-dimensional initial simulation scene and current environmental parameters corresponding to the real-time shooting scene, determining the specific position of the vehicle to be identified in the real-time shooting scene according to the image to be identified, and obtaining the vehicle position, the real-time shooting scene is the actual scene where the vehicle to be identified is located, and the current environmental parameters include current light intensity, current light position and current visibility;

[0069] Among them, obtaining the current light intensity, the current light position and the current visibility, and determining the vehicle position in the image are all existing technologies. The present application is to obtain this information for subsequent scene simulation, so it does not further describe how to obtain the current light intensity, the current light position and the current visibility, and determine the vehicle position in the image.

[0070] In addition, the 3D body model can be simulated and established when needed. This is only necessary when a high degree of accuracy is required for vehicle type recognition. In this case, a corresponding 3D body model needs to be established to execute the following steps. Not all vehicle type recognitions require the establishment of a 3D body model. That is, if a high degree of accuracy is not required for vehicle type recognition, it is usually sufficient to output the first vehicle type in step S1.

[0071] S4, simulating an environment corresponding to the real-time shooting scene in the three-dimensional initial simulation scene according to the current environmental parameters and the image environmental information other than the vehicle to be identified in the image to be identified, obtaining a three-dimensional environmental simulation scene, placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environmental simulation scene, and obtaining a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set;

[0072] In this embodiment, obtaining the three-dimensional environment simulation scene in step S4 specifically includes the following steps:

[0073] A simulated light source is generated at the current lighting position of the three-dimensional initial simulation scene, and the simulated light source is made to emit light according to the current lighting intensity, and at the same time, the visibility of the three-dimensional initial simulation scene is made to be the current visibility, so as to obtain a three-dimensional environment simulation scene.

[0074] S5, obtaining shooting position information and shooting direction information of the first camera shooting the image to be identified in the real-time shooting scene, generating a simulated camera at the shooting position information corresponding to the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, controlling the simulated camera in the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set to shoot according to the shooting direction information, and obtaining a vehicle scene simulation image of each vehicle model in the preliminary vehicle model set;

[0075] Among them, after placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environment simulation scene in step S5, the following steps are also included:

[0076] The 3D vehicle body model is light processed according to the luminescence in the three-dimensional environment simulation scene.

[0077] That is, in this embodiment, visibility and lighting are taken into consideration together and used as training data to obtain a model that focuses on visible effective information to distinguish different types of vehicles. It has more effective information, making the obtained vehicle type recognition results more accurate and reliable.

[0078] For example, different curvature treatments on the vehicle body lead to differences in refracted and reflected light, slight deviations in size lead to projection differences between different models, and similar situations in which effective information is increased or reduced due to the influence of light. These situations can be taken into account by the vehicle identification sub-model, so that the vehicle identification sub-model obtained in step S6 is more realistic and accurate.

[0079] S6. Use the original training images of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation images as new training data to train and generate a vehicle model recognition sub-model. Perform vehicle model recognition on the vehicle to be recognized in the image to be recognized based on the trained vehicle model recognition sub-model to obtain a second vehicle model. Output the second vehicle model as the vehicle model result of the vehicle image to be recognized. The second vehicle model is one of the models in the preliminary vehicle model set.

[0080] In this embodiment, considering the problem of the amount of training data of the vehicle type identification sub-model, the stability of the result may be poor, so step S6 specifically includes the following steps:

[0081] The original training images of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation images are used as new training data to train and generate a vehicle model recognition sub-model;

[0082] Based on the trained vehicle type recognition sub-model, the vehicle to be recognized in the recognition image is recognized, and the similarity of each vehicle type in the preliminary vehicle type set is obtained. The final similarity of each vehicle type in the preliminary vehicle type set is obtained according to the similarity of each vehicle type in the preliminary vehicle type set obtained in the vehicle type recognition model, the similarity obtained in the preliminary vehicle type set, and the preset weight values ​​corresponding to the two similarities;

[0083] The vehicle model with the highest final similarity is taken as the second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified. The second vehicle model is one of the models in the preliminary vehicle model set.

[0084] Therefore, for the vehicle model recognition model, if the similarity of a certain vehicle model is very high, it can be determined to be that vehicle model. If the similarity of multiple vehicle models is high and close, only a possible result is provided. Therefore, secondary targeted vehicle model recognition is performed in real scenarios to improve the accuracy of vehicle model recognition and greatly reduce the misjudgment rate of vehicle models.

[0085] In other equivalent embodiments, step S6 is replaced by the following steps:

[0086] The vehicle scene simulation image of each vehicle model in the preliminary vehicle model set is input into the vehicle model recognition model as new training data to obtain a new vehicle model recognition model, and the vehicle to be recognized in the image to be recognized is recognized based on the new vehicle model recognition model to obtain a second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be recognized. The second vehicle model is one of the models in the preliminary vehicle model set.

[0087] That is, when applied to different scenarios, the same basic vehicle model recognition model is adjusted in a targeted manner. Compared with the homogeneous vehicle model recognition model, the updated vehicle model recognition model has higher accuracy in the corresponding real scenarios because it is further trained for real scenarios.

[0088] Please refer to Figure 1 , Embodiment 2 of the present invention is:

[0089] The vehicle type identification method, based on the above embodiment 1, step S4 specifically includes the following steps:

[0090] S41, acquiring a plurality of real-time shooting images taken by a plurality of cameras in a real-time shooting scene within a second preset time interval at intervals of a first preset time, obtaining differentiated static object information of an actual scene according to the plurality of real-time shooting images, and updating the three-dimensional initial simulation scene in real time according to the differentiated static object information to obtain an updated three-dimensional initial simulation scene;

[0091] In the above step S41, the first preset time can be one day, and the second preset time interval can be any 30-second time interval between sunrise and sunset. In this way, it can quickly adapt to changes in the real scene. For higher real-time response, the first preset time can be one hour, etc.

[0092] S411, analyzing the body local information of the to-be-recognized vehicle appearing in the to-be-recognized image, and performing three-dimensional processing on the 3D body model corresponding to each vehicle model in the preliminary vehicle model set to obtain a 3D body local model with only the body local information remaining;

[0093] For example, if the local body information only includes the front of the car, the subsequent 3D body model only needs to display the front of the car. Of course, the local body information in this embodiment does not mean that the scope of division is the front, body, rear, etc. It should be known that the local body information is determined by the position ratio relationship appearing in the image to be identified, that is, the local body information is limited by the proportion compared to the body. For example, considering that the critical position of the body on the camera side is the front door handle, and the critical position of the body on the side away from the camera is the critical point between the front of the car and the front door, according to the image to be identified, the critical positions of the body on both sides are connected to divide the entire body into two parts, one of which is determined as the local body information based on whether it is in the image to be identified.

[0094] S42, simulating an environment corresponding to the real-time shooting scene in the updated three-dimensional initial simulation scene according to the current environmental parameters and the image environmental information other than the vehicle to be identified in the image to be identified, to obtain a three-dimensional environmental simulation scene, placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environmental simulation scene, to obtain a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, wherein the image environmental information includes living body information;

[0095] Among them, the living body information is unpredictable during the training process, because the updated three-dimensional initial simulation scene only includes still objects updated in real time, while in the recognition process, living body information inevitably exists. Therefore, before obtaining the three-dimensional environment simulation scene, it also includes identifying the living body information to simulate and generate a three-dimensional living body information based on the two-dimensional living body information in the picture, and then add the three-dimensional living body information to the three-dimensional environment simulation scene to ensure a high degree of restoration in different scenes.

[0096] In this embodiment, step S5 is replaced by the following steps:

[0097] Generate multiple simulated cameras aimed at the 3D vehicle body model at different angles in the three-dimensional vehicle body simulation scene corresponding to each vehicle model in the preliminary vehicle model set, so as to obtain vehicle scene simulation images taken by the multiple simulated cameras at different angles on the 3D vehicle body model, and mark each obtained vehicle scene simulation image with a corresponding vehicle model mark;

[0098] Among them, the 3D body model in step S42 and step S5 is a partial 3D body model. In this way, even if there are only a few car models, each car model has multiple images at different angles, which ensures the amount of training data. At the same time, no manual labeling is required, so multiple labeled training pictures can be obtained quickly and accurately.

[0099] In this embodiment, step S6 is replaced by the following steps:

[0100] The simulated vehicle scene images taken by multiple simulated cameras at different angles for each vehicle model in the preliminary vehicle model set are used as new training data to train and generate a vehicle model recognition sub-model. The vehicle to be recognized in the image to be recognized is recognized based on the trained vehicle model recognition sub-model to obtain a second vehicle model. The second vehicle model is output as the vehicle model result of the vehicle image to be recognized. The second vehicle model is one of the models in the preliminary vehicle model set.

[0101] Therefore, this embodiment places 3D body partial models of multiple similar vehicle models in a three-dimensional initial simulation scene under the same real scene, and obtains multiple vehicle scene simulation images of each vehicle model through cameras at different angles, so as to effectively increase the training picture data corresponding to a vehicle model, so that the pictures used as training data are more in line with the real scene and have a sufficient number of training, so that the vehicle model recognition sub-model obtained has higher vehicle model recognition accuracy.

[0102] Please refer to Figure 1 , Embodiment 3 of the present invention is:

[0103] The vehicle model identification method, based on the above-mentioned embodiment 1, in this embodiment, further includes the following steps before step S1:

[0104] S01, obtaining multiple test vehicle images taken at different angles of preset common vehicle models in a local database;

[0105] S02, adjusting the output result of the vehicle type recognition model to output the top N vehicle types with the highest similarity, where N is set with reference to the definition in the first embodiment;

[0106] S03, inputting each test vehicle picture of each preset common vehicle model into the adjusted vehicle model recognition model to obtain the top N vehicle models of each test vehicle picture of each preset common vehicle model;

[0107] S04, summarizing the top N models of all test vehicle images of the same preset common model to obtain similar models of each preset common model;

[0108] S05, determining whether there is a similar model to the preset common model, if so, pre-establishing a 3D body model of the preset common model and its corresponding similar model, if not, ignoring the preset common model;

[0109] S06. Repeat the above step S05 for each preset common vehicle model to obtain a 3D model database including all 3D vehicle body models in step S05.

[0110] At this time, step S2 specifically includes the following steps:

[0111] The first N vehicle model recognition results with similarity ranking are obtained from the vehicle model recognition model as a preliminary vehicle model set, and it is determined whether the 3D body model of each vehicle model in the preliminary vehicle model set is in the 3D model database. If so, the 3D body model of the corresponding vehicle model is obtained, otherwise the 3D body model is re-established, and N is a positive integer greater than or equal to 2.

[0112] Therefore, in the above step S01, the commonly used car models preset may be the car models ranked by sales volume on the market. In a preferred implementation, several car models with similar models may be obtained through solicitation or other channels as the preset commonly used car models.

[0113] Therefore, in step S2 , all reconstructed 3D vehicle body models are stored in the 3D model database.

[0114] Please refer to Figure 2 , Embodiment 4 of the present invention is:

[0115] The vehicle type identification terminal 1 includes a memory 3, a processor 2, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, the steps of the above-mentioned embodiment 1 are implemented.

[0116] In summary, the vehicle type recognition method and terminal provided by the present invention recognizes the vehicle type in the image to be recognized based on the trained vehicle type recognition model, and selects multiple vehicle type recognition results to establish a 3D vehicle body partial model in the case of uncertainty, and restores multiple vehicle scene simulation images at different angles according to the three-dimensional initial simulation scene and current environmental parameters, and then retrains to generate a vehicle type recognition sub-model or updates the vehicle type recognition model to obtain the final vehicle type recognition result. This further improves the accuracy of the vehicle type recognition result.

[0117] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A vehicle type identification method, characterized in that: The following steps are involved: S1, based on the trained vehicle type recognition model, the vehicle to be recognized in the image to be recognized is recognized, and a first vehicle type is obtained, and it is determined whether the similarity of the first vehicle type in the vehicle type recognition model exceeds a first threshold value. If so, the first vehicle type is output as the vehicle type result of the image to be recognized. Otherwise, step S2 is executed, and the vehicle type recognition model is a deep convolutional neural network model. S2. Obtaining the top N vehicle model recognition results in similarity from the vehicle model recognition model as a preliminary vehicle model set, and obtaining a 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set, where N is a positive integer greater than or equal to 2; S3, obtaining a three-dimensional initial simulation scene and current environmental parameters corresponding to the real-time shooting scene, determining the specific position of the vehicle to be identified in the real-time shooting scene according to the image to be identified, and obtaining the vehicle position, wherein the real-time shooting scene is the actual scene where the vehicle to be identified is located, and the current environmental parameters include current light intensity, current light position and current visibility; S4, simulating an environment corresponding to the real-time shooting scene in the three-dimensional initial simulation scene according to the current environmental parameters and the image environmental information other than the vehicle to be identified in the image to be identified, to obtain a three-dimensional environmental simulation scene, and placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environmental simulation scene, to obtain a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set; S5, obtaining shooting position information and shooting direction information of the first camera that shoots the image to be identified in the real-time shooting scene, generating a simulated camera on the shooting position information corresponding to the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, controlling the simulated camera in the three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set to shoot according to the shooting direction information, and obtaining a vehicle scene simulation image of each vehicle model in the preliminary vehicle model set; S6. Use the original training image of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation image as new training data to train and generate a vehicle model recognition sub-model, perform vehicle model recognition on the vehicle to be recognized in the image to be recognized based on the trained vehicle model recognition sub-model to obtain a second vehicle model, and output the second vehicle model as the vehicle model result of the vehicle image to be recognized, where the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

2. The vehicle type identification method according to claim 1, characterized in that: The step S4 specifically comprises the following steps: S41, acquiring a plurality of real-time shooting images taken by a plurality of cameras in the real-time shooting scene within a second preset time interval at intervals of a first preset time, obtaining differentiated static object information of the actual scene according to the plurality of real-time shooting images, and updating the three-dimensional initial simulation scene in real time according to the differentiated static object information to obtain an updated three-dimensional initial simulation scene; S42, simulating an environment corresponding to the real-time shooting scene in the updated three-dimensional initial simulation scene according to the current environment parameters and the image environment information other than the vehicle to be identified in the image to be identified, to obtain a three-dimensional environment simulation scene, placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environment simulation scene, to obtain a three-dimensional vehicle model simulation scene corresponding to each vehicle model in the preliminary vehicle model set, wherein the image environment information includes living body information; The step S5 is replaced by the following steps: Generate multiple simulated cameras aimed at the 3D vehicle body model at different angles in the three-dimensional vehicle body simulation scene corresponding to each vehicle body in the preliminary vehicle body set, so as to obtain vehicle scene simulation images taken by the multiple simulated cameras at different angles on the 3D vehicle body model, and mark each of the obtained vehicle scene simulation images with a corresponding vehicle model mark; The step S6 is replaced by the following steps: The simulated vehicle scene images taken by the multiple simulated cameras at different angles for each vehicle model in the preliminary vehicle model set are used as new training data to train and generate a vehicle model recognition sub-model, and the vehicle to be identified in the image to be identified is identified based on the trained vehicle model recognition sub-model to obtain a second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified, and the second vehicle model is one of the models in the preliminary vehicle model set.

3. The vehicle type identification method according to claim 2, characterized in that: The step S42 also includes the following steps: Analyzing the body local information of the to-be-recognized vehicle appearing in the to-be-recognized image, and performing three-dimensional processing on the 3D body model corresponding to each vehicle model in the preliminary vehicle model set to obtain a 3D body local model with only the body local information remaining; The 3D vehicle body model in step S42 and step S5 is a partial model of the 3D vehicle body.

4. The vehicle type identification method according to claim 2, characterized in that: The step S6 is replaced by the following steps: The vehicle scene simulation image of each vehicle model in the preliminary vehicle model set is input into the vehicle model recognition model as new training data to obtain a new vehicle model recognition model, and the vehicle model to be identified in the image to be identified is identified based on the new vehicle model recognition model to obtain a second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified, and the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

5. The vehicle type identification method according to claim 1, characterized in that: The step S6 specifically comprises the following steps: Using the original training image of each vehicle model in the preliminary vehicle model set and the corresponding vehicle scene simulation image as new training data to train and generate a vehicle model recognition sub-model; Based on the trained vehicle type recognition sub-model, the vehicle to be recognized in the image to be recognized is recognized by model recognition, and a second similarity of each vehicle type in the preliminary vehicle type set is obtained; and a final similarity of each vehicle type in the preliminary vehicle type set is obtained according to the similarity of each vehicle type in the preliminary vehicle type set obtained in the vehicle type recognition model, the second similarity obtained in the preliminary vehicle type set, and preset weight values ​​corresponding to the two similarities; The vehicle model with the highest final similarity is taken as the second vehicle model, and the second vehicle model is output as the vehicle model result of the vehicle image to be identified, and the second vehicle model is one of the vehicle models in the preliminary vehicle model set.

6. The vehicle type identification method according to claim 1, characterized in that: The N is a preset number or the number of all vehicle type recognition results in the vehicle type recognition model whose similarity is greater than a second threshold.

7. The vehicle type identification method according to claim 6, characterized in that: The N is the number of all vehicle type recognition results in the vehicle type recognition model whose similarity is greater than the highest similarity*preset percentage and does not exceed the preset number.

8. The vehicle type identification method according to claim 1, characterized in that: The step S4 of obtaining the three-dimensional environment simulation scene specifically includes the following steps: A simulated light source is generated at the current lighting position of the three-dimensional initial simulation scene, and the simulated light source is made to emit light according to the current lighting intensity, and the visibility of the three-dimensional initial simulation scene is made to be the current visibility, so as to obtain a three-dimensional environment simulation scene.

9. The vehicle type identification method according to claim 8, characterized in that: After placing the 3D vehicle body model corresponding to each vehicle model in the preliminary vehicle model set at the vehicle position in the three-dimensional environment simulation scene in step S5, the following steps are also included: The 3D vehicle body model is subjected to light processing according to the light emission in the three-dimensional environment simulation scene.

10. A vehicle type identification terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the vehicle model identification method as described in any one of claims 1 to 9 is implemented.

Images