A vehicle recognition method, device, electronic device and storage medium

Abstract

The present application discloses a vehicle recognition method, device, electronic device and storage medium. The vehicle recognition method includes: obtaining a set of video images to be detected; for each frame of the video images to be detected in the set of video images to be detected, determining the clarity level of the video image to be detected according to the saturation of the pixel points of the video image to be detected; inputting the video image to be detected into an image clarity correction model corresponding to the clarity level to obtain corrected image feature information; inputting the corrected image feature information into a preset vehicle recognition model to obtain a vehicle recognition result of the video image to be detected; sequentially matching the vehicle recognition result of the current frame of the video image to be detected with the vehicle recognition result of the next frame of the video image to be detected to obtain a matching result, and generating a vehicle trajectory according to the matching result.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation, and in particular to a vehicle identification method, device, electronic device and storage medium. Background Art

[0002] Vehicle tracking is an important function in intelligent transportation. By identifying vehicles in the video captured by the camera, the vehicle's running trajectory can be obtained, which provides support for subsequent analysis of vehicle behavior, identification of abnormal vehicles, and statistics of vehicle flow. Vehicle tracking generally involves tracking vehicles across cameras, identifying the same vehicle in video images captured by different cameras and tracking it.

[0003] At present, vehicle tracking mainly relies on three subtasks: vehicle detection, vehicle identification and vehicle fusion. Among them, the commonly used vehicle identification method is the deep learning algorithm. The features extracted by the deep learning network model can better reflect the characteristics of the target vehicle. However, the existing vehicle identification method based on deep learning algorithm is mainly for vehicle identification under normal weather conditions, and is not applicable to vehicle identification under severe weather conditions (such as foggy days), and the identification accuracy is low. Summary of the invention

[0004] In order to solve the problems in the background technology, the embodiments of the present application provide a vehicle identification method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present application provides a vehicle identification method, comprising:

[0006] Obtain a set of video images to be detected;

[0007] For each frame of the video image to be detected in the set of video images to be detected, determining the clarity level of the video image to be detected according to the saturation of the pixel points of the video image to be detected;

[0008] Inputting the video image to be detected into the image definition correction model corresponding to the definition level to obtain corrected image feature information, wherein the image definition correction models corresponding to different definition levels are obtained by training a preset deep learning model based on each sample image in a set of sample images of corresponding definition levels;

[0009] Inputting the corrected image feature information into a preset vehicle recognition model to obtain a vehicle recognition result of the video image to be detected;

[0010] The vehicle recognition result of the current frame of the video image to be detected is matched with the vehicle recognition result of the next frame of the video image to be detected in sequence to obtain a matching result, and a vehicle trajectory is generated according to the matching result.

[0011] In a possible implementation, the vehicle recognition results of the current frame of the video image to be detected are sequentially matched with the vehicle recognition results of the next frame of the video image to be detected to obtain a matching result, which specifically includes:

[0012] Predict the current vehicle driving speed according to the clarity level of the current frame of the video image to be detected and a preset vehicle speed regression prediction model;

[0013] For each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected, determine the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected according to the vehicle driving speed;

[0014] Predict the position information of the vehicle feature in the next frame of the video image to be detected according to a preset filter and the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected;

[0015] Determine the target matching position interval of the vehicle feature in the next frame of the video image to be detected according to the predicted position information;

[0016] Match the vehicle feature with the vehicle features in the target matching position interval to obtain a matching result.

[0017] In a possible implementation, determine the clarity level of the video image to be detected according to the saturation of the pixel points of the video image to be detected, which specifically includes:

[0018] Determine the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in a preset reference image respectively, where the preset reference image is a clear image of the scene corresponding to the video image to be detected;

[0019] Perform histogram statistics on the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in the preset reference image respectively, and determine the number of pixel points whose saturation values fall into each of at least two preset saturation intervals in the video image to be detected and the preset reference image respectively;

[0020] Determine a histogram offset value according to the number of pixel points whose saturation values fall into each saturation interval in the video image to be detected and the number of pixel points whose saturation values fall into each saturation interval in the preset reference image;

[0021] Determine the clarity level of the video image to be detected according to the histogram offset value and preset clarity thresholds for each level.

[0022] In a possible implementation manner, a histogram offset value is determined according to the number of pixel points whose saturation values fall within each saturation interval in the video image to be detected and the number of pixel points whose saturation values fall within each saturation interval in the preset reference image. Specifically, it includes:

[0023] Calculate the histogram offset through the following formula:

[0024]

[0025] where, ΔHist represents the histogram offset value;

[0026] Hist_original i represents the number of pixel points whose saturation values fall within the i-th saturation interval in the preset reference image, and n represents the number of saturation intervals;

[0027] Hist_cur i represents the number of pixel points whose saturation values fall within the i-th saturation interval in the video image to be detected.

[0028] In a possible implementation manner, the preset filter includes a Kalman filter, and the preset vehicle speed regression prediction model is a linear regression model representing the linear relationship between the clarity level and the vehicle driving speed.

[0029] In a second aspect, an embodiment of the present application provides a vehicle recognition device, including:

[0030] An acquisition unit, configured to acquire a set of video images to be detected;

[0031] A determination unit, configured to determine the clarity level of each frame of the video image to be detected in the set of video images to be detected according to the saturation of the pixel points of the video image to be detected;

[0032] A correction unit, configured to input the video image to be detected into an image clarity correction model corresponding to the clarity level to obtain corrected image feature information, where the image clarity correction models corresponding to different clarity levels are trained by a preset deep learning model using each sample image in a set of sample images corresponding to the respective clarity levels;

[0033] An identification unit, configured to input the corrected image feature information into a preset vehicle recognition model to obtain a vehicle recognition result of the video image to be detected;

[0034] A generation unit, configured to sequentially match the vehicle recognition result of the current frame of the video image to be detected with the vehicle recognition result of the next frame of the video image to be detected to obtain a matching result, and generate a vehicle trajectory according to the matching result.

[0035] In a possible implementation, the generating unit is specifically configured to:

[0036] Predict the current vehicle driving speed according to the clarity level of the current frame of the video image to be detected and a preset vehicle speed regression prediction model;

[0037] For each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected, determine the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected according to the vehicle driving speed;

[0038] Predict the position information of the vehicle feature in the next frame of the video image to be detected according to a preset filter and the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected;

[0039] Determine the target matching position interval of the vehicle feature in the next frame of the video image to be detected according to the predicted position information;

[0040] Match the vehicle feature with the vehicle features in the target matching position interval to obtain a matching result.

[0041] In a possible implementation, the determining unit is specifically configured to:

[0042] Respectively determine the saturation values of each pixel point in the video image to be detected, and the saturation values of each pixel point in a preset reference image, where the preset reference image is a clear image of the scene corresponding to the video image to be detected;

[0043] Perform histogram statistics on the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in the preset reference image respectively, and respectively determine the number of pixel points in each saturation interval in which the saturation values in the video image to be detected and the preset reference image fall into at least two set saturation intervals;

[0044] Determine a histogram offset value according to the number of pixel points in each saturation interval in which the saturation values in the video image to be detected fall and the number of pixel points in each saturation interval in which the saturation values in the preset reference image fall;

[0045] Determine the clarity level of the video image to be detected according to the histogram offset value and preset clarity thresholds for each level.

[0046] In a possible implementation, the determining unit is specifically configured to calculate the histogram offset through the following formula:

[0047]

[0048] where ΔHist represents the histogram offset value;

[0049] Hist_original i represents the number of pixel points in the preset reference image whose saturation values fall within the i-th saturation interval, and n represents the number of saturation intervals;

[0050] Hist_cur i represents the number of pixel points in the video image to be detected whose saturation values fall within the i-th saturation interval.

[0051] In a possible implementation manner, the preset filter includes a Kalman filter, and the preset vehicle speed regression prediction model is a linear regression model characterizing the linear relationship between the clarity level and the vehicle driving speed.

[0052] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the vehicle recognition method described in the present application is implemented.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in the vehicle recognition method described in the present application are implemented.

[0054] The beneficial effects of the embodiments of the present application are as follows:

[0055] In the vehicle recognition method provided by the embodiments of the present application, a vehicle detection device obtains a set of video images to be detected. For each frame of the video image to be detected in the set of video images to be detected, the clarity level of the video image to be detected is determined according to the saturation of the pixel points of the video image to be detected. The video image to be detected is input into an image clarity correction model corresponding to the clarity level to obtain corrected image feature information. Among them, the image clarity correction models corresponding to different clarity levels are obtained by training a preset deep learning model with each sample image in the sample image set of the corresponding clarity level. Furthermore, the corrected image feature information is input into a preset vehicle recognition model to obtain the vehicle recognition result of the video image to be detected. The vehicle recognition result of the current frame of the video image to be detected and the vehicle recognition result of the next frame of the video image to be detected are sequentially matched to obtain a matching result, and a vehicle trajectory is generated according to the matching result. Compared with the prior art, in the embodiments of the present application, a preset deep learning is trained in advance with the sample images in the sample image sets of different clarity levels to obtain image clarity correction models corresponding to different clarity levels. After obtaining the set of video images to be detected, the clarity level of the video image to be detected is first determined, and the video image to be detected is clarity-corrected according to the image clarity correction model corresponding to the determined clarity level. After obtaining the corrected video image to be detected, vehicle recognition is performed on the corrected image feature information and a preset vehicle recognition model to obtain the vehicle recognition result of each frame of the video image to be detected. Furthermore, the vehicle recognition result of the current frame of the video image to be detected and the vehicle recognition result of the next frame of the video image to be detected are sequentially matched to obtain a matching result, and a vehicle trajectory is generated according to the matching result. In the present application, by detecting the clarity level of the video image to be detected and fusing the image clarity correction model corresponding to the clarity level to restore the image features, the risk of being unable to detect vehicles and track and recognize them due to unclear features and low feature discrimination under adverse weather conditions is reduced, and the accuracy of vehicle recognition and trajectory tracking is improved.

[0056] Other features and advantages of the present application will be described in the following specification, and part of them will become obvious from the specification or be understood by implementing the present application. The objectives and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the written specification, claims, and drawings. Description of the Drawings

[0057] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0058] Figure 1Schematic diagram of the implementation process of the vehicle recognition method provided by the embodiment of the present application;

[0059] Figure 2 Schematic diagram of the implementation process of determining the clarity level of the video image to be detected provided by the embodiment of the present application;

[0060] Figure 3 Schematic diagram of the implementation process of generating a vehicle trajectory provided by the embodiment of the present application;

[0061] Figure 4 Schematic diagram of the structure of the vehicle recognition device provided by the embodiment of the present application;

[0062] Figure 5 Schematic diagram of the structure of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0063] To solve the problems in the background art, the embodiment of the present application provides a vehicle recognition method, device, electronic device and storage medium.

[0064] The following describes the preferred embodiments of the present application with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. And without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0065] As Figure 1 shown, it is a schematic diagram of the implementation process of the vehicle recognition method provided by the embodiment of the present application, which may include the following steps:

[0066] S11. Obtain a set of video images to be detected.

[0067] Specifically, when implemented, the vehicle detection device obtains a set of video images to be detected captured by a camera in the target detection section.

[0068] S12. For each frame of the video image to be detected in the set of video images to be detected, determine the clarity level of the video image to be detected according to the saturation of the pixel points of the video image to be detected.

[0069] Specifically, when implemented, the vehicle detection device determines the clarity level of each frame of the video image to be detected in the set of video images to be detected according to the process as Figure 2 shown, which may include the following steps:

[0070] S121. Determine the saturation values of the respective pixel points in the detected image and the saturation values of the respective pixel points in the preset reference image, respectively.

[0071] During specific implementation, each camera set in the target detection section captures a clear picture under good weather conditions as the reference image for the monitoring video image of this camera. When the vehicle detection device acquires the video image to be detected captured by this camera, it also acquires this reference image, and this reference image is the clear image of the corresponding scene of the video image to be detected.

[0072] Specifically, the HSV values of each pixel point of the image can be calculated based on the RGB values of each pixel point. The specific calculation method is as follows:

[0073] For any pixel point, if the R value of this pixel point is the maximum value among the R value, G value, and B value of this pixel point, then the H value of this pixel point is: (G - B) / [max(R, G, B) - min(R, G, B)]; if the G value of this pixel point is the maximum value among the R value, G value, and B value of this pixel point, then the H value of this pixel point is: 2 + (B - R) / [max(R, G, B) - min(R, G, B)]; if the B value of this pixel point is the maximum value among the R value, G value, and B value of this pixel point, then the H value of this pixel point is: 4 + (R - G) / [max(R, G, B) - min(R, G, B)]; the H value of this pixel point is: max(R, G, B); the S value of this pixel point is: [max(R, G, B) - min(R, G, B)] / max(R, G, B), and the S value is the saturation value of this pixel point.

[0074] The vehicle detection device calculates the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in the preset reference image respectively through the above calculation method.

[0075] S122. Perform histogram statistics on the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in the preset reference image respectively, and respectively determine the number of pixel points whose saturation values in the video image to be detected and the preset reference image fall into each of at least two set saturation intervals.

[0076] During specific implementation, the vehicle detection device presets at least two saturation intervals. For example, the following 10 saturation intervals can be set: [0, 0.1), [0.1, 0.2), [0.2, 0.3), [0.3, 0.4), [0.4, 0.5), [0.5, 0.6), [0.6, 0.7), [0.7, 0.8), [0.8, 0.9), [0.9, 1]. The embodiments of the present application do not limit this.

[0077] In the specific implementation process, assume that the saturation value of a certain pixel point in the video image to be detected is 0.55. Then, the saturation value of this pixel point falls within the saturation interval [0.5, 0.6). The number of pixel points whose saturation values fall within each of the set at least two saturation intervals in the video image to be detected and the preset reference image is respectively counted, and the saturation histograms corresponding to the video image to be detected and the preset reference image are generated.

[0078] S123. Determine the histogram offset value according to the number of pixel points whose saturation values fall within each saturation interval in the video image to be detected and the number of pixel points whose saturation values fall within each saturation interval in the preset reference image.

[0079] In specific implementation, the histogram offset between the video image to be detected and the preset reference image is calculated by the following formula:

[0080]

[0081] Among them, ΔHist represents the histogram offset value between the video image to be detected and the preset reference image;

[0082] Hist_original i represents the number of pixel points whose saturation values fall within the i-th saturation interval in the preset reference image, and n represents the number of saturation intervals;

[0083] Hist_cur i represents the number of pixel points whose saturation values fall within the i-th saturation interval in the video image to be detected.

[0084] S124. Determine the clarity level of the video image to be detected according to the histogram offset value and the preset clarity thresholds of each level.

[0085] In specific implementation, the vehicle detection device pre-sets n clarity thresholds of each level: Th1 to Thn, which are used to determine the clarity level of the image. The value of n can be set by itself, and this application embodiment does not limit this. The clarity level of the image can be correspondingly set to n levels, which can be expressed as: level 1, level 2,..., level n. It can be indicated that the clarity decreases from level 1 to level n. Level 1 is a clear image captured by the camera under normal sunny weather conditions, with the highest clarity, and level n represents the image captured by the camera under extremely bad weather conditions. Taking foggy days as an example, the greater the fog, the lower the visibility, and the lower the clarity level of the captured image, and the higher the value of the clarity level. For example, the clarity of an image with a clarity level of 2 is greater than the clarity of an image with a clarity level of 5.

[0086] Specifically, when it is determined that the histogram offset value between the video image to be detected and the preset reference image in the calculation is less than the first threshold Th1, the clarity level of the video image to be detected is determined to be level 1; when it is determined that the histogram offset value is less than the second threshold Th2, the clarity level of the video image to be detected is determined to be level 2; when it is determined that the histogram offset value is less than the third threshold Th3, the clarity level of the video image to be detected is determined to be level 3, and so on. By analogy, when it is determined that the histogram offset value is less than the nth threshold Thn, the clarity level of the video image to be detected is determined to be level n.

[0087] S13. Input the video image to be detected into the image clarity correction model corresponding to the clarity level to obtain the corrected image feature information.

[0088] In specific implementation, the vehicle detection device has pre-set the correspondence between the clarity level and the image clarity correction model. Different clarity levels use different image clarity correction models for correction. The vehicle detection device determines the image clarity correction model corresponding to the determined clarity level of the video image to be detected, inputs the video image to be detected into the determined image clarity correction model, and obtains the corrected image feature information. Among them, the image clarity correction models corresponding to different clarity levels are obtained by training a preset deep learning model with each sample image in the sample image set of the corresponding clarity level. The preset deep learning model can be any deep learning model, and the embodiments of the present application do not limit this.

[0089] In implementation, during the process of training the image clarity correction models for different clarity levels, the sample image sets of different clarity levels and the clear images of each sample image are respectively obtained, and the deep learning model parameters are trained according to the sample image sets of different clarity levels and the clear images of each sample image, so as to obtain the image clarity correction models corresponding to each clarity level. Among them, the division of the clarity levels of the sample images is the same as the determination method of the clarity levels of the video images to be detected in steps S121 to S124, which will not be elaborated here. The sample images can be images taken under different degrees of foggy days (or smoggy days, rainy days, etc.). The embodiments of the present application do not limit this.

[0090] S14. Input the corrected image feature information into the preset vehicle recognition model to obtain the vehicle recognition result of the video image to be detected.

[0091] In specific implementation, for each frame of the video image to be detected, the corrected image feature information of the video image to be detected is input into a preset vehicle recognition model to obtain the recognized vehicle features and the position information of each vehicle feature in the video image to be detected, thereby obtaining the vehicle recognition result. Among them, the preset vehicle recognition model can be, but is not limited to, network models such as Encoder-Decoder and Unet. The embodiments of the present application do not limit this.

[0092] S15. Sequentially match the vehicle recognition result of the current frame of the video image to be detected with the vehicle recognition result of the next frame of the video image to be detected to obtain a matching result, and generate a vehicle trajectory according to the matching result.

[0093] In specific implementation, according to the Figure 3 shown process to generate a vehicle trajectory, it may include the following steps:

[0094] S151. Predict the current vehicle driving speed according to the clarity level of the current frame of the video image to be detected and a preset vehicle speed regression prediction model.

[0095] In specific implementation, the preset vehicle speed regression prediction model is a linear regression model representing the linear relationship between the clarity level and the vehicle driving speed. The vehicle speed regression prediction model can be pre-constructed by collecting the driving speeds of vehicles corresponding to a large number of weather conditions corresponding to different clarity levels.

[0096] The vehicle detection device's vehicle speed regression prediction model can predict the driving speed of the vehicle corresponding to the clarity level according to the clarity level of the current frame of the video image to be detected.

[0097] S152. For each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected, determine the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected according to the vehicle driving speed.

[0098] In specific implementation, for each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected by the vehicle detection device, the predicted vehicle driving speed is multiplied by the time interval between the current frame of the video image to be detected and the next frame of the video image to be detected to calculate the driving distance of the vehicle corresponding to the vehicle feature (i.e., the vehicle represented by the vehicle feature) in the next frame of the video image to be detected.

[0099] S153. Predict the position information of the vehicle feature in the next frame of the video image to be detected according to a preset filter and the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected.

[0100] In specific implementation, the preset filter may, but is not limited to, be a Kalman filter. The embodiments of the present application do not limit this, and the Kalman filter is taken as an example for illustration:

[0101] For each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected, the vehicle detection device can predict the position information of the vehicle feature in the next frame of the video image to be detected according to the Kalman filter and the calculated driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected.

[0102] Since adverse weather conditions (such as foggy days) will affect the driving speed of vehicles, if the influence of weather on the driving speed of vehicles is considered, the prediction of the position of vehicle features in the next frame of the video image to be detected using the Kalman filter will be inaccurate. Based on this, the embodiments of the present application consider the influence of weather conditions on the image clarity level. First, the current vehicle driving speed is predicted based on the clarity level and vehicle speed regression prediction model, and the driving distance of the vehicle from the current frame of the video image to be detected to the next frame of the video image to be detected is further calculated according to the predicted vehicle driving speed, which can more accurately determine the position matching range of vehicle features. Thus, the position predicted by the Kalman filter is corrected, improving the accuracy of vehicle position detection and further improving the accuracy of vehicle recognition.

[0103] S154. Determine the target matching position interval of the vehicle feature in the next frame of the video image to be detected according to the predicted position information.

[0104] In specific implementation, after predicting the position information of the vehicle feature in the next frame of the video image to be detected, the range preset around this position can be determined as the target matching position interval with this position as the center point. For example, a rectangular frame can be drawn with this position as the center point, and the rectangular frame can be determined as the target matching position interval. The length and width of the rectangular frame can be set by oneself, and the embodiments of the present application do not limit this.

[0105] S155. Match the vehicle feature with the vehicle feature in the target matching position interval to obtain a matching result.

[0106] In specific implementation, according to the matching results of each vehicle feature and the vehicle feature in its corresponding target matching position interval, the vehicle features that are successfully matched in each frame of the video image to be detected and the next frame of the video image to be detected are determined as the same vehicle, and a vehicle trajectory corresponding to each vehicle is generated, that is, vehicle tracking is completed.

[0107] In the vehicle recognition method provided by the embodiment of the present application, the vehicle detection device acquires a set of video images to be detected. For each frame of the video image to be detected in the set of video images to be detected, the clarity level of the video image to be detected is determined according to the saturation of the pixel points of the video image to be detected. The video image to be detected is input into the image clarity correction model corresponding to the clarity level to obtain the corrected image feature information. Among them, the image clarity correction models corresponding to different clarity levels are obtained by training a preset deep learning model with each sample image in the sample image set corresponding to the corresponding clarity level. Furthermore, the corrected image feature information is input into the preset vehicle recognition model to obtain the vehicle recognition result of the video image to be detected. The vehicle recognition result of the current frame of the video image to be detected and the vehicle recognition result of the next frame of the video image to be detected are sequentially matched to obtain a matching result, and a vehicle trajectory is generated according to the matching result. Compared with the prior art, in the embodiment of the present application, the preset deep learning is trained in advance with the sample images in the sample image sets corresponding to different clarity levels to obtain the image clarity correction models corresponding to different clarity levels. After acquiring the set of video images to be detected, first determine the clarity level of the video image to be detected, and perform clarity correction on the video image to be detected according to the image clarity correction model corresponding to the determined clarity level to obtain the corrected video image to be detected. Then, vehicle recognition is performed on the basis of the obtained corrected image feature information and the preset vehicle recognition model to obtain the vehicle recognition result of each frame of the video image to be detected. Furthermore, the vehicle recognition result of the current frame of the video image to be detected and the vehicle recognition result of the next frame of the video image to be detected are sequentially matched to obtain a matching result, and a vehicle trajectory is generated according to the matching result. In the present application, by detecting the clarity level of the video image to be detected and fusing the image clarity correction model corresponding to the clarity level to restore the image features, the risk of being unable to detect vehicles and track and recognize them due to unclear features and low feature discrimination under adverse weather conditions is reduced, and the accuracy of vehicle recognition and trajectory tracking is improved. Moreover, the image clarity correction model in the embodiment of the present application does not need to obtain a clear video image through the model, but only needs to obtain the image feature information of the intermediate layer, and transmits the image feature information to the subsequent vehicle recognition model for vehicle detection and vehicle feature extraction, effectively improving the efficiency of the overall algorithm.

[0108] Based on the same inventive concept, the embodiment of the present application further provides a vehicle recognition device. Since the principle of solving problems by the above vehicle recognition device is similar to that of the vehicle recognition method, the implementation of the above device can refer to the implementation of the method, and the repeated parts will not be described again.

[0109] As Figure 4 shown, it is a schematic structural diagram of the vehicle recognition device provided by the embodiment of the present application, which may include:

[0110] An acquisition unit 21, configured to acquire a set of video images to be detected;

[0111] A determination unit 22, configured to determine a clarity level of each frame of the video image to be detected in the set of video images to be detected according to the saturation of the pixel points of the video image to be detected;

[0112] A correction unit 23, configured to input the video image to be detected into an image clarity correction model corresponding to the clarity level to obtain corrected image feature information, where the image clarity correction models corresponding to different clarity levels are obtained by training a preset deep learning model with each sample image in a set of sample images corresponding to the respective clarity levels;

[0113] An identification unit 24, configured to input the corrected image feature information into a preset vehicle identification model to obtain a vehicle identification result of the video image to be detected;

[0114] A generation unit 25, configured to sequentially match the vehicle identification result of the current frame of the video image to be detected with the vehicle identification result of the next frame of the video image to be detected to obtain a matching result, and generate a vehicle trajectory according to the matching result.

[0115] In a possible implementation manner, the generation unit 25 is specifically configured to:

[0116] Predict the current vehicle driving speed according to the clarity level of the current frame of the video image to be detected and a preset vehicle speed regression prediction model;

[0117] For each vehicle feature included in the vehicle identification result of the current frame of the video image to be detected, determine the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected according to the vehicle driving speed;

[0118] Predict the position information of the vehicle feature in the next frame of the video image to be detected according to a preset filter and the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected;

[0119] Determine a target matching position interval of the vehicle feature in the next frame of the video image to be detected according to the predicted position information;

[0120] Match the vehicle feature with the vehicle features in the target matching position interval to obtain a matching result.

[0121] In a possible implementation manner, the determination unit 22 is specifically configured to:

[0122] Determine the saturation values of each pixel in the video image to be detected and the saturation values of each pixel in a preset reference image, where the preset reference image is a clear image of the scene corresponding to the video image to be detected;

[0123] Perform histogram statistics on the saturation values of each pixel in the video image to be detected and the saturation values of each pixel in the preset reference image respectively, and determine the number of pixels in each saturation interval where the saturation values in the video image to be detected and the preset reference image fall into at least two set saturation intervals respectively;

[0124] Determine a histogram offset value based on the number of pixels in each saturation interval where the saturation values in the video image to be detected fall and the number of pixels in each saturation interval where the saturation values in the preset reference image fall;

[0125] Determine the clarity level of the video image to be detected according to the histogram offset value and preset clarity thresholds for each level.

[0126] In a possible implementation manner, the determining unit 22 is specifically configured to: calculate the histogram offset through the following formula:

[0127]

[0128] where ΔHist represents the histogram offset value;

[0129] Hist_original i represents the number of pixels in the preset reference image where the saturation values fall into the i-th saturation interval, and n represents the number of saturation intervals;

[0130] Hist_cur i represents the number of pixels in the video image to be detected where the saturation values fall into the i-th saturation interval.

[0131] In a possible implementation manner, the preset filter includes a Kalman filter, and the preset vehicle speed regression prediction model is a linear regression model representing the linear relationship between the clarity level and the vehicle driving speed.

[0132] Based on the same technical concept, the embodiments of the present application also provide an electronic device 300. Refer to Figure 5As shown, the electronic device 300 is used to implement the vehicle recognition method described in the above method embodiments. The electronic device 300 in this embodiment may include: a memory 301, a processor 302, and a computer program stored in the memory and executable on the processor, such as a vehicle recognition program. When the processor executes the computer program, it implements the steps in the above vehicle recognition method embodiments, such as Figure 1 the steps S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above device embodiments, such as 21.

[0133] In the embodiments of the present application, the specific connection medium between the above-mentioned memory 301 and the processor 302 is not limited. In the embodiments of the present application Figure 5 it is shown that the memory 301 and the processor 302 are connected through a bus 303. The bus 303 is represented by a thick line in Figure 5 The connection manners between other components are only for illustrative purposes and are not to be construed as limiting. The bus 303 may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 5 it is only represented by a thick line in, but it does not mean that there is only one bus or one type of bus.

[0134] The memory 301 may be a volatile memory, such as a random-access memory (RAM); the memory 301 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), or the memory 301 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 301 may be a combination of the above memories.

[0135] The processor 302 is used to implement a vehicle recognition method as shown in Figure 1 including:

[0136] The processor 302 is used to call the computer program stored in the memory 301 to execute the steps S11 to S15 as shown in Figure 1 .

[0137] The embodiments of the present application also provide a computer-readable storage medium, storing computer-executable instructions required for the above-mentioned processor to execute, which include programs required for the above-mentioned processor to execute.

[0138] In some possible embodiments, various aspects of the vehicle recognition method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product runs on an electronic device, the program code is used to cause the electronic device to execute the steps in the vehicle recognition method according to various exemplary embodiments of this application described above in this specification. For example, the electronic device can execute steps S11 to S15 as shown in Figure 1 shown.

[0139] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a device, or a computer program product. Therefore, this application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0140] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one or more flows and / or Figure 1 blocks.

[0141] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in Figure 1 one or more flows and / or Figure 1 blocks.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Therefore, the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in Figure 1 one or more flows and / or Figure 1 blocks.

[0143] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present application.

[0144] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A vehicle recognition method, characterized in that, it includes: Obtain a set of video images to be detected; For each frame of the video image to be detected in the set of video images to be detected, determine the clarity level of the video image to be detected according to the saturation of the pixel points of the video image to be detected; Input the video image to be detected into the image clarity correction model corresponding to the clarity level to obtain the corrected image feature information, wherein the image clarity correction models corresponding to different clarity levels are obtained by training a preset deep learning model with each sample image in the sample image set of the corresponding clarity level; Input the corrected image feature information into a preset vehicle recognition model to obtain the vehicle recognition result of the video image to be detected; Sequentially match the vehicle recognition result of the current frame of the video image to be detected with the vehicle recognition result of the next frame of the video image to be detected to obtain a matching result, and generate a vehicle trajectory according to the matching result; Sequentially match the vehicle recognition result of the current frame of the video image to be detected with the vehicle recognition result of the next frame of the video image to be detected to obtain a matching result, specifically including: predicting the current vehicle driving speed according to the clarity level of the current frame of the video image to be detected and a preset vehicle speed regression prediction model, and the preset vehicle speed regression prediction model is a linear regression model representing the linear relationship between the clarity level and the vehicle driving speed; for each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected, determine the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected according to the vehicle driving speed; predict the position information of the vehicle feature in the next frame of the video image to be detected according to a preset filter and the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected; determine the target matching position interval of the vehicle feature in the next frame of the video image to be detected according to the predicted position information; match the vehicle feature with the vehicle features in the target matching position interval to obtain a matching result.

2. The method according to claim 1, characterized in that, Determining the clarity level of the video image to be detected according to the saturation of the pixel points of the video image to be detected specifically includes: Respectively determine the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in a preset reference image, and the preset reference image is a clear image of the scene corresponding to the video image to be detected; Perform histogram statistics on the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in the preset reference image respectively, and respectively determine the number of pixel points in the video image to be detected and the preset reference image whose saturation values fall into each of at least two set saturation intervals; Determine the histogram offset value according to the number of pixel points in the video image to be detected whose saturation values fall into each saturation interval and the number of pixel points in the preset reference image whose saturation values fall into each saturation interval; Determine the clarity level of the video image to be detected according to the histogram offset value and the preset clarity thresholds for each level.

3. The method according to claim 2, wherein, determine the histogram offset value according to the number of pixel points in the video image to be detected whose saturation values fall within each saturation interval and the number of pixel points in the preset reference image whose saturation values fall within each saturation interval, specifically including: calculate the histogram offset through the following formula: where ΔHist represents the histogram offset value; Hist_original i represents the number of pixel points in the preset reference image whose saturation values fall within the \(i\)-th saturation interval, and \(n\) represents the number of saturation intervals; Hist_cur i Indicates the number of pixel points in the video image to be detected whose saturation values fall within the i-th saturation interval.

4. The method according to claim 1, wherein, the preset filter includes a Kalman filter, and the preset vehicle speed regression prediction model is a linear regression model representing the linear relationship between the clarity level and the vehicle driving speed.

5. A vehicle recognition device, wherein, comprising: an acquisition unit for acquiring a set of video images to be detected; a determination unit for determining the clarity level of each frame of the video image to be detected in the set of video images to be detected according to the saturation of the pixel points of the video image to be detected; a correction unit for inputting the video image to be detected into the image clarity correction model corresponding to the clarity level to obtain corrected image feature information, wherein the image clarity correction models corresponding to different clarity levels are obtained by training a preset deep learning model with each sample image in the sample image set corresponding to the respective clarity levels; a recognition unit for inputting the corrected image feature information into a preset vehicle recognition model to obtain the vehicle recognition result of the video image to be detected; a generation unit for sequentially matching the vehicle recognition result of the current frame of the video image to be detected with the vehicle recognition result of the next frame of the video image to be detected to obtain a matching result, and generating a vehicle trajectory according to the matching result; The generation unit is specifically used for: predicting the current vehicle driving speed according to the clarity level of the current frame of the video image to be detected and the preset vehicle speed regression prediction model, and the preset vehicle speed regression prediction model is a linear regression model representing the linear relationship between the clarity level and the vehicle driving speed; for each vehicle feature included in the vehicle recognition result of the current frame of the video image to be detected, determining the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected according to the vehicle driving speed; predicting the position information of the vehicle feature in the next frame of the video image to be detected according to the preset filter and the driving distance of the vehicle corresponding to the vehicle feature in the next frame of the video image to be detected; determining the target matching position interval of the vehicle feature in the next frame of the video image to be detected according to the predicted position information; and matching the vehicle feature with the vehicle features in the target matching position interval to obtain a matching result.

6. The device according to claim 5, wherein, the determination unit is specifically used for: Determine the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in a preset reference image respectively, where the preset reference image is a clear image of the corresponding scene of the video image to be detected; Perform histogram statistics on the saturation values of each pixel point in the video image to be detected and the saturation values of each pixel point in the preset reference image respectively, and determine the number of pixel points whose saturation values fall into each of at least two preset saturation intervals in the video image to be detected and the preset reference image respectively; Determine a histogram offset value according to the number of pixel points whose saturation values fall into each of the saturation intervals in the video image to be detected and the number of pixel points whose saturation values fall into each of the saturation intervals in the preset reference image; Determine the clarity level of the video image to be detected according to the histogram offset value and preset clarity thresholds for each level; 7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the vehicle recognition method according to any one of claims 1 to 4; 8. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the program is executed by a processor, it implements the steps in the vehicle recognition method according to any one of claims 1 to 4.

Images