Image data processing method and system

By combining the image data processing methods of visible light and infrared cameras, using grayscale reconstruction and thermal parameter correction, the problem of low vehicle recognition accuracy at night is solved, the accuracy of vehicle type recognition is improved, and the service quality of parking lots is improved. Operational efficiency.

CN119091391AInactive Publication Date: 2024-12-06CHENGDU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411594733.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-10
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing vehicle identification technology has low accuracy at night or under poor lighting, which affects the parking lot's judgment of vehicle type and the supporting facilities of charging facilities, and reduces service quality and operational efficiency.

Method used

The image data processing method combined with visible light camera and infrared camera is adopted, and the recognition type of vehicle type is improved through grayscale processing, grayscale reconstruction, target segmentation and convolutional neural network feature recognition, combined with thermal parameter correction, to improve the recognition accuracy of vehicle type.

Benefits of technology

The accuracy of vehicle type identification is significantly improved at night or under poor lighting conditions, ensuring the accurate judgment of vehicle type in the parking lot and the adaptability of charging facilities, and improving service quality and operational efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091391A_ABST
    Figure CN119091391A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to the technical field of computer vision, and discloses an image data processing method and system, the method is applied to a vehicle visual recognition device.The method comprises the steps that firstly, binary gray reconstruction and target segmentation are conducted on a vehicle image shot by a visible light camera to obtain a target vehicle image, and therefore the target vehicle image is obtained through binary gray reconstruction; the effects of enhancing the target and inhibiting the background are achieved under the condition that the original outline of the vehicle is reserved, so that the accuracy of obtaining the target vehicle image is improved; and then feature recognition is carried out through a convolutional neural network to pre-recognize the type of the target vehicle, and under the condition that the recognition probability of pre-recognition is low, pre-recognition correction or confirmation is carried out through the thermal parameters of the target vehicle to obtain the target recognition type. Therefore, the identification accuracy of the vehicle type at night or under the condition of poor light is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image data processing method and system. Background Art

[0002] With the rapid growth of car ownership, parking management has become an indispensable part of urban management and traffic planning. In the daily operation of parking lots, accurately identifying the types of vehicles entering and leaving is crucial to optimizing resource allocation, improving user experience, and ensuring safety. In particular, identifying the difference between fuel vehicles and electric vehicles is directly related to whether the parking lot can provide suitable charging facilities to meet the growing demand for electric vehicle charging.

[0003] However, existing vehicle recognition technology can usually achieve good recognition results under conditions of sufficient light and clear environment. However, at night or in a dimly lit environment, the accuracy of the visual recognition system is often greatly reduced due to factors such as decreased image quality and blurred vehicle features. This not only affects the parking lot's judgment of vehicle types, but may also lead to mismatch or idleness of supporting facilities such as charging piles, reducing the service quality and operational efficiency of the parking lot.

[0004] Therefore, how to improve the accuracy of vehicle type recognition at night or under poor lighting conditions has become a technical problem that needs to be solved urgently in the field of parking management. Summary of the invention

[0005] The main purpose of the present invention is to provide an image data processing method and system, aiming to solve the technical problem in the prior art of low accuracy in identifying vehicle types at night or under poor lighting conditions.

[0006] To achieve the above-mentioned purpose, in a first aspect, an embodiment of the present application provides an image data processing method, which is applied to a vehicle visual recognition device, wherein the vehicle visual recognition device includes a first camera and a second camera, wherein the first camera is a visible light camera, and the second camera is an infrared camera, and the method includes: Acquire a target image captured by the first camera, wherein the target image is an image captured when the vehicle reaches a preset distance from the first camera; The target image is grayscaled and grayscale reconstructed to obtain a target grayscale image; Performing target segmentation on the target grayscale image to obtain a target vehicle image; Inputting the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, wherein the pre-recognition type of the target vehicle is a fuel vehicle or an electric vehicle; When the recognition probability of the target vehicle is lower than the probability threshold, a thermal sequence frame image captured by the second camera is obtained, wherein the thermal sequence frame image is a combination of thermal image sequences captured continuously by the infrared camera within a preset time period; Determining thermal parameters of the target vehicle according to the thermal sequence frame images; The pre-identified type of the target vehicle is corrected or confirmed based on the thermal parameters of the target vehicle.

[0007] In a possible implementation, grayscale processing and grayscale reconstruction of the target image to obtain a target grayscale image includes: Decolorizing and graying the target image to obtain an original grayscale image; Extracting the grayscale value of each pixel of the original grayscale image to form a distribution curve diagram of the pixel and the grayscale value; Determine a grayscale threshold for grayscale reconstruction according to the distribution curve of the pixel points and grayscale values, wherein the grayscale threshold is the grayscale value corresponding to the maximum slope point of the curve segment before the peak of the distribution curve; The target grayscale image is obtained by performing grayscale reconstruction on the original grayscale image according to the grayscale threshold.

[0008] In a possible implementation, the grayscale reconstruction of the original grayscale image according to the grayscale threshold to obtain the target grayscale image includes: Compare the grayscale value of each pixel of the original grayscale image with the grayscale threshold; Assigning a first grayscale value to pixels in the original grayscale image whose grayscale values ​​are greater than or equal to the grayscale threshold, and assigning a second grayscale value to pixels in the original grayscale image whose grayscale values ​​are less than the grayscale threshold, wherein the difference between the first grayscale value and the second grayscale value is greater than the maximum difference in grayscale values ​​between pixels in the original grayscale image; The target grayscale image is obtained by performing grayscale reconstruction based on assigning the first grayscale value and the second grayscale value to the original grayscale image.

[0009] In a possible implementation, the step of performing target segmentation on the target grayscale image to obtain a target vehicle image includes: Based on edge detection technology, the target grayscale image is subjected to boundary feature recognition to obtain a local boundary line or a complete boundary line of the target vehicle, wherein the local boundary line includes a vehicle head boundary line or a vehicle top boundary line; Target segmentation is performed according to the local boundary line or the complete boundary line of the target vehicle to obtain a target vehicle image.

[0010] In a possible implementation, after inputting the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, the method further includes: When the recognition probability of the target vehicle is higher than or equal to the probability threshold, the pre-recognition type of the target vehicle obtained by the convolutional neural network is used as the target recognition type.

[0011] In a possible implementation, the thermal parameters include thermal parameters of the front face of the vehicle, and determining the thermal parameters of the target vehicle according to the thermal sequence frame images includes: The maximum change in the thermal value of the front face of the target vehicle is determined based on the thermal sequence frame image.

[0012] In a possible implementation, the correcting or confirming the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle includes: In the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum change in the calorific value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identified type of the target vehicle is confirmed; if the maximum change in the calorific value of the front face of the target vehicle is less than the first preset value, the pre-identified type of the target vehicle is corrected; When the pre-identification type of the target vehicle is an electric vehicle, if the change in the maximum thermal value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identification type of the target vehicle is corrected; if the change in the maximum thermal value of the front face of the target vehicle is less than the first preset value, the pre-identification type of the target vehicle is confirmed.

[0013] In a possible implementation, the thermal parameters include thermal parameters of a top portion of a vehicle, and determining the thermal parameters of the target vehicle according to the thermal sequence frame images includes: The maximum thermal value of the roof of the target vehicle is determined according to the thermal sequence frame image.

[0014] In a possible implementation, the correcting or confirming the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle includes: In the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum calorific value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is corrected; if the maximum calorific value of the roof of the target vehicle is less than or equal to the second preset value, the pre-identified type of the target vehicle is confirmed; In the case where the pre-identified type of the target vehicle is an electric vehicle, if the maximum thermal value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is confirmed; if the maximum thermal value of the roof of the target vehicle is less than or equal to the preset value, the pre-identified type of the target vehicle is corrected; Wherein, the second preset value is greater than or equal to the current vehicle outside temperature.

[0015] In a second aspect, an embodiment of the present application further provides an image data processing system, comprising a memory and a processor, wherein the memory is used to store program code, and the processor is used to call the program code to execute the method described in the first aspect.

[0016] Different from the prior art, the image data processing method provided in the embodiment of the present application first obtains the target image captured by the first camera; then grayscales and reconstructs the target image to obtain a target grayscale image; then segmentes the target grayscale image to obtain a target vehicle image; then inputs the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, wherein the pre-recognition type of the target vehicle is a fuel vehicle or an electric vehicle; when the recognition probability of the target vehicle is lower than a probability threshold, obtains a thermal sequence frame image captured by the second camera; then determines the thermal parameters of the target vehicle based on the thermal sequence frame image; finally, corrects or confirms the pre-recognition type of the target vehicle based on the thermal parameters of the target vehicle. That is, firstly, the vehicle image taken by the visible light camera is reconstructed by binary grayscale and the target is segmented to obtain the target vehicle image. In this way, the binary grayscale reconstruction can achieve the effect of enhancing the target and suppressing the background while retaining the original outline of the vehicle, thereby improving the accuracy of the target vehicle image acquisition. Then, the convolutional neural network is used to perform feature recognition to pre-identify the target vehicle. When the recognition probability of pre-identification is low, the thermal parameters of the target vehicle are used to correct or confirm the pre-identification to obtain the target recognition type. In this way, the recognition accuracy of the vehicle type at night or in poor light conditions is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0018] Figure 1A schematic diagram of a flow chart of an image data processing method in some embodiments of the present application; Figure 2 Schematic diagram of the flow of image data processing methods in other embodiments of the present application; Figure 3 A distribution curve diagram of the original grayscale image pixels and grayscale values ​​in some embodiments of the present application; Figure 4 Schematic diagram of the hardware structure of the image data processing system in some embodiments of the present application.

[0019] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, "and / or" in the full text includes three solutions. Taking A and / or B as an example, it includes technical solution A, technical solution B, and technical solution that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0023] With the rapid growth of car ownership, parking management has become an indispensable part of urban management and traffic planning. In the daily operation of parking lots, accurately identifying the types of vehicles entering and leaving is crucial to optimizing resource allocation, improving user experience, and ensuring safety. In particular, identifying the difference between fuel vehicles and electric vehicles is directly related to whether the parking lot can provide suitable charging facilities to meet the growing demand for electric vehicle charging.

[0024] However, existing vehicle recognition technology can usually achieve good recognition results under conditions of sufficient light and clear environment. However, at night or in a dimly lit environment, the accuracy of the visual recognition system is often greatly reduced due to factors such as decreased image quality and blurred vehicle features. This not only affects the parking lot's judgment of vehicle types, but may also lead to mismatch or idleness of supporting facilities such as charging piles, reducing the service quality and operational efficiency of the parking lot.

[0025] Therefore, how to improve the accuracy of vehicle type recognition at night or under poor lighting conditions has become a technical problem that needs to be solved urgently in the field of parking management.

[0026] In response to the above problems, the present application proposes an image data processing method, which is applied to a vehicle visual recognition device, wherein the vehicle visual recognition device includes a first camera and a second camera, wherein the first camera is a visible light camera and the second camera is an infrared camera. The visible light camera operates within the visible spectrum, which is roughly the same as the range of light that the human eye can perceive. It captures the visible light reflected or emitted by an object to generate the color or black and white images that we commonly see. The infrared camera can detect the infrared radiation emitted by an object and convert it into a visible or system-recognizable thermal image, and the thermal image carries information such as temperature.

[0027] The overall idea of ​​the image data processing method of the present application is: at night or under poor lighting conditions, the distinction between the target and the background in the image taken by the visible light camera is poor, resulting in low accuracy in vehicle type recognition; and relying solely on the infrared camera for vehicle type recognition, there are many external influencing factors, resulting in low accuracy in vehicle type recognition. Therefore, the technical solution of the present application first performs binary grayscale reconstruction and target segmentation on the vehicle image taken by the visible light camera to obtain the target vehicle image. In this way, through binary grayscale reconstruction, the effect of enhancing the target and suppressing the background is achieved while retaining the original contour of the vehicle, thereby improving the accuracy of the target vehicle image acquisition; and then performing feature recognition through a convolutional neural network to pre-recognize the target vehicle. When the pre-recognition probability is low, the pre-recognition correction or confirmation is performed through the thermal parameters of the target vehicle to obtain the target recognition type. In this way, the recognition accuracy of the vehicle type at night or under poor lighting conditions is greatly improved.

[0028] like Figure 1-3 As shown, the following will mainly describe the specific steps of the image data processing method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described here can be performed in a different order. Figure 1 , the method comprises the following steps: S100, obtaining a target image captured by the first camera, wherein the target image is an image captured when the vehicle reaches a preset distance from the first camera; To ensure that the camera has a clear shooting effect, the distance between the first camera and the vehicle is set to a preset distance for capturing images. For example, when the vehicle stops and waits in front of the barrier, the vehicle image is captured. The shooting distance can be determined according to the resolution and pixels of the camera to obtain a clear vehicle image. For example, the distance between the first camera and the vehicle barrier is set to 50 cm.

[0029] It can be understood that the target image includes the target vehicle area and the background area. Under dark or poor light conditions, the target image captured by the first camera has the defect that the target object and the background cannot be clearly distinguished, especially when the vehicle color is black or dark. Therefore, it is necessary to preprocess the target image in order to accurately extract the target object in the target image, so as to accurately judge the vehicle type.

[0030] S200, graying and reconstructing the target image to obtain a target gray image; After obtaining a target image (the target image includes a target vehicle and a background) captured by a camera, the present application performs grayscale processing and grayscale binary reconstruction on the target image to obtain a target grayscale image.

[0031] In one embodiment, the step S200: graying the target image and reconstructing the grayscale to obtain a target grayscale image includes: S210, performing decolorization and grayscale processing on the target image to obtain an original grayscale image; S220, extracting the grayscale value of each pixel of the original grayscale image to form a distribution curve diagram of the pixel and the grayscale value; S230, determining a grayscale threshold for grayscale reconstruction according to the distribution curve of the pixel points and grayscale values, wherein the grayscale threshold is the grayscale value corresponding to the maximum slope point of the curve segment before the peak of the distribution curve; S240 , reconstructing the grayscale of the original grayscale image according to the grayscale threshold to obtain a target grayscale image.

[0032] Specifically, the target image is first decolorized and grayscaled to obtain the original grayscale image, and then the grayscale value of each pixel in the original grayscale image is extracted to form a distribution curve graph of the pixel and the grayscale value. Then, the grayscale threshold for grayscale reconstruction is determined according to the distribution curve graph of the pixel and the grayscale value, and then the original grayscale image is reconstructed according to the grayscale threshold to obtain the target grayscale image.

[0033] It should be noted that selecting an appropriate grayscale threshold is the key to target segmentation. Figure 3 As shown in the figure, in the original grayscale image, the grayscale at both ends of the image has a significant decrease, and the number of pixels with grayscale distribution in the range of 0 to 10 forms a peak, which corresponds to the main grayscale of the background image in the dark area; the vehicle image occupies a small number of pixels, and the grayscale is lower than most of the background image, and should be located on the left side of the peak; the grayscale of the vehicle image is similar to the grayscale corresponding to the peak, but the number of pixels is different, and the peak has a steep front slope. The steepest position of the front slope corresponds to the optimal grayscale threshold for segmenting the target vehicle and the background. Therefore, the implementation of this application sets the maximum slope point of the curve segment before the peak of the distribution curve ( Figure 3 That is, before the gray value reaches its peak value, the gray value at the point with the largest gray value change (the steepest point of the curve) is used as the gray value threshold, thereby segmenting the target image and the background image, greatly improving the accuracy of acquiring the target vehicle image.

[0034] After obtaining the grayscale threshold, the original grayscale image needs to be reconstructed into binary grayscale to obtain a target grayscale image that clearly distinguishes the target object from the background, thereby facilitating target extraction to obtain a target vehicle image.

[0035] In one embodiment, the step S240 of reconstructing the grayscale of the original grayscale image according to the grayscale threshold to obtain the target grayscale image includes: S241, numerically comparing the grayscale value of each pixel of the original grayscale image with the grayscale threshold; S242, assigning a first grayscale value to pixels in the original grayscale image whose grayscale values ​​are greater than or equal to the grayscale threshold, and assigning a second grayscale value to pixels in the original grayscale image whose grayscale values ​​are less than the grayscale threshold, wherein the difference between the first grayscale value and the second grayscale value is greater than the maximum difference in grayscale values ​​between pixels in the original grayscale image; S243, completing grayscale reconstruction based on assigning a first grayscale value and a second grayscale value to the original grayscale image to obtain a target grayscale image.

[0036] It can be understood that, due to insufficient light at night, the background color is darker, for example, completely black, so the pixels in the original grayscale image with grayscale values ​​greater than or equal to the grayscale threshold are likely to be the background image area, and the pixels in the original grayscale image with grayscale values ​​less than the grayscale threshold are likely to be the target vehicle image area. Therefore, in order to further improve the distinction between the target vehicle and the background, thereby achieving the effect of enhancing the target and suppressing the background, the first grayscale value is assigned to the pixels in the original grayscale image with grayscale values ​​greater than or equal to the grayscale threshold, and the second grayscale value is assigned to the pixels in the original grayscale image with grayscale values ​​less than the grayscale threshold, and the difference between the first grayscale value assignment and the second grayscale value assignment is large enough, for example, the first grayscale value is 0, the second grayscale value is 255, or the first grayscale value is 255, and the second grayscale value is 0. In this way, the target vehicle can be distinguished from the background to the maximum extent, thereby obtaining a target grayscale image.

[0037] S300, performing target segmentation on the target grayscale image to obtain a target vehicle image; After obtaining a target grayscale image that maximizes the distinction between the target vehicle and the background, it is necessary to perform target segmentation on the target grayscale image to obtain a target vehicle image.

[0038] In one embodiment, the step S300: performing target segmentation on the target grayscale image to obtain a target vehicle image, includes: performing boundary feature recognition on the target grayscale image based on edge detection technology to obtain a local boundary line or a complete boundary line of the target vehicle, wherein the local boundary line includes a vehicle head boundary line or a vehicle roof boundary line.

[0039] Specifically, edge detection technology can be used to identify the entire boundary line of the target vehicle to obtain a complete vehicle image, and the complete vehicle image is used as the target vehicle image for the next step of identification; edge detection technology can also be used to identify the local boundary line of the target vehicle to obtain a local vehicle image, such as a front image or a top image, and the front image or the top image is used as the target vehicle image for the next step of identification.

[0040] S400, inputting the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, wherein the pre-recognition type of the target vehicle is a fuel vehicle or an electric vehicle; After obtaining the target vehicle image, the target vehicle image is input into a pre-trained convolutional neural network for feature recognition to obtain the pre-recognition type of the target vehicle and the corresponding recognition probability.

[0041] It can be understood that when the recognition probability obtained by the initial recognition of the convolutional neural network (CNN) is large (for example, greater than the first probability threshold), it means that the recognition accuracy is high. At this time, the target vehicle recognition type obtained by the convolutional neural network feature recognition can be used as the target recognition type; when the predicted probability obtained by the initial recognition of the convolutional neural network (CNN) is small (less than or equal to the first probability threshold), it means that the recognition accuracy is low, which may be caused by poor image background suppression during the image processing process. Therefore, recognition correction / verification is required through other methods to improve the recognition accuracy.

[0042] S500, when the recognition probability of the target vehicle is lower than the probability threshold, obtaining a thermal sequence frame image captured by the second camera, wherein the thermal sequence frame image is a combination of thermal image sequences captured continuously by the infrared camera within a preset time period; In one embodiment, if the recognition probability of the target vehicle is lower than the probability threshold, it means that the recognition accuracy is low and cannot meet the requirements of vehicle recognition. At this time, the thermal sequence frame image captured by the second camera can be obtained to re-identify the target vehicle through the thermal sequence frame image.

[0043] Thermal images can carry thermal parameters of different parts of the vehicle, and there can be many types of thermal parameters, such as high temperature area parameters, high temperature area shape parameters, maximum temperature parameters or temperature change parameters, etc. Each parameter has a different effect on target recognition.

[0044] S600, determining thermal parameters of the target vehicle according to the thermal sequence frame image; In one embodiment, the thermal parameters include thermal parameters of the front face of the vehicle, and the step S600: determining the thermal parameters of the target vehicle according to the thermal sequence frame image includes: The maximum change in the thermal value of the front face of the target vehicle is determined based on the thermal sequence frame image.

[0045] In other embodiments, the thermal parameters further include thermal parameters of the top of the vehicle, and the step S600: determining the thermal parameters of the target vehicle according to the thermal sequence frame image includes: The maximum thermal value of the roof of the target vehicle is determined according to the thermal sequence frame image.

[0046] It should be noted that the thermal sequence frame images of the present application are thermal sequence frame images captured by the second camera within a preset time period when the vehicle is traveling from far to near (with the infrared camera as a reference point).

[0047] It is understandable that due to the different types of vehicles, the thermal parameters carried by the vehicle thermal image may also be different. Therefore, the type of vehicle can be determined based on the thermal parameters to correct or further confirm the pre-identified type of the target vehicle.

[0048] S700: Correct or confirm the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle.

[0049] In one embodiment, the pre-recognized type of the target vehicle may be corrected or further confirmed based on the thermal parameters of the front face of the vehicle.

[0050] Specifically, the step S700: correcting or confirming the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle includes: S710: In the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum change in the calorific value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identified type of the target vehicle is confirmed; if the maximum change in the calorific value of the front face of the target vehicle is less than the first preset value, the pre-identified type of the target vehicle is corrected; S720. When the pre-identification type of the target vehicle is an electric vehicle, if the change in the maximum calorific value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identification type of the target vehicle is corrected; if the change in the maximum calorific value of the front face of the target vehicle is less than the first preset value, the pre-identification type of the target vehicle is confirmed.

[0051] It should be noted that since the generator of the fuel vehicle is set in front of the vehicle and the front face of the vehicle is equipped with an air intake / heat dissipation grille, the front face of the fuel vehicle has obvious heating phenomenon, and the closer the vehicle is to the infrared camera, the higher the temperature of the front face is recognized. Therefore, the target vehicle can be judged as a fuel vehicle or an electric vehicle based on the change in temperature value (with the infrared camera as the reference point) during the vehicle's driving from far to near. If the current face temperature change is large, it can be confirmed or confirmed with a high probability that the target vehicle is a fuel vehicle. If the current face temperature change is small or almost unchanged, it can be confirmed or confirmed with a high probability that the target vehicle is an electric vehicle.

[0052] The temperature change of the front face of the vehicle can be the maximum thermal value change of the front face, that is, the maximum thermal value of the front face of each frame of the thermal image is taken, and then the maximum thermal value in the first frame and the last frame are subtracted to obtain the maximum thermal value change.

[0053] Therefore, in the case where the pre-identification type of the target vehicle is a fuel vehicle, if the change in the maximum calorific value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identification type of the target vehicle is confirmed, that is, the target vehicle is confirmed to be a fuel vehicle; if the change in the maximum calorific value of the front face of the target vehicle is less than the first preset value, the pre-identification type of the target vehicle is corrected, that is, the target vehicle identification result is corrected to an electric vehicle.

[0054] In the case where the pre-identification type of the target vehicle is an electric vehicle, if the change in the maximum calorific value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identification type of the target vehicle is corrected, that is, the target vehicle identification result is corrected to a fuel vehicle; if the change in the maximum calorific value of the front face of the target vehicle is less than the first preset value, the pre-identification type of the target vehicle is confirmed, that is, the target vehicle is confirmed to be an electric vehicle.

[0055] In other embodiments, the pre-identified type of the target vehicle may also be corrected or further confirmed based on the thermal parameters of the top of the vehicle.

[0056] Specifically, the step S700: correcting or confirming the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle includes: S730: In the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum calorific value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is corrected; if the maximum calorific value of the roof of the target vehicle is less than or equal to the second preset value, the pre-identified type of the target vehicle is confirmed; S740: When the pre-identified type of the target vehicle is an electric vehicle, if the maximum thermal value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is confirmed; if the maximum thermal value of the roof of the target vehicle is less than or equal to the preset value, the pre-identified type of the target vehicle is corrected; Wherein, the second preset value is greater than or equal to the current vehicle outside temperature.

[0057] It should be noted that since the top of the electric vehicle is equipped with electronic equipment such as laser radar and cameras, the electronic equipment will generate a certain amount of heat when working. The top of the electric vehicle has obvious heating phenomenon, and the closer the vehicle is to the infrared camera, the higher the temperature of the top of the vehicle is recognized. Therefore, the temperature value of the top of the vehicle can be used to determine whether the target vehicle is a fuel vehicle or an electric vehicle. When the temperature value of the top of the vehicle is large (greater than the ambient temperature), it can be confirmed or confirmed with a high probability that the target vehicle is an electric vehicle. When the temperature value of the top of the vehicle is small (less than or equal to the ambient temperature), it can be confirmed or confirmed with a high probability that the target vehicle is a fuel vehicle. The temperature value of the top of the vehicle can be the maximum thermal value of the top of the vehicle during the vehicle's driving from far to near (with the infrared camera as the reference point).

[0058] Therefore, in the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum calorific value of the roof of the target vehicle is greater than the second preset value, the pre-identified type of the target vehicle is corrected, that is, the target vehicle identification result is corrected to an electric vehicle; if the maximum calorific value of the roof of the target vehicle is less than or equal to the second preset value, the pre-identified type of the target vehicle is confirmed, that is, the target vehicle is confirmed to be a fuel vehicle; In the case where the pre-identified type of the target vehicle is an electric vehicle, if the maximum calorific value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is confirmed, that is, the target vehicle is confirmed to be an electric vehicle; if the maximum calorific value of the roof of the target vehicle is less than or equal to the second preset value, the pre-identified type of the target vehicle is corrected, that is, the identification result of the target vehicle is corrected to a fuel vehicle.

[0059] Based on this, the image data processing method of the present application first performs binary grayscale reconstruction and target segmentation on the vehicle image taken by the visible light camera to obtain the target vehicle image. In this way, through binary grayscale reconstruction, the effect of enhancing the target and suppressing the background is achieved while retaining the original outline of the vehicle, thereby improving the accuracy of acquiring the target vehicle image, and then performing feature recognition through a convolutional neural network to pre-identify the target vehicle. When the recognition probability of pre-identification is low, the pre-identification is corrected or confirmed through the thermal parameters of the target vehicle to obtain the target recognition type. In this way, the recognition accuracy of the vehicle type at night or in poor light conditions is greatly improved.

[0060] The present application also provides an image data processing system. Figure 4 , Figure 4 A schematic diagram of the hardware structure of an image data processing system provided for some embodiments of the present application; comprising a memory 110 and a processor 120, wherein the memory 110 is used to store program code, and the processor 120 is used to call the program code to execute the method described above.

[0061] Among them, the processor 120 is used to provide computing and control capabilities to control the image data processing system to perform corresponding tasks, for example, to control the image data processing system to perform the image data processing method in any of the above method embodiments, the method comprising: obtaining a target image captured by the first camera, wherein the target image is an image captured when the vehicle reaches a preset distance from the first camera; graying the target image and reconstructing the grayscale to obtain a target grayscale image; performing target segmentation on the target grayscale image to obtain a target vehicle image; inputting the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, wherein the pre-recognition type of the target vehicle is a fuel vehicle or an electric vehicle; when the recognition probability of the target vehicle is lower than a probability threshold, obtaining a thermal sequence frame image captured by the second camera, wherein the thermal sequence frame image is a combination of thermal image sequences continuously captured by an infrared camera within a preset time length; determining thermal parameters of the target vehicle according to the thermal sequence frame images; and correcting or confirming the pre-recognition type of the target vehicle according to the thermal parameters of the target vehicle.

[0062] The processor 120 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0063] The memory 110 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the image data processing method in the embodiment of the present application. The processor 120 can implement the image data processing method in any of the above method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 110.

[0064] Specifically, the memory 110 may include a volatile memory (VM), such as a random access memory (RAM); the memory 110 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD) or other non-transitory solid-state storage device; the memory 110 may also include a combination of the above types of memory.

[0065] In summary, the image data processing system of the present application adopts the technical solution of any one of the above-mentioned image data processing method embodiments, and therefore, has at least the beneficial effects brought by the technical solution of the above-mentioned embodiments, which will not be described one by one here.

[0066] The embodiment of the present application also provides a computer-readable storage medium, such as a memory including a program code, and the program code can be executed by a processor to complete the image data processing method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0067] The present application also provides a computer program product, which includes one or more program codes stored in a computer-readable storage medium. The processor of the image data processing system reads the program code from the computer-readable storage medium, and the processor executes the program code to complete the steps of the image data processing method provided in the above embodiment.

[0068] A person skilled in the art will appreciate that all or part of the steps for implementing the above embodiments may be accomplished by hardware or by hardware associated with a program code, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0069] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0070] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that each embodiment can be implemented by means of software plus a general hardware platform, or by hardware. It can be understood by those skilled in the art that all or part of the processes in the above embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0071] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An image data processing method is applied to a vehicle visual recognition device, wherein the vehicle visual recognition device comprises a first camera and a second camera, wherein the first camera is a visible light camera and the second camera is an infrared camera. It is characterized in that The method comprises: Acquire a target image captured by the first camera, wherein the target image is an image captured when the vehicle reaches a preset distance from the first camera; The target image is grayscaled and grayscale reconstructed to obtain a target grayscale image; Performing target segmentation on the target grayscale image to obtain a target vehicle image; Inputting the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, wherein the pre-recognition type of the target vehicle is a fuel vehicle or an electric vehicle; When the recognition probability of the target vehicle is lower than the probability threshold, a thermal sequence frame image captured by the second camera is obtained, wherein the thermal sequence frame image is a combination of thermal image sequences captured continuously by the infrared camera within a preset time period; Determining thermal parameters of the target vehicle according to the thermal sequence frame images; The pre-identified type of the target vehicle is corrected or confirmed based on the thermal parameters of the target vehicle.

2. The image data processing method according to claim 1, It is characterized in that The grayscale processing and grayscale reconstruction of the target image to obtain the target grayscale image includes: Decolorizing and graying the target image to obtain an original grayscale image; Extracting the grayscale value of each pixel of the original grayscale image to form a distribution curve diagram of the pixel and the grayscale value; Determine a grayscale threshold for grayscale reconstruction according to the distribution curve of the pixel points and grayscale values, wherein the grayscale threshold is the grayscale value corresponding to the maximum slope point of the curve segment before the peak of the distribution curve; The target grayscale image is obtained by performing grayscale reconstruction on the original grayscale image according to the grayscale threshold.

3. The image data processing method according to claim 2, It is characterized in that The grayscale reconstruction of the original grayscale image according to the grayscale threshold to obtain the target grayscale image includes: Compare the grayscale value of each pixel of the original grayscale image with the grayscale threshold; Assigning a first grayscale value to pixels in the original grayscale image whose grayscale values ​​are greater than or equal to the grayscale threshold, and assigning a second grayscale value to pixels in the original grayscale image whose grayscale values ​​are less than the grayscale threshold, wherein the difference between the first grayscale value and the second grayscale value is greater than the maximum difference in grayscale values ​​between pixels in the original grayscale image; The target grayscale image is obtained by performing grayscale reconstruction based on assigning the first grayscale value and the second grayscale value to the original grayscale image.

4. The image data processing method according to claim 1, It is characterized in that The step of performing target segmentation on the target grayscale image to obtain a target vehicle image includes: Based on edge detection technology, the target grayscale image is subjected to boundary feature recognition to obtain a local boundary line or a complete boundary line of the target vehicle, wherein the local boundary line includes a vehicle head boundary line or a vehicle top boundary line; Target segmentation is performed according to the local boundary line or the complete boundary line of the target vehicle to obtain a target vehicle image.

5. The image data processing method according to claim 1, It is characterized in that After inputting the target vehicle image into a pre-trained convolutional neural network for feature recognition to obtain a pre-recognition type of the target vehicle and a corresponding recognition probability, the method further includes: When the recognition probability of the target vehicle is higher than or equal to the probability threshold, the pre-recognition type of the target vehicle obtained by the convolutional neural network is used as the target recognition type.

6. The image data processing method according to claim 1, It is characterized in that The thermal parameters include the thermal parameters of the front face of the vehicle, and the step of determining the thermal parameters of the target vehicle according to the thermal sequence frame images includes: The maximum change in the thermal value of the front face of the target vehicle is determined based on the thermal sequence frame image.

7. The image data processing method according to claim 6, It is characterized in that The correcting or confirming the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle includes: In the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum change in the calorific value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identified type of the target vehicle is confirmed; if the maximum change in the calorific value of the front face of the target vehicle is less than the first preset value, the pre-identified type of the target vehicle is corrected; When the pre-identification type of the target vehicle is an electric vehicle, if the change in the maximum thermal value of the front face of the target vehicle is greater than or equal to a first preset value, the pre-identification type of the target vehicle is corrected; if the change in the maximum thermal value of the front face of the target vehicle is less than the first preset value, the pre-identification type of the target vehicle is confirmed.

8. The image data processing method according to claim 1, It is characterized in that The thermal parameters include thermal parameters of the top of the vehicle, and determining the thermal parameters of the target vehicle according to the thermal sequence frame image includes: The maximum thermal value of the roof of the target vehicle is determined according to the thermal sequence frame image.

9. The image data processing method according to claim 8, It is characterized in that The correcting or confirming the pre-identified type of the target vehicle according to the thermal parameters of the target vehicle includes: In the case where the pre-identified type of the target vehicle is a fuel vehicle, if the maximum calorific value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is corrected; if the maximum calorific value of the roof of the target vehicle is less than or equal to the second preset value, the pre-identified type of the target vehicle is confirmed; In the case where the pre-identified type of the target vehicle is an electric vehicle, if the maximum thermal value of the roof of the target vehicle is greater than a second preset value, the pre-identified type of the target vehicle is confirmed; if the maximum thermal value of the roof of the target vehicle is less than or equal to the preset value, the pre-identified type of the target vehicle is corrected; Wherein, the second preset value is greater than or equal to the current vehicle outside temperature.

10. An image data processing system, It is characterized in that The method comprises a memory and a processor, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method according to any one of claims 1 to 9.