In-vehicle AI camera visual recognition system
By combining thermal imaging and visible image data, edge extraction and image combination algorithms are used to solve the recognition accuracy of the vehicle-mounted AI camera visual recognition system in harsh environments, improving the safety and recognition efficiency of autonomous driving, and reducing computing overhead.
Patent Information
- Application Number
- CN202410846619.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-27
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-06-27
AI Technical Summary
The existing car-mounted AI camera visual recognition system has reduced the accuracy of visual recognition in severe weather or insufficient light, and has high development and maintenance costs, huge data demands, and requires continuous collection and processing of large amounts of image and sensor data.
The combination of thermal imaging module and camera module is adopted to enhance image data through edge extraction and image combination algorithms, improve the recognition effect of the recognition module, and work stably under different environmental conditions to improve the accuracy and response speed of recognition.
Effectively identify and respond to obstacles and pedestrians on the road in complex driving environments, enhance the safety of autonomous driving of vehicles, improve identification accuracy and response speed, reduce the calculation overhead of subsequent identification processing, and simplify the calculation burden of target recognition models.
Smart Images

Figure CN118397233B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of AI recognition, and in particular to a vehicle-mounted AI camera visual recognition system. Background Art
[0002] The on-board AI camera visual recognition system is a vehicle camera system that integrates artificial intelligence technology and is mainly used to enhance the vehicle's autonomous driving and assisted driving functions. This system captures visual information around the vehicle through a camera, and then uses AI algorithms to analyze and recognize this information to identify road signs, pedestrians, other vehicles, and various road conditions.
[0003] Existing technical solutions mainly include the use of deep learning models, such as convolutional neural networks (CNN) and recurrent neural networks (RNN), to process and parse image data. These models are able to learn and recognize complex patterns and features from large amounts of image data. In addition, some systems also combine radar and laser radar (LIDAR) technology to improve the accuracy and reliability of recognition.
[0004] The benefits of these systems include increased road safety, fewer traffic accidents, and greater convenience and comfort for drivers. For example, by recognizing and interpreting traffic signs and signals, the system can help drivers better comply with traffic regulations or automatically adjust vehicle speed by identifying potential collision hazards.
[0005] However, these systems also have some disadvantages. First, they are highly dependent on the accuracy of the algorithm and environmental conditions. For example, in bad weather or low light conditions, the accuracy of visual recognition may be reduced. In addition, the development and maintenance costs of these systems are relatively high, and the demand for data is huge. A large amount of image and sensor data needs to be continuously collected and processed to keep the system effective and updated. Summary of the invention
[0006] This application provides a vehicle-mounted AI camera visual recognition system, including:
[0007] A thermal imaging module, used to obtain thermal imaging image data;
[0008] A camera module, used to obtain visible light image data;
[0009] A processing module, used for processing thermal imaging image data and visible light image data;
[0010] The recognition module is used to identify and determine the corresponding object information according to the image data processed by the processing module;
[0011] A response module, used to make a corresponding response according to the object information generated by the recognition module;
[0012] Wherein, the thermal imaging module and the camera module are communicably connected to the processing module, the processing module is communicably connected to the recognition module, and the response module is communicably connected to the recognition module;
[0013] Wherein, the thermal imaging module and the camera module are arranged adjacent to each other, and the thermal imaging module and the camera module have the same shooting direction;
[0014] Among them, the processing module includes an edge extraction module and an image combination module, the edge extraction module can be communicatively connected to the thermal imaging module, the image combination module can be communicatively connected to the edge extraction module and the camera module, and the edge extraction module and the image combination module can be communicatively connected to the recognition module.
[0015] By adopting the above technical solution, the on-board AI camera visual recognition system can enhance image data by combining thermal imaging images and visible light images, thereby improving the recognition effect of the recognition module, and automatically responding according to the recognition results, thereby enhancing the vehicle's autonomous driving and safety monitoring capabilities.
[0016] Optionally, the vehicle-mounted AI camera visual recognition system further includes an image recognition strategy, including the following steps:
[0017] A1, obtaining thermal imaging image data through the thermal imaging module;
[0018] A2, obtaining visible light image data through the camera module;
[0019] A3, extracting and generating edge image data through the edge extraction module according to the thermal imaging image data;
[0020] A4, determining corresponding object pixel area data using a preset image segmentation algorithm according to the thermal imaging image data;
[0021] A5, generating corresponding edge enhanced image data according to the combination of the edge image data and the visible light image data;
[0022] A6, generating object recognition information through the recognition module according to the object pixel area data and the edge enhanced image data.
[0023] By adopting the above technical solution, the on-board AI camera visual recognition system can work stably under different environmental conditions by combining thermal imaging and visible light image data, and can further enhance image details and contrast through edge extraction and image fusion algorithms, thereby improving recognition accuracy; especially in complex driving environments, it can effectively identify and respond to obstacles and pedestrians on the road, enhancing the vehicle's autonomous driving safety.
[0024] Optionally, the image recognition strategy further comprises the following steps for generating edge image data:
[0025] B1, generating thermal imaging noise reduction image data using a preset image noise reduction algorithm according to the thermal imaging image data;
[0026] B2, generating thermal imaging contrast adjustment image data according to the thermal imaging noise reduction image data using a preset contrast enhancement algorithm;
[0027] B3, generating thermal imaging sharpened image data by using a preset sharpening filter algorithm according to the thermal imaging contrast adjustment image data;
[0028] B4, generating edge image data according to the thermal imaging sharpened image data using a preset edge detection algorithm.
[0029] By adopting the above technical solution, the on-board AI camera visual recognition system can significantly improve the quality of thermal imaging and edge clarity through image noise reduction, contrast enhancement, sharpening and edge detection, so that the system can effectively extract important edge image data under different environmental conditions.
[0030] Optionally, the image recognition strategy further includes the following steps for determining object pixel area data:
[0031] C1, dividing the thermal imaging image data into multiple thermal imaging local image data of equal size according to a preset dividing size;
[0032] C2, respectively calculate the sum of the pixel grayscale values of the local image data of each thermal imaging edge;
[0033] C3, if the sum of the pixel grayscale values of the thermal imaging local image data is greater than a preset pixel grayscale threshold, the thermal imaging local image data is defined as valid thermal imaging local image data;
[0034] C4, determining corresponding effective area data according to each effective thermal imaging local image data;
[0035] C5, generates object pixel area data based on all valid area data combinations.
[0036] By adopting the above technical solution, the on-board AI camera visual recognition system can accurately identify areas with significant temperature differences by dividing the thermal imaging image into multiple local images and calculating the pixel grayscale values, thereby effectively distinguishing objects from the background. It is suitable for determining the image area of the object in a complex environment, reducing the computational overhead of subsequent recognition processing, and can effectively improve the recognition accuracy and response speed.
[0037] Optionally, the image recognition strategy further comprises the following steps for generating edge enhanced image data:
[0038] D1, obtaining corresponding visible light image size data according to the visible light image data;
[0039] D2, generating edge up-sampled image data by using a preset up-sampling algorithm according to the edge image data and the visible light image size data;
[0040] D3, generating corresponding visible light grayscale image data according to the visible light image data using a preset grayscale image conversion algorithm;
[0041] D4, generating edge enhanced image data according to the edge up-sampled image data and the visible light grayscale image data using a preset image fusion algorithm.
[0042] By adopting the above technical solution, the on-board AI camera visual recognition system can create an edge-enhanced image by upsampling the edge image data to the size of a visible light image and fusing it with the visible light grayscale image data, thereby making the edges of objects in the image more clearly visible and improving the recognition accuracy of objects in the environment.
[0043] Optionally, the image fusion algorithm comprises the following steps:
[0044] E1, sample the image data on the edge to determine the corresponding non-zero value pixels and define them as valid edge pixels;
[0045] E2, according to the pixel position of each effective edge pixel and the visible light grayscale image data, the corresponding pixel position is determined and defined as the edge enhancement pixel position;
[0046] E3, obtaining the corresponding pixel grayscale value according to each edge enhancement pixel position;
[0047] E4, multiplying the pixel grayscale value of each edge enhanced pixel position by a preset enhancement ratio to calculate a pixel update value;
[0048] E5, replacing the corresponding pixel grayscale value according to the pixel update value of each edge enhanced pixel position.
[0049] By adopting the above technical solution, the on-board AI camera visual recognition system can significantly improve the visual characteristics of the edge, ensure that the edge information of the object is highlighted in the image, and by identifying and enhancing the effective edge pixels, not only the local contrast and clarity of the image are enhanced, but also the recognizability of the object contour is improved. Especially when the visible light image is covered, an image completion effect can be formed to increase the stability of object recognition.
[0050] Optionally, the image recognition strategy further includes the following steps for identifying and generating object recognition information:
[0051] F1, respectively obtain the pixel position of each pixel in the edge enhanced image data;
[0052] F2, if the pixel position is outside the object pixel area data, then the pixel gray value corresponding to the pixel position in the edge enhanced image data is set to a preset pixel default value;
[0053] F3, define the edge enhanced image data as background mask image data;
[0054] F4, generating pixel location data of each target and corresponding target classification information according to the background mask image data using a preset target recognition model;
[0055] F5, object recognition information is generated based on the combination of all target pixel positioning data and corresponding target classification information.
[0056] By adopting the above technical solution, the vehicle-mounted AI camera visual recognition system can effectively reduce background noise by setting the pixel grayscale value of the non-target area to a preset default value, making the characteristics of the target object more prominent. This method makes the separation of background and foreground clearer, thereby simplifying the calculation burden of the target recognition model and improving the processing speed.
[0057] Optionally, the image recognition strategy further includes the following steps for generating training data to train the object recognition model:
[0058] G1, obtaining corresponding material image data and corresponding material label information from a preset training material database;
[0059] G2, extracting the corresponding material edge image data according to each material image data;
[0060] G3, selecting a corresponding amount of material image data from each material image data according to a preset selection ratio and defining them as simulation image data;
[0061] G4, performing partial masking on each simulated image data to generate corresponding masked image data;
[0062] G5, generating corresponding material grayscale image data according to each material image data;
[0063] G6, generating corresponding cover grayscale image data according to each cover image data;
[0064] G7, generating edge enhanced material grayscale image data according to the fusion of each material grayscale image data, the mask grayscale image data and the corresponding material edge image data;
[0065] G8, generating a corresponding material grayscale image label pair according to the grayscale image data of each edge enhancement material and the corresponding material label information;
[0066] G9 generates training data based on all the grayscale image label pairs of the materials.
[0067] By adopting the above technical solution, the on-board AI camera visual recognition system can use existing image materials and generate corresponding materials in harsh environments through edge enhancement and local masking simulation to create a high-quality data set suitable for training. This not only increases the adaptability of the model to real-world conditions, but also ensures that the model can accurately identify and classify different targets when faced with diverse scenarios and conditions.
[0068] In summary, the present application includes at least one of the following beneficial technical effects:
[0069] 1. The thermal imaging image and the visible light image can be combined to enhance the image data, thereby improving the recognition effect of the recognition module, and automatically responding according to the recognition result, thereby enhancing the vehicle's autonomous driving and safety monitoring capabilities.
[0070] 2. By combining thermal imaging and visible light image data, it can work stably under different environmental conditions, and can further enhance image details and contrast through edge extraction and image fusion algorithms, thereby improving recognition accuracy; especially in complex driving environments, it can effectively identify and respond to obstacles and pedestrians on the road, enhancing the vehicle's autonomous driving safety.
[0071] 3. The quality of thermal imaging and edge clarity can be significantly improved through image noise reduction, contrast enhancement, sharpening and edge detection, so that the system can effectively extract important edge image data under different environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 It is a schematic diagram of the principle of a vehicle-mounted AI camera visual recognition system of the present invention.
[0073] Figure 2 It is a flow chart of the vehicle-mounted AI camera visual recognition system of the present invention. DETAILED DESCRIPTION
[0074] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0075] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.
[0076] refer to Figure 1 The present invention provides a vehicle-mounted AI camera visual recognition system, which identifies objects in the image by simultaneously acquiring visible light images and thermal imaging images of lanes and the surroundings of the vehicle, and makes corresponding response actions based on the recognition results.
[0077] The vehicle-mounted AI camera visual recognition system includes:
[0078] Thermal imaging module 10, used to obtain thermal imaging image data;
[0079] Camera module 20, used to obtain visible light image data;
[0080] A processing module 30, used for processing thermal imaging image data and visible light image data;
[0081] The recognition module 40 is used to recognize and determine the corresponding object information according to the image data processed by the processing module 30;
[0082] A response module 50, configured to make a corresponding response according to the object information generated by the recognition module 40;
[0083] The thermal imaging module 10 and the camera module 20 are communicably connected to the processing module 30, the processing module 30 is communicably connected to the recognition module 40, and the response module 50 is communicably connected to the recognition module 40;
[0084] Wherein, the thermal imaging module 10 and the camera module 20 are arranged adjacent to each other, and the thermal imaging module 10 and the camera module 20 have the same shooting direction;
[0085] Among them, the processing module 30 includes an edge extraction module 31 and an image combination module 32, the edge extraction module 31 can be communicatively connected to the thermal imaging module 10, the image combination module 32 can be communicatively connected to the edge extraction module 31 and the camera module 20, and the edge extraction module 31 and the image combination module 32 can be communicatively connected to the recognition module 40.
[0086] The thermal imaging module 10 is mainly used to obtain thermal imaging images of the lane and the surroundings of the vehicle, which are used for analysis and processing by the processing module 30.
[0087] The camera module 20 is mainly used to obtain visible light images of the lane and the surroundings of the vehicle, which are used for analysis and processing by the processing module 30.
[0088] In bad weather, factors such as rain or fog may cause the visible light image acquired by the camera module 20 to be partially or completely covered, making the effect of visible light image analysis alone unstable. However, the thermal imaging image acquired through infrared fluctuations is not easily affected by bad weather and can complement the visible light image, thereby improving the recognition effect of the recognition module 40 and improving driving safety.
[0089] The processing module 30 is mainly used to process and combine the thermal imaging image and the visible light image to generate a corresponding fused image, and fuse the feature information in the thermal imaging image and the visible light image to improve the subsequent object recognition effect of the recognition module 40.
[0090] The recognition module 40 is mainly used to select and identify objects based on the fused image data generated by the processing module 30 to generate recognition information, so as to determine the category information and relative position of the object so that the response module 50 can make an appropriate response.
[0091] The response module 50 is mainly used to take corresponding actions according to the identification information generated by the identification module 40. For example, if a human body is identified in front of the vehicle, a warning message is sent to the driver or the driver actively brakes to improve driving safety.
[0092] The edge extraction module 31 is mainly used to extract and generate a corresponding object edge image according to the thermal imaging image.
[0093] The image combination module 32 is mainly used to combine the visible light image and the object edge image to enhance and complete the visible light image, so as to increase its information content and further improve the recognition effect of the recognition module 40 .
[0094] In bad weather, especially rainy days, the refraction of light caused by water droplets may cause the edges of objects in the visible light image obtained by the camera module 20 to be lost or blurred, thereby affecting the recognition efficiency. Therefore, by extracting the corresponding object edges through the less susceptible thermal imaging image and then adding the object edges to the visible light image, the recognition stability in bad environments can be improved, and the recognition effect can also be improved under normal circumstances.
[0095] By arranging the thermal imaging module 10 and the camera module 20 adjacently and in the same direction, the degree of pixel position offset between the thermal imaging image and the visible light image can be reduced, so as to facilitate image fusion. Furthermore, the thermal imaging module 10 and the camera module 20 can be set to have the same focal length, which can reduce the difference between the thermal imaging image and the visible light image.
[0096] Through the above technical solution, the on-board AI camera visual recognition system can enhance image data by combining thermal imaging images and visible light images, thereby improving the recognition effect of the recognition module, and automatically responding according to the recognition results, thereby enhancing the vehicle's autonomous driving and safety monitoring capabilities.
[0097] Further, refer to Figure 2 The vehicle-mounted AI camera visual recognition system also includes an image recognition strategy, which includes the following steps:
[0098] A1, obtaining thermal imaging image data through the thermal imaging module 10;
[0099] The thermal imaging image data is the thermal imaging grayscale image data acquired instantly by the thermal imaging module 10 .
[0100] A2, obtaining visible light image data through the camera module 20;
[0101] The visible light image data is the image data of visible light fluctuations acquired instantly by the camera module 20 , that is, the image acquired by a general camera.
[0102] A3, extracting and generating edge image data through the edge extraction module 31 according to the thermal imaging image data;
[0103] The edge image data is thermal imaging image data extracted by the edge extraction module 31 to generate image data, which can be obtained by setting a corresponding edge extraction algorithm in the edge extraction module 31. The commonly used edge extraction algorithms include Roberts algorithm, Sobel algorithm, Kirsch algorithm and Canny algorithm, etc., which can extract the corresponding edge image according to the image, among which the Canny algorithm is better;
[0104] The edge image data usually generated is binary image data, that is, each pixel is 0 or 1, the value of non-edge pixels is 0 or black pixels, and the value of edge pixels is 1 or white pixels.
[0105] A4, determining corresponding object pixel area data using a preset image segmentation algorithm according to the thermal imaging image data;
[0106] The object pixel region data is the pixel position data that may contain the corresponding object in the edge image data;
[0107] The existing algorithms for image segmentation include threshold segmentation algorithm, region growing algorithm and graph-based image segmentation algorithm, which can segment the image to determine the corresponding object area;
[0108] By determining the pixel area data of the object, the background content with less object information can be removed to reduce the subsequent image processing overhead;
[0109] Because the human body or other vehicles that need to be identified and avoided when the vehicle is driving have a certain temperature, and the background temperature is generally lower, image segmentation through thermal imaging will have a better segmentation effect, and the background content can usually be distinguished to reduce subsequent data processing.
[0110] A5, generating corresponding edge enhanced image data according to the combination of the edge image data and the visible light image data;
[0111] The edge-enhanced image data is an enhanced image generated by combining edge image data and visible light image data with an image fusion algorithm or an algorithm with a similar effect, so as to highlight the outline of the object in the visible light image data, which can improve or ensure the subsequent object recognition effect;
[0112] Especially when there is image obstruction in the visible light image data, by appending the object contour in the thermal imaging to the visible light image, on the one hand, the object recognition in the visible light image can be enhanced, and on the other hand, the obscured contour can be compensated, so that the recognition result has better stability.
[0113] A6, generating object recognition information through the recognition module 40 according to the object pixel area data and the edge enhanced image data;
[0114] The object recognition information is information generated by combining the object pixel area data and the edge enhanced image data through the recognition module 40;
[0115] The object recognition information may include position information of the object in the edge-enhanced image data and object classification information;
[0116] By setting a pre-trained recognition model in the recognition module 40, corresponding object recognition can be performed. The recognition model can be an algorithm model such as R-CNN, FastR-CNN, SSD, YOLO, etc., and the detection area of the target can be narrowed by the object pixel area data, which can reduce the computational overhead in the recognition process and improve the recognition efficiency.
[0117] Through the object pixel area data, each recognition algorithm can avoid detecting the target in the background area that does not contain the object, reducing unnecessary calculations and improving the overall recognition efficiency.
[0118] Through the above steps, the on-board AI camera visual recognition system can work stably under different environmental conditions by combining thermal imaging and visible light image data, and can further enhance image details and contrast through edge extraction and image fusion algorithms, thereby improving recognition accuracy; especially in complex driving environments, it can effectively identify and respond to obstacles and pedestrians on the road, enhancing the vehicle's autonomous driving safety.
[0119] Furthermore, the image recognition strategy further includes the following steps for generating edge image data:
[0120] B1, generating thermal imaging noise reduction image data using a preset image noise reduction algorithm according to the thermal imaging image data;
[0121] The image noise reduction algorithm is a pre-set algorithm used to reduce noise or smooth the image to reduce the noise generated during the shooting process and affect the subsequent edge extraction effect;
[0122] Usually, the sensor or module used to capture thermal images has a low resolution, and the impact of noise on the image will be particularly obvious;
[0123] Commonly used noise reduction algorithms include Gaussian blur, bilateral filtering, non-local mean denoising, etc., which can achieve the effect of noise reduction;
[0124] The thermal imaging noise reduction image data is image data generated after the thermal imaging image data is subjected to noise reduction processing.
[0125] B2, generating thermal imaging contrast adjustment image data according to the thermal imaging noise reduction image data using a preset contrast enhancement algorithm;
[0126] The contrast enhancement algorithm is a pre-set algorithm used to enhance the contrast of the image to improve the subsequent edge extraction effect;
[0127] In thermal imaging images, different objects present different grayscale values due to different temperatures. Contrast enhancement can make the temperature difference more obvious, making it easier to identify the objects in the image.
[0128] Existing contrast enhancement algorithms include histogram stretching, histogram equalization, adaptive histogram equalization, gamma correction, etc., which can be selected and adjusted according to specific image characteristics and processing requirements;
[0129] The thermal imaging contrast-adjusted image data is image data generated after contrast adjustment of the thermal imaging noise reduction image data.
[0130] B3, generating thermal imaging sharpened image data by using a preset sharpening filter algorithm according to the thermal imaging contrast adjustment image data;
[0131] The sharpening filter algorithm is a pre-set algorithm used to sharpen images. It is mainly used to enhance the details in the image, especially the edges and textures, to make the image look clearer. For thermal images, it can help highlight areas with smaller temperature differences.
[0132] Existing sharpening filter algorithms include Laplace filter, high-pass filter, UnsharpMask, high-frequency boost filter, etc. These sharpening technologies can help improve the quality of thermal images and enhance the edge distinction between objects with low temperature differences;
[0133] The thermal imaging sharpened image data is image data generated by processing the thermal imaging contrast adjusted image data with a sharpening filter algorithm.
[0134] B4, generating edge image data using a preset edge detection algorithm according to the thermal imaging sharpened image data;
[0135] The edge detection algorithm is a pre-set algorithm used to extract the corresponding object edge image based on the thermal imaging image;
[0136] The existing edge detection algorithms include Roberts algorithm, Sobel algorithm, Kirsch algorithm and Canny algorithm, etc., which can extract the corresponding edge image according to the image, among which Canny algorithm is better;
[0137] The edge image data is image data generated by edge detection algorithm of thermal imaging sharpened image data; usually, the edge image data is binary image data.
[0138] Through the above steps, the on-board AI camera visual recognition system can significantly improve the quality of thermal imaging and edge clarity through image noise reduction, contrast enhancement, sharpening and edge detection, so that the system can effectively extract important edge image data under different environmental conditions.
[0139] Furthermore, the image recognition strategy further includes the following steps for determining object pixel area data:
[0140] C1, dividing the thermal imaging image data into multiple thermal imaging local image data of equal size according to a preset dividing size;
[0141] The segmentation size is a preset size value, which is used to segment the thermal imaging image data into multiple equal-sized blocks; for example, if the segmentation size is set to 10*10 pixels, then a 100*100 pixel image can be segmented into 100 10*10 pixel block images;
[0142] The thermal imaging local image data is block image data formed after the thermal imaging image data is segmented.
[0143] C2, respectively calculate the sum of the pixel grayscale values of the local image data of each thermal imaging edge;
[0144] The sum of the pixel grayscale values is the sum of the grayscale values of each pixel point of the local image data of the thermal imaging edge. Generally, the lower the grayscale value, the lower the temperature of the content captured in the image. The low temperature is usually the background part, such as the sky, buildings and other distant scenes.
[0145] C3, if the sum of the pixel grayscale values of the thermal imaging local image data is greater than a preset pixel grayscale threshold, the thermal imaging local image data is defined as valid thermal imaging local image data;
[0146] The pixel grayscale threshold is a preset reference value used to determine whether the total pixel grayscale value of the local thermal imaging image data reaches a certain level;
[0147] When the temperature reaches a certain level, it may be a thermal image containing human bodies and vehicles;
[0148] The effective thermal imaging local image data refers to the thermal imaging local image data whose total pixel grayscale value is greater than a preset pixel grayscale threshold.
[0149] C4, determining corresponding effective area data according to each effective thermal imaging local image data;
[0150] The effective area data is the position data information corresponding to the effective thermal imaging local image data in the thermal imaging image data.
[0151] C5, generates object pixel area data based on all valid area data combinations.
[0152] The object pixel area data is a collection of all valid area data, reflecting all image areas in the thermal imaging image data that may contain human bodies, vehicles or other objects with higher temperatures.
[0153] Through the above steps, the vehicle-mounted AI camera visual recognition system can accurately identify areas with significant temperature differences by dividing the thermal imaging image into multiple local images and calculating the pixel grayscale values, thereby effectively distinguishing objects from the background. It is suitable for determining the image area of the object in a complex environment, reducing the computational overhead of subsequent recognition processing, and can effectively improve the recognition accuracy and response speed.
[0154] Furthermore, the image recognition strategy also includes the following steps for generating edge enhanced image data:
[0155] D1, obtaining corresponding visible light image size data according to the visible light image data;
[0156] The visible light image size data is the size corresponding to the visible light image data, for example, 1000 pixels in the X-axis direction and 800 pixels in the Y-axis direction.
[0157] D2, generating edge up-sampled image data by using a preset up-sampling algorithm according to the edge image data and the visible light image size data;
[0158] The upsampling algorithm is a pre-set algorithm used to enlarge the image to a certain size;
[0159] Existing upsampling algorithms include nearest neighbor interpolation, linear interpolation, Lanczos resampling, etc.
[0160] The edge upsampled image data is image data generated by enlarging the edge image data to visible light image size data through an upsampling algorithm, that is, the edge image data is enlarged to the same size as the visible light image data.
[0161] D3, generating corresponding visible light grayscale image data according to the visible light image data using a preset grayscale image conversion algorithm;
[0162] The grayscale image conversion algorithm is a pre-set algorithm used to convert a color image into a grayscale image;
[0163] Common grayscale image conversion algorithms include averaging method, weighting method, decomposition method, etc., which can convert color images into corresponding grayscale images;
[0164] The visible light grayscale image data is a grayscale image generated after the visible light image data is grayscaled.
[0165] D4, generating edge enhanced image data by using a preset image fusion algorithm according to the edge up-sampled image data and the visible light grayscale image data;
[0166] The image fusion algorithm is a pre-set algorithm used to fuse two different images into a single image while retaining their respective image features;
[0167] Existing image fusion algorithms include high-pass filter fusion, principal component analysis, multi-resolution fusion, etc.
[0168] The edge enhanced image data is an image generated by fusion of the edge up-sampled image data and the visible light grayscale image data.
[0169] Through the above steps, the on-board AI camera visual recognition system can create an edge-enhanced image by upsampling the edge image data to the size of the visible light image and fusing it with the visible light grayscale image data, so that the edges of objects in the image are more clearly visible and the recognition accuracy of objects in the environment is improved.
[0170] Furthermore, the image fusion algorithm comprises the following steps:
[0171] E1, sample the image data on the edge to determine the corresponding non-zero value pixels and define them as valid edge pixels;
[0172] A non-zero value pixel is a pixel whose pixel value is not 0;
[0173] The edge image generated by extraction is usually a binary image, in which each pixel is generally represented by 0 as a non-edge pixel and 1 or 255 as an edge pixel;
[0174] The valid edge pixels are the pixels corresponding to the edge.
[0175] E2, according to the pixel position of each effective edge pixel and the visible light grayscale image data, the corresponding pixel position is determined and defined as the edge enhancement pixel position;
[0176] The edge enhancement pixel position is the position of the pixel point that needs to be enhanced in the visible light grayscale image;
[0177] Since the edge upsampled image data and the visible light grayscale image data are of the same size, the corresponding object edge position can be found in the visible light grayscale image data according to the edge upsampled image data, and edge enhancement processing can be performed.
[0178] E3, obtaining the corresponding pixel grayscale value according to each edge enhancement pixel position;
[0179] Obtain each edge-enhanced pixel position of the visible light grayscale image data to obtain the grayscale value of the corresponding pixel.
[0180] E4, multiplying the pixel grayscale value of each edge enhanced pixel position by a preset enhancement ratio to calculate a pixel update value;
[0181] The enhancement ratio is a preset ratio value, which is used to adjust the pixel gray value at each edge enhancement pixel position to highlight the edge of the object;
[0182] The pixel update value is a new pixel grayscale value generated after the enhancement ratio is adjusted.
[0183] E5, replacing the corresponding pixel grayscale value according to the pixel update value of each edge enhanced pixel position.
[0184] The pixel update value generated after the enhanced adjustment replaces the original pixel grayscale value to enhance the pixel.
[0185] Through the above steps, the on-board AI camera visual recognition system can significantly improve the visual characteristics of the edge, ensure that the edge information of the object is highlighted in the image, and by identifying and enhancing the effective edge pixels, not only the local contrast and clarity of the image are enhanced, but also the recognizability of the object contour is improved. Especially when the visible light image is covered, an image completion effect can be formed to increase the stability of object recognition.
[0186] Furthermore, the image recognition strategy further includes the following steps for generating object recognition information:
[0187] F1, respectively obtain the pixel position of each pixel in the edge enhanced image data;
[0188] F2, if the pixel position is outside the object pixel area data, then the pixel gray value corresponding to the pixel position in the edge enhanced image data is set to a preset pixel default value;
[0189] The pixel default value is a preset value, which is used to replace the pixel values of the pixels that are not located in the object pixel area data in the edge enhanced image data to remove background information irrelevant to the object. For example, the pixel default value can be set to 0, and the pixel values of all pixels that are not in the object pixel area data are set to 0 to reduce the interference of the background image on subsequent recognition;
[0190] F3, define the edge enhanced image data as background mask image data;
[0191] The background masked image data is the edge enhanced image data processed in step F2, from which the background image information is removed.
[0192] F4, generating pixel location data of each target and corresponding target classification information according to the background mask image data using a preset target recognition model;
[0193] The target recognition model is a pre-set model used to identify and determine the object information in the image based on the image;
[0194] The recognition model can be R-CNN, FastR-CNN, SSD, YOLO and other algorithm models;
[0195] The target pixel location data is the data of the pixel location of the identified object in the image;
[0196] The target classification information is the category information of the identified object, for example, the classification information identified as a car, pedestrian, bicycle, cat, dog, etc.
[0197] F5, generating object recognition information based on all target pixel positioning data and corresponding target classification information;
[0198] The generated object recognition information is a collection of all target pixel location data and corresponding target classification information.
[0199] Through the above steps, the vehicle-mounted AI camera visual recognition system can effectively reduce background noise by setting the pixel grayscale value of the non-target area to a preset default value, making the features of the target object more prominent. This method makes the separation of background and foreground clearer, thereby simplifying the computational burden of the target recognition model and improving the processing speed.
[0200] Furthermore, the image recognition strategy further includes the following steps for generating training data to train the object recognition model:
[0201] G1, obtaining corresponding material image data and corresponding material label information from a preset training material database;
[0202] The training material database is a pre-selected database storing material images used for recognition training;
[0203] The material image data is the image used for recognition model training;
[0204] The material tag information is classification information corresponding to the material image, indicating classification information corresponding to the object in the material image.
[0205] G2, extracting the corresponding material edge image data according to each material image data;
[0206] The material edge image data is an edge image generated by extracting the material image data, and can be extracted and generated through the aforementioned related steps.
[0207] G3, selecting a corresponding amount of material image data from each material image data according to a preset selection ratio and defining them as simulation image data;
[0208] The selected ratio is a preset ratio value used to determine the image ratio to be processed;
[0209] The analog image data is material image data selected to be subjected to image processing.
[0210] G4, performing partial masking on each simulated image data to generate corresponding masked image data;
[0211] The masked image data is an image generated after partial masking of the simulated image data;
[0212] Partial masking can be used to simulate the situation where visible light images are blocked in harsh environments, and can be used to train recognition models.
[0213] G5, generating corresponding material grayscale image data according to each material image data;
[0214] The material grayscale image data is a grayscale image corresponding to each material image data.
[0215] G6, generating corresponding cover grayscale image data according to each cover image data;
[0216] The mask grayscale image data is a grayscale image corresponding to each mask image data.
[0217] G7, generating edge enhanced material grayscale image data according to the fusion of each material grayscale image data, the mask grayscale image data and the corresponding material edge image data;
[0218] The edge-enhanced material grayscale image data is an image generated by fusing the material grayscale image data and the masking grayscale image data with the corresponding material edge image data.
[0219] G8, generating a corresponding material grayscale image label pair according to the grayscale image data of each edge enhancement material and the corresponding material label information;
[0220] The material grayscale image label pair is a data pair generated by combining edge enhanced material grayscale image data and corresponding material label information.
[0221] G9 generates training data based on all the grayscale image label pairs of the materials.
[0222] The training data is the collection of all material grayscale image label pairs, which is used to train the corresponding recognition model.
[0223] Through the above steps, the on-board AI camera visual recognition system can use existing image materials and generate corresponding materials in harsh environments through edge enhancement and local masking simulation to create a high-quality data set suitable for training. This not only increases the adaptability of the model to real-world conditions, but also ensures that the model can accurately identify and classify different targets when faced with diverse scenarios and conditions.
[0224] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.
Claims
1. A vehicle-mounted AI camera visual recognition system, characterized in that: include: A thermal imaging module, used to obtain thermal imaging image data; A camera module, used to obtain visible light image data; A processing module, used for processing thermal imaging image data and visible light image data; The recognition module is used to identify and determine the corresponding object information according to the image data processed by the processing module; A response module, used to make a corresponding response according to the object information generated by the recognition module; Wherein, the thermal imaging module and the camera module are communicably connected to the processing module, the processing module is communicably connected to the recognition module, and the response module is communicably connected to the recognition module; Wherein, the thermal imaging module and the camera module are arranged adjacent to each other, and the thermal imaging module and the camera module have the same shooting direction; Wherein, the processing module includes an edge extraction module and an image combination module, the edge extraction module can be communicatively connected to the thermal imaging module, the image combination module can be communicatively connected to the edge extraction module and the camera module, and the edge extraction module and the image combination module can be communicatively connected to the recognition module; The vehicle-mounted AI camera visual recognition system further includes an image recognition strategy, including the following steps: A1, obtaining thermal imaging image data through the thermal imaging module; A2, obtaining visible light image data through the camera module; A3, extracting and generating edge image data through the edge extraction module according to the thermal imaging image data; A4, determining corresponding object pixel area data using a preset image segmentation algorithm according to the thermal imaging image data; A5, generating corresponding edge enhanced image data according to the combination of the edge image data and the visible light image data; A6, generating object recognition information through the recognition module according to the object pixel area data and the edge enhanced image data; The image recognition strategy further includes the following steps for identifying and generating object recognition information: F1, respectively obtain the pixel position of each pixel in the edge enhanced image data; F2, if the pixel position is outside the object pixel area data, then the pixel gray value corresponding to the pixel position in the edge enhanced image data is set to a preset pixel default value; F3, define the edge enhanced image data as background mask image data; F4, generating pixel location data of each target and corresponding target classification information according to the background mask image data using a preset target recognition model; F5, generating object recognition information based on all target pixel positioning data and corresponding target classification information; The image recognition strategy further includes the following steps for generating training data to train the target recognition model: G1, obtaining corresponding material image data and corresponding material label information from a preset training material database; G2, extracting the corresponding material edge image data according to each material image data; G3, selecting a corresponding amount of material image data from each material image data according to a preset selection ratio and defining them as simulation image data; G4, performing partial masking on each simulated image data to generate corresponding masked image data; G5, generating corresponding material grayscale image data according to each material image data; G6, generating corresponding cover grayscale image data according to each cover image data; G7, generating edge enhanced material grayscale image data according to the fusion of each material grayscale image data, the mask grayscale image data and the corresponding material edge image data; G8, generating a corresponding material grayscale image label pair according to the grayscale image data of each edge enhancement material and the corresponding material label information; G9 generates training data based on all the grayscale image label pairs of the materials.
2. The vehicle-mounted AI camera visual recognition system according to claim 1, characterized in that: The image recognition strategy further comprises the following steps for generating edge image data: B1, generating thermal imaging noise reduction image data using a preset image noise reduction algorithm according to the thermal imaging image data; B2, generating thermal imaging contrast adjustment image data according to the thermal imaging noise reduction image data using a preset contrast enhancement algorithm; B3, generating thermal imaging sharpened image data by using a preset sharpening filter algorithm according to the thermal imaging contrast adjustment image data; B4, generating edge image data according to the thermal imaging sharpened image data using a preset edge detection algorithm.
3. The vehicle-mounted AI camera visual recognition system according to claim 2, characterized in that: The image recognition strategy further includes the following steps for determining object pixel area data: C1, dividing the thermal imaging image data into multiple thermal imaging local image data of equal size according to a preset dividing size; C2, respectively calculate the sum of the pixel grayscale values of the local image data of each thermal imaging edge; C3, if the sum of the pixel grayscale values of the thermal imaging local image data is greater than a preset pixel grayscale threshold, the thermal imaging local image data is defined as valid thermal imaging local image data; C4, determining corresponding effective area data according to each effective thermal imaging local image data; C5, generates object pixel area data based on all valid area data combinations.
4. The vehicle-mounted AI camera visual recognition system according to claim 2, characterized in that: The image recognition strategy further comprises the following steps for generating edge-enhanced image data: D1, obtaining corresponding visible light image size data according to the visible light image data; D2, generating edge up-sampled image data by using a preset up-sampling algorithm according to the edge image data and the visible light image size data; D3, generating corresponding visible light grayscale image data according to the visible light image data using a preset grayscale image conversion algorithm; D4, generating edge enhanced image data according to the edge up-sampled image data and the visible light grayscale image data using a preset image fusion algorithm.
5. The vehicle-mounted AI camera visual recognition system according to claim 4, characterized in that: The image fusion algorithm comprises the following steps: E1, sample the image data on the edge to determine the corresponding non-zero value pixels and define them as valid edge pixels; E2, according to the pixel position of each valid edge pixel and the visible light grayscale image data, the corresponding pixel position is determined and defined as the edge enhancement pixel position; E3, obtaining the corresponding pixel grayscale value according to each edge enhancement pixel position; E4, multiplying the pixel grayscale value of each edge enhanced pixel position by a preset enhancement ratio to calculate a pixel update value; E5, replacing the corresponding pixel grayscale value according to the pixel update value of each edge enhanced pixel position.
