Whole vehicle intelligent cabin TSR enhanced pattern recognition system and method based on AI

Abstract

The invention provides a whole vehicle intelligent cabin TSR enhanced pattern recognition system and method based on AI, and the system is based on an intelligent cabin controller, and comprises an image obtaining module which is used for obtaining a traffic sign image through a whole vehicle front camera; the preprocessing module is used for preprocessing the acquired traffic sign image, including graying, noise reduction and enhancement operations, and specifically relates to basic enhancement, dynamic adaptation of special scenes and model iterative optimization; the pattern recognition module is used for inputting the preprocessed image into an AI-based pattern recognition model, performing feature extraction and analysis and outputting a recognition result of the traffic sign; and the post-processing module is used for post-processing the recognition result, judging the reliability of the recognition result according to the confidence coefficient, and further processing an unreliable result or prompting a driver. According to the invention, the accuracy and reliability of traffic sign recognition can be improved, more accurate traffic sign information is provided for a driver, and the driving safety is improved.

Description

Technical Field

[0001] This invention relates to the field of automotive intelligent cockpit technology, and in particular to an AI-based TSR enhanced graphic recognition system and method for intelligent cockpits in vehicles. Background Technology

[0002] With the development of automotive intelligence, the functions of intelligent cockpits are becoming increasingly diverse, among which traffic sign recognition (TSR) is an important safety assistance function. Although existing TSR technology can recognize some common traffic signs, its recognition accuracy needs to be improved in complex environments, such as when signs are blurred, obscured, or in poor lighting conditions.

[0003] Traditional image recognition methods struggle to effectively handle these complex situations and fail to provide drivers with accurate and reliable traffic sign information, thus impacting driving safety. Therefore, an AI-based enhanced image recognition technology is needed to improve TSR performance. Summary of the Invention

[0004] To address the aforementioned issues, this invention discloses an AI-based intelligent cockpit TSR enhanced graphic recognition system and method, which can improve the accuracy and reliability of traffic sign recognition, provide drivers with more accurate traffic sign information, and enhance driving safety.

[0005] The specific plan is as follows: An AI-based enhanced image recognition system for vehicle intelligent cockpit TSR, based on an intelligent cockpit controller, includes: The image acquisition module is used to acquire traffic sign images through the vehicle's front camera; The preprocessing module is used to preprocess the acquired traffic sign images, including operations such as grayscale conversion, noise reduction, and enhancement, specifically involving basic enhancement, dynamic adaptation for special scenes, and model iterative optimization. The image recognition module is used to input the preprocessed image into the AI-based image recognition model for feature extraction and analysis, and output the recognition results of traffic signs. The post-processing module is used to post-process the recognition results, determine the reliability of the recognition results based on the confidence level, and further process or prompt the driver for unreliable results.

[0006] Furthermore, in the image acquisition module, the front camera of the vehicle completes the raw acquisition of traffic sign images, i.e., TSR images, and establishes a high-speed communication link between the camera and the intelligent cockpit controller. During the intelligent cockpit power-on initialization phase, the cockpit controller supplies power to the camera through the internal power management module and initiates the communication handshake process: the cockpit controller sends configuration instructions based on the I2C protocol, including camera frame rate, exposure time, and white balance parameters; after receiving the configuration, the camera returns an ACK response and device status code. If the handshake fails three times in a row, the cockpit controller triggers a fault diagnosis mechanism, sends a "camera communication failure" message to the vehicle gateway through the CAN bus, and prompts the driver on the display screen.

[0007] Furthermore, in the preprocessing module, the basic enhancement process is as follows: For typical scenarios, the AI ​​algorithm library executes the following steps in sequence: Sharpening: Employing a multi-scale convolution kernel based on the Laplacian operator to enhance the edge features of traffic signs; Smoothing and Denoising: Combining Gaussian filtering and nonlocal mean denoising algorithms to remove sensor noise and motion artifacts while preserving details; Grayscale Enhancement: Expanding the dynamic range of the image through adaptive histogram equalization to solve the problem of insufficient contrast between signs and the background; Key Information Extraction: Using a semantic segmentation model to perform pixel-level classification of the image, retaining only the area where the traffic signs are located, removing irrelevant backgrounds, and compressing the data volume by more than 60%.

[0008] Furthermore, in the preprocessing module, the dynamic adaptation strategy for special scenarios is as follows: For extreme scenarios such as heavy fog, rainstorms, and nighttime, the AI ​​algorithm automatically switches processing logic by analyzing image features in real time: Low visibility scenarios: First, edge information is preserved through bilateral filtering, and then guided filtering is used to further smooth the blurred areas caused by fog; at the same time, the color distribution of the sign is statistically analyzed through point operation histogram, and the contrast of the color gamut is forcibly stretched to highlight the outline of the sign; Nighttime scenarios: The active light-emitting area of ​​the sign is identified by combining infrared channel data, and the overexposure of highlights caused by vehicle lights is suppressed through Gamma correction; Incomplete / occluded sign scenarios: The AI ​​algorithm calls the temporal correlation model to perform feature matching on the image sequence of 5 consecutive frames, aligns the incomplete area through motion compensation algorithm, and then reconstructs the complete sign image based on the completion module of generative adversarial network.

[0009] Furthermore, in the preprocessing module, the model iterative optimization mechanism is as follows: the preprocessing results of each vehicle and the driver's manually corrected data will be encrypted and uploaded to the cloud training platform. When the cumulative data volume reaches 100,000 records, the model update is triggered; the updated lightweight model is pushed to the cockpit controller via OTA.

[0010] Furthermore, the image recognition module employs an AI TSR recognition module, which compares image features with a local TSR database based on a pre-trained deep residual network and attention mechanism, and outputs recognition results, including the adjudication and response of multiple types of markers and cross-system collaborative control.

[0011] Furthermore, the adjudication and response for the multiple types of flags employ the AI ​​TSR adjudication module to perform differentiated processing based on flag attributes: 1) Warning, Prohibition, Work Area, and Notice categories: The adjudication module immediately generates the highest priority alarm signal, including the semantic description of the sign and the warning level, and simultaneously triggers a triple response: a red flashing icon and text prompt pop up on the cockpit display screen; a voice warning is played on the vehicle audio system; and control commands, including the sign location and required vehicle speed parameters, are sent to the vehicle gateway via the CAN FD bus. 2) Directional, Indication, and Tourist Area Information: The adjudication module packages the information into structured data and sends it to the navigation application via the in-cabin Ethernet: Indication information is used to optimize route planning; Tourist area information is combined with user preferences to recommend nearby services.

[0012] Furthermore, the cross-system collaborative control is as follows: after the vehicle gateway receives the TSR command sent by the cockpit, it forwards it to the ADAS controller. The ADAS executes specific control logic according to the sign type: speed limit sign: adjust the target speed of ACC; stop and yield sign: trigger AEB preparatory mode to shorten the braking response time; no overtaking sign: limit power output to prevent the driver from forcibly overtaking.

[0013] Furthermore, in the post-processing module, the MCU of the smart cockpit integrates TSR alarm information and vehicle status data, and presents them to the driver in the following ways: At the display level: Level 1 warning occupies 1 / 3 of the display screen area and is displayed with a red dynamic icon and countdown; Level 2 prompts are displayed as a top banner, which automatically collapses after 3 seconds; At the interaction level: the driver can retrieve historical TSR information through steering wheel buttons or voice commands, improving the ease of operation.

[0014] An AI-based method for enhancing TSR (Transmission Signal Recognition) images in a vehicle's intelligent cockpit includes the following steps: S1. The front camera of the vehicle performs raw acquisition of traffic sign images, i.e., TSR images; S2, the cockpit controller sequentially preprocesses, performs AI processing, and AI TSR recognition and adjudication on the acquired traffic sign images; S3. The vehicle gateway receives the TSR command sent from the cockpit and forwards it to the vehicle ADAS controller. The vehicle ADAS executes specific control logic according to the flag type. S4: The MCU in the smart cockpit integrates TSR alarm information and vehicle status data, which are displayed on the cockpit screen and accessible to the driver.

[0015] The beneficial effects of this invention are as follows: 1. This invention uses an AI-based image recognition model, which can accurately identify traffic signs in complex environments, thus improving the recognition accuracy and reliability of TSR.

[0016] 2. By preprocessing and postprocessing the images, the recognition results were further optimized, providing drivers with more accurate and reliable traffic sign information, which helps to improve driving safety.

[0017] 3. The system structure of the present invention is simple and easy to implement. It can be integrated into existing vehicle intelligent cockpits and has good application prospects. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the framework of the present invention. Detailed Implementation

[0019] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0020] like Figure 1 As shown, this invention provides an AI-based intelligent cockpit TSR enhanced graphic recognition system, comprising: 1. Construction of the image acquisition and hardware interaction module This system uses a front-view digital camera to acquire raw traffic sign images (TSR images). The camera employs a 12-megapixel ultra-high-definition sensor with a wide dynamic range (≥120dB) and global shutter function, preventing image blurring in high-speed driving scenarios. A high-speed communication link is established between the camera and the intelligent cockpit controller via the LVDS FPD-LINK III protocol, supporting a single-channel data transmission rate of 4Gbps, meeting the requirements for real-time image transmission at 1080P@60fps.

[0021] During the power-on initialization phase of the intelligent cockpit, the cockpit controller outputs a stable 12V DC power supply (ripple factor ≤5%) to the camera through the internal power management module and initiates the communication handshake process: The cockpit controller sends configuration commands based on the I2C protocol, including camera frame rate (default 30fps, automatically switches to 60fps in high-speed scenes), exposure time (dynamic range 100μs-10ms), white balance parameters, etc. After receiving the configuration, the camera returns an ACK response and a device status code (0x00 indicates normal, 0x01 indicates abnormal). If the handshake fails three times in a row, the cockpit controller triggers the fault diagnosis mechanism, sends a "camera communication failure" message to the vehicle gateway via the CAN bus, and displays a message to the driver on the screen.

[0022] This hardware interaction design ensures the stability and real-time performance of image acquisition, providing high-quality raw data input for subsequent AI processing.

[0023] 2. Image preprocessing mechanism based on AI adaptive algorithm After receiving the raw image, the cockpit controller calls the built-in AI image enhancement algorithm library for multi-dimensional preprocessing. The core logic lies in dynamically adapting the processing strategy according to the scene, specifically including: 2.1 Basic Enhancement Process For typical scenarios (such as sunny days and good lighting), the AI ​​algorithm library executes the following sequentially: Sharpening: Multi-scale convolution kernels (3×3, 5×5) based on the Laplacian operator are used to enhance the edge features of traffic signs (such as the circular outline of speed limit signs and the red border of prohibition signs). Smoothing and Denoising: Combining Gaussian filtering and nonlocal means denoising algorithms, sensor noise and motion artifacts are removed while preserving details; Grayscale enhancement: The dynamic range of the image is expanded by adaptive histogram equalization (CLAHE), which solves the problem of insufficient contrast between the logo and the background; Key information extraction: The image is classified at the pixel level using a semantic segmentation model (such as a lightweight version of U-Net), retaining only the region of origin (ROI) where traffic signs are located, and removing irrelevant backgrounds such as the sky and trees, compressing the data volume by more than 60% and improving the efficiency of subsequent processing.

[0024] 2.2 Dynamic Adaptation Strategies for Special Scenarios For extreme scenarios such as heavy fog, torrential rain, and nighttime, the AI ​​algorithm automatically switches processing logic by analyzing image features in real time (such as contrast, noise density, and average brightness). Low visibility scenarios (dense fog / heavy rain): The "multimodal filtering mechanism" is activated—first, edge information is preserved through bilateral filtering, and then guided filtering is used to further smooth the blurred areas caused by fog; at the same time, the color distribution of the sign is statistically analyzed through point operation histogram (such as the HSV color gamut range of the red prohibition sign), and the contrast of the color gamut is forcibly stretched to highlight the outline of the sign.

[0025] Nighttime scene: The "adaptive threshold segmentation" algorithm is activated, and the active light-emitting area of ​​the sign (such as a reflective film sign) is identified by combining infrared channel data (if the camera has integrated infrared function), and the overexposure of the highlights caused by the vehicle lights is suppressed by Gamma correction.

[0026] Scenes where the sign is incomplete or obscured: The AI ​​algorithm invokes a temporal correlation model to perform feature matching on a sequence of five consecutive frames (within 166ms), aligns missing regions using a motion compensation algorithm, and then reconstructs the complete sign image based on a generative adversarial network (GAN) completion module. For example, for a speed limit sign partially obscured by a truck, the model can generate a high-confidence completed image based on the complete shape of the sign in historical frames and the currently visible portion.

[0027] 2.3 Model Iterative Optimization Mechanism The system is designed with an "online learning closed loop": the preprocessing results for each vehicle and the driver's manual correction data (such as the driver correcting misidentified signs via voice or touch) are encrypted and uploaded to the cloud training platform. The model is updated when the cumulative data volume reaches 100,000 records. The updated lightweight model (≤500MB) is pushed to the cockpit controller via OTA, achieving a self-evolution capability of "the wider the scene coverage, the higher the preprocessing accuracy".

[0028] 3. TSR identification and adjudication and functional linkage logic The preprocessed image enters the AI ​​TSR recognition module, which is based on a pre-trained deep residual network (ResNet-50) and attention mechanism. It compares the image features with the local TSR database (containing 3,200 traffic sign templates from 120 countries and regions around the world) and outputs the recognition results (including sign type, semantic information, and confidence level ≥99.5%).

[0029] 3.1 Adjudication and Response Strategies for Multiple Types of Markers The AI ​​TSR adjudication module performs differential processing based on the flag attributes: Warning signs (e.g., Beware of falling rocks), prohibition signs (e.g., No Entry), work zone signs (e.g., Road Construction), and notice signs (e.g., Emergency Escape Lanes): The adjudication module immediately generates a highest-priority alarm signal, including a semantic description of the sign (such as "road construction 500 meters ahead") and a warning level (red, level one warning), and simultaneously triggers a triple response: A flashing red icon and text prompt appeared on the cockpit display screen; The car audio system plays voice warnings (such as "Please note that there is road construction ahead, it is recommended to reduce speed to 30km / h"). The system sends control commands (frame ID: 0x18F00100) to the vehicle gateway via the CAN FD bus, including parameters such as the marker location and required vehicle speed.

[0030] Directional signs (e.g., straight arrows), directional signs (e.g., highway exit information), and tourist area signs (e.g., directions to scenic spots): The adjudication module packages the information into structured data (JSON format) and sends it to the navigation application via the in-cabin Ethernet (SOME / IP protocol): Directional information is used to optimize route planning (e.g., "2km from Exit XX, it is recommended to merge into the rightmost lane"). Tourist area information is combined with user preferences (learned from historical data) to recommend nearby services (such as "XX scenic spot is 3km ahead, there are plenty of parking spaces available, do you want to navigate?").

[0031] 3.2 Cross-system collaborative control After receiving the TSR command sent from the cockpit, the vehicle gateway forwards it to the ADAS (Advanced Driver Assistance System) controller. The ADAS then executes specific control logic based on the flag type. Speed ​​limit sign: Adjust the target speed of ACC (adaptive cruise control) (e.g., after recognizing "speed limit 80km / h", automatically reduce the cruise speed from 100km / h to 80km / h). Stop sign: Triggers AEB (Automatic Emergency Braking) preparatory mode, shortening braking response time; No Overtaking Sign: This sign restricts power output to prevent drivers from forcibly overtaking.

[0032] 4. Human-computer interaction and information display The smart cockpit's MCU (microcontroller unit) integrates TSR alarm information and vehicle status data (such as current speed and distance to distance markers), presenting it to the driver in the following ways: Display level: "Multi-level priority rendering" is adopted - Level 1 warning (prohibition / work area) occupies 1 / 3 of the display screen area and is displayed with a red dynamic icon + countdown (such as "1000m to construction area → 500m → 100m"); Level 2 prompt (direction / tourist area) is displayed as a top banner and automatically collapses after 3 seconds.

[0033] Interaction level: Drivers can retrieve historical TSR information via steering wheel buttons or voice commands (such as "repeat the previous prompt"), improving ease of operation.

[0034] Through the technical solution of the present invention, the entire process from image acquisition and AI enhancement processing to multi-system linkage has been optimized, which significantly improves the accuracy of TSR recognition in complex scenarios (the actual recognition rate is ≥98.2% in rainy weather, which is 23.5% higher than the traditional solution), and provides core technical support for the active safety function of the intelligent cockpit.

[0035] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. An AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles, characterized in that, Based on the intelligent cockpit controller, including: The image acquisition module is used to acquire traffic sign images through the vehicle's front camera; The preprocessing module is used to preprocess the acquired traffic sign images, including grayscale conversion, noise reduction, and enhancement operations, specifically involving basic enhancement, dynamic adaptation for special scenes, and model iterative optimization. The image recognition module is used to input the preprocessed image into the AI-based image recognition model for feature extraction and analysis, and output the recognition results of traffic signs. The post-processing module is used to post-process the recognition results, determine the reliability of the recognition results based on the confidence level, and further process or prompt the driver for unreliable results.

2. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 1, characterized in that, In the image acquisition module, the front camera of the vehicle completes the raw acquisition of traffic sign images, i.e., TSR images, and establishes a high-speed communication link between the camera and the intelligent cockpit controller. During the intelligent cockpit power-on initialization phase, the cockpit controller supplies power to the camera through the internal power management module and initiates the communication handshake process: the cockpit controller sends configuration instructions based on the I2C protocol, including camera frame rate, exposure time, and white balance parameters; after receiving the configuration, the camera returns an ACK response and device status code. If the handshake fails three times in a row, the cockpit controller triggers a fault diagnosis mechanism, sends a "camera communication failure" message to the vehicle gateway through the CAN bus, and prompts the driver on the display screen.

3. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 1, characterized in that, In the preprocessing module, the basic enhancement process is as follows: For common scenarios, the AI ​​algorithm library executes the following steps in sequence: Sharpening: Using a multi-scale convolution kernel based on the Laplacian operator to enhance the edge features of traffic signs; Smoothing and Denoising: Combining Gaussian filtering and nonlocal means denoising algorithms, sensor noise and motion artifacts are removed while preserving details; Grayscale enhancement: Expanding the dynamic range of the image through adaptive histogram equalization solves the problem of insufficient contrast between the logo and the background; Key information extraction: The image is classified at the pixel level using a semantic segmentation model, retaining only the area where traffic signs are located and removing irrelevant background, thus compressing the data volume by more than 60%.

4. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 1, characterized in that, In the preprocessing module, the dynamic adaptation strategy for special scenarios is as follows: For extreme scenarios such as heavy fog, rainstorms, and nighttime, the AI ​​algorithm automatically switches processing logic by analyzing image features in real time: Low visibility scenarios: First, edge information is preserved through bilateral filtering, and then guided filtering is used to further smooth the blurred areas caused by fog; at the same time, the color distribution of the sign is statistically analyzed through point operation histogram, and the contrast of the color gamut is forcibly stretched to highlight the outline of the sign; Nighttime scenarios: The active light-emitting area of ​​the sign is identified by combining infrared channel data, and the overexposure of highlights caused by headlights is suppressed through Gamma correction; Incomplete / occluded sign scenarios: The AI ​​algorithm calls the temporal correlation model to perform feature matching on the image sequence of 5 consecutive frames, aligns the incomplete area through motion compensation algorithm, and then reconstructs the complete sign image based on the completion module of generative adversarial network.

5. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 1, characterized in that, In the preprocessing module, the model iterative optimization mechanism is as follows: the preprocessing results of each vehicle and the driver's manually corrected data will be encrypted and uploaded to the cloud training platform. When the cumulative data volume reaches 100,000 records, the model update is triggered; the updated lightweight model is pushed to the cockpit controller via OTA.

6. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 1, characterized in that, The image recognition module adopts an AI TSR recognition module, which compares image features with the local TSR database based on a pre-trained deep residual network and attention mechanism, and outputs recognition results, including the adjudication and response of multiple types of signs and cross-system collaborative control.

7. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 6, characterized in that, The adjudication and response for the various types of flags are performed using the AI ​​TSR adjudication module, which performs differentiated processing based on the flag attributes. 1) Warning, Prohibition, Work Area, and Notice categories: The adjudication module immediately generates the highest priority alarm signal, including the semantic description of the sign and the warning level, and simultaneously triggers a triple response: a red flashing icon and text prompt pop up on the cockpit display screen; a voice warning is played on the vehicle audio system; and control commands, including the sign location and required vehicle speed parameters, are sent to the vehicle gateway via the CAN FD bus. 2) Directional, Indication, and Tourist Area Information: The adjudication module packages the information into structured data and sends it to the navigation application via the in-cabin Ethernet: Indication information is used to optimize route planning; Tourist area information is combined with user preferences to recommend nearby services.

8. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 6, characterized in that, The cross-system collaborative control is as follows: After the vehicle gateway receives the TSR command sent by the cockpit, it forwards it to the ADAS controller. The ADAS executes specific control logic according to the sign type: speed limit sign: adjust the target speed of ACC; stop and yield sign: trigger AEB preparatory mode to shorten the braking response time; no overtaking sign: limit power output to prevent the driver from forcibly overtaking.

9. The AI-based intelligent cockpit TSR enhanced graphic recognition system for vehicles according to claim 1, characterized in that, In the post-processing module, the MCU of the smart cockpit integrates TSR alarm information and vehicle status data, and presents them to the driver in the following ways: At the display level: Level 1 warning occupies 1 / 3 of the display screen area and is displayed with a red dynamic icon and countdown; Level 2 prompts are displayed as a top banner, which automatically collapses after 3 seconds; At the interaction level: the driver can retrieve historical TSR information through steering wheel buttons or voice commands, improving the convenience of operation.

10. An AI-based method for enhancing TSR (Transmission Signal Recognition) image recognition in a vehicle's intelligent cockpit, based on the system described in any one of claims 1-9, comprising the following steps: S1. The front camera of the vehicle performs raw acquisition of traffic sign images, i.e., TSR images; S2, the cockpit controller performs preprocessing, AI processing, AITSR recognition and adjudication on the acquired traffic sign images in sequence; S3. The vehicle gateway receives the TSR command sent from the cockpit and forwards it to the vehicle ADAS controller. The vehicle ADAS executes specific control logic according to the flag type. S4: The MCU in the smart cockpit integrates TSR alarm information and vehicle status data, which are displayed on the cockpit screen and accessible to the driver.

Images