AI vision algorithm processing method based on vehicle-mounted intelligent terminal and electronic equipment

Abstract

The invention provides an AI vision algorithm processing method based on a vehicle-mounted intelligent terminal and electronic equipment. The AI vision algorithm processing method comprises the steps of collecting driving scene image data in real time based on the vehicle-mounted intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotations; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transform inference model, dynamically adjusting the number of attention heads and a feature channel compression ratio, and outputting an identification result including a driving environment key target; and performing cross validation based on an identification result and heterogeneous data of the vehicle-mounted sensor, and performing real-time closed-loop correction on a feature extraction weight and a reasoning threshold value of an AI visual algorithm to complete dynamic visual perception of the driving scene. According to the method, the defects that the driving scene cannot be dynamically adapted, the identification precision cannot be balanced and the closed-loop optimization of the visual algorithm cannot be realized at present are overcome.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an AI vision algorithm processing method and electronic device based on an in-vehicle intelligent terminal. Background Technology

[0002] With the rapid development of intelligent connected vehicles and autonomous driving technologies, in-vehicle AI vision systems, as a core module of environmental perception, directly impact driving safety in terms of recognition accuracy, real-time performance, and robustness. In-vehicle intelligent terminals collect image data of driving scenes through cameras and utilize AI vision algorithms to identify key targets such as lane lines, obstacles, and traffic signs, providing data support for autonomous driving decision-making and control. This has become a core research direction in current in-vehicle perception technology.

[0003] However, existing in-vehicle AI vision algorithms still face many technical bottlenecks in practical applications, making it difficult to meet the high requirements of complex driving scenarios: Insufficient scene adaptability and weak targeted feature enhancement: Driving scenarios are dynamic and varied, with significant differences between high-speed driving and urban congestion, bright sunlight and low light at night, and dry roads and rainy / foggy weather. The feature representation of key targets varies drastically in different scenarios. Existing technologies mostly adopt feature enhancement strategies with fixed parameters, failing to differentiate the enhancement of key target features (such as lane lines and nearby obstacles) based on scene priority. This leads to insufficient feature extraction and reduced target recognition in complex scenarios, thus affecting subsequent recognition accuracy.

[0004] Balancing computing power and recognition accuracy is challenging: The computing power of in-vehicle intelligent terminals is limited by installation space, power consumption budget, and other constraints, making it incomparable to cloud computing platforms. Existing AI vision algorithms mostly employ standard Transformer or deep learning models, which, while ensuring recognition accuracy, have a large number of model parameters and high computational complexity, making real-time inference difficult on in-vehicle terminals. Some lightweight improvement solutions simplify the model by using a fixed compression ratio, failing to dynamically adjust the model structure (such as the number of attention heads and feature channels) according to the real-time computing power status of the in-vehicle hardware. This results in increased inference latency under high hardware load or underutilization of computing power under low load, leading to wasted accuracy and an inability to achieve a dynamic optimal match between computing power and accuracy.

[0005] Insufficient robustness and lack of closed-loop optimization mechanisms: Vehicle-mounted visual recognition is susceptible to environmental interference (such as changes in lighting, occlusion, and road bumps), and the recognition results from single visual data are prone to misjudgment. Existing technologies do not dynamically adjust verification standards based on scene complexity and environmental factors; moreover, most solutions only stay at the level of verifying recognition results, without establishing a real-time closed-loop correction mechanism for the core parameters of AI visual algorithms (feature extraction weights, inference thresholds). This results in the algorithm performance not being able to adaptively optimize according to changes in actual driving scenarios, and recognition failures are prone to occur in extreme scenarios, affecting driving safety. At the same time, driving image data in some specific scenarios cannot be saved in a timely manner, making it easy to be damaged or at risk of data leakage, which is inconvenient for subsequent inspections.

[0006] Therefore, designing an AI vision algorithm processing method that can adapt to dynamic driving scenarios, balance the computing power of onboard hardware with recognition accuracy, and achieve closed-loop optimization of the algorithm through multi-source data fusion has become the key to solving the pain points of existing technologies and improving the robustness and real-time performance of onboard vision perception systems. This is of great significance to promoting the industrial application of autonomous driving technology. Summary of the Invention

[0007] The main objective of this invention is to provide an AI vision algorithm processing method and electronic device based on an in-vehicle intelligent terminal, aiming to overcome the shortcomings of current methods that cannot dynamically adapt to driving scenarios, cannot balance recognition accuracy, and cannot achieve closed-loop optimization of vision algorithms.

[0008] To achieve the above objectives, the present invention provides an AI vision algorithm processing method based on an in-vehicle intelligent terminal, comprising the following steps: Real-time acquisition of driving scene image data based on in-vehicle intelligent terminal; Multi-scale feature enhancement is performed on driving scene image data to generate enhanced feature maps with scene priority annotations; The enhanced feature map is input into an AI vision algorithm based on a lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs recognition results including key targets in the driving environment. Cross-validation is performed based on the recognition results and heterogeneous data from vehicle sensors. The feature extraction weights and inference thresholds of the AI ​​vision algorithm are corrected in real time in a closed loop to achieve dynamic visual perception of driving scenarios.

[0009] Furthermore, before performing multi-scale feature enhancement on the driving scene image data, the following steps are also included: The sampling frame rate and pixel cropping area of ​​driving scene image data are dynamically adjusted based on the vehicle's dynamic operating parameters to form driving scene image data adapted to the current driving scene.

[0010] Furthermore, key targets of the driving environment include lane markings, obstacles, and traffic signs.

[0011] Furthermore, multi-scale feature enhancement is performed on the driving scene image data to generate enhanced feature maps with scene priority annotations, including: The driving scene image data is downsampled to obtain multiple image feature layers of different scales, and the resolution of each scale image feature layer decreases by 1 / 2 gradient. Calculate the edge gradient values ​​of lane line regions in the image feature layers at each scale, and assign lane line feature weights based on the edge gradient values. Regions with edge gradient values ​​higher than a preset threshold are assigned a first weight, and regions with edge gradient values ​​lower than a preset threshold are assigned a second weight, with the first weight > the second weight. Predict the motion trend vector of traffic participants in a driving scenario, wherein the motion trend vector includes the direction of motion and the speed of motion; The lane line feature weights of image feature layers at each scale are weighted and fused with the motion trend vectors of traffic participants in the corresponding area; among them, traffic participants whose motion trend vectors point to the vehicle's driving path are assigned a third weight, and traffic participants whose motion trend vectors deviate from the vehicle's driving path are assigned a fourth weight, with the third weight being greater than the fourth weight. Based on the weighted fusion results, feature fusion and pixel-level enhancement are performed on image feature layers at various scales to generate enhanced feature maps with scene priority annotations.

[0012] Furthermore, the enhanced feature map is input into the lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs recognition results including key targets in the driving environment, including: Real-time collection of hardware computing power status parameters of in-vehicle intelligent terminals; Multiple sets of computing power threshold ranges are preset, and the corresponding number of attention heads is dynamically matched according to the computing power threshold range to which the current hardware computing power status parameters belong. Adjusting the feature channel compression ratio based on scene priority annotation of enhanced feature maps; The lightweight Transformer inference model uses a hierarchical attention mechanism to perform self-attention calculations on the adjusted feature channels, strengthens the feature association weights of high-priority regions, and outputs the category labels, location coordinates, and confidence scores of key targets in the driving environment to form a complete recognition result.

[0013] Furthermore, the hardware computing power status parameters include CPU utilization, GPU memory usage, and NPU computing throughput.

[0014] Furthermore, based on cross-validation of the recognition results and heterogeneous data from vehicle sensors, the feature extraction weights and inference thresholds of the AI ​​vision algorithm are corrected in real time using a closed-loop mechanism to achieve dynamic visual perception of driving scenarios, including: Based on the recognition accuracy and scene complexity of each target category in historical driving data, a real-time verification pass rate threshold is dynamically generated. Calculate the matching pass rate between the visual recognition results of each type of target and the heterogeneous data of the vehicle sensor. If the matching pass rate is higher than the real-time verification pass rate threshold, the recognition result is deemed valid and the current algorithm parameters are maintained. If the matching pass rate is lower than the real-time verification pass rate threshold, an intelligent closed-loop correction mechanism is activated: an attention-oriented weight allocation model is introduced, and the effective target features are verified based on heterogeneous data. The weights of the effective feature channels are dynamically allocated, and the weight coefficients of the effective feature channels are adaptively increased according to the proportion of the matching pass rate gap. Among them, the proportion of the matching pass rate gap = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold; the weight coefficients of the invalid feature channels are sparsified and suppressed through regularization constraints. The real-time verification pass rate threshold is dynamically adjusted using the ratio of the difference between the matching pass rate and the real-time verification pass rate threshold as the core factor and the ambient light intensity as the auxiliary factor. When the matching pass rate of a certain type of target is lower than the proportion threshold for multiple consecutive frames, the system automatically calls the preset scene-specific feature template to replace the current feature extraction logic, and feeds the corrected parameters back to the AI ​​vision algorithm in real time to achieve adaptive upgrade of dynamic visual perception in driving scenarios.

[0015] The present invention also provides an AI vision algorithm processing device based on an in-vehicle intelligent terminal, comprising: The acquisition unit is used to acquire real-time image data of driving scenes based on the vehicle-mounted intelligent terminal. The generation unit is used to perform multi-scale feature enhancement on driving scene image data and generate enhanced feature maps with scene priority annotations. The output unit is used to input the enhanced feature map into the AI ​​vision algorithm based on the lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs the recognition results including key targets in the driving environment. The correction unit is used to perform cross-validation based on the recognition results and heterogeneous data from vehicle sensors, and to perform real-time closed-loop correction of the feature extraction weights and inference thresholds of the AI ​​vision algorithm to complete the dynamic visual perception of driving scenarios.

[0016] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods described above.

[0018] This invention provides an AI vision algorithm processing method and electronic device based on an in-vehicle intelligent terminal, comprising: real-time acquisition of driving scene image data based on the in-vehicle intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotations; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, wherein the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs a recognition result including key targets in the driving environment; performing cross-validation based on the recognition result and heterogeneous data from in-vehicle sensors, and performing real-time closed-loop correction on the feature extraction weights and inference thresholds of the AI ​​vision algorithm to complete dynamic visual perception of the driving scene. In this invention, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority annotations, dynamically adapting to the driving scene; the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio to balance in-vehicle computing power and recognition accuracy; and the AI ​​vision algorithm is performed in real-time closed-loop correction based on the recognition result and heterogeneous data from in-vehicle sensors to achieve closed-loop optimization of the AI ​​vision algorithm. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the steps of an AI vision algorithm processing method based on an in-vehicle intelligent terminal in one embodiment of the present invention; Figure 2 This is a structural block diagram of an AI vision algorithm processing device based on an in-vehicle intelligent terminal in one embodiment of the present invention; Figure 3 This is a schematic block diagram of the structure of an electronic device according to an embodiment of the present invention.

[0020] The implementation, functional features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] It is particularly important to note that all technical steps, algorithm applications, and parameter settings in the technical solution of this application have clear technical objectives and application value. They do not utilize complex steps and algorithmic formulas to achieve simple functions. To provide detailed explanations of each step and avoid ambiguity, some conventional algorithms are used for illustration. However, this does not mean that the algorithms and technical features listed herein are the only way to implement the technical solution of this application, nor is it intended to limit the scope of protection of this application. This application is not a combination or stacking of the listed algorithms and technical features; its essence is to exemplify the implementation methods of this application to fully explain it. It does not pursue formal complexity by adding meaningless technical steps, nor does it involve the accumulation of technologies divorced from practical needs; it conforms to the conventional logic of technical improvement and design.

[0023] Reference Figure 1 One embodiment of the present invention provides an AI vision algorithm processing method based on an in-vehicle intelligent terminal, comprising the following steps: Step S1: Real-time acquisition of driving scene image data based on the vehicle-mounted intelligent terminal; Step S2: Perform multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotations; Step S3: Input the enhanced feature map into the AI ​​vision algorithm based on the lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs the recognition results including key targets in the driving environment. Step S4: Based on the recognition results and heterogeneous data from vehicle sensors, cross-validation is performed to real-time closed-loop correction of the feature extraction weights and inference thresholds of the AI ​​vision algorithm, thereby completing the dynamic visual perception of the driving scene.

[0024] In this embodiment, as described in step S1 above, the vehicle-mounted intelligent terminal is equipped with a high-definition visual acquisition module (such as a CMOS camera). This module acquires image data from all directions and continuously from the front, sides, and rear of the vehicle during its driving process, according to a preset initial sampling standard (such as a default 30fps frame rate and 1920×1080 pixel resolution). During the acquisition process, the camera maintains continuous operation based on the stable power supply of the vehicle's power supply. The image data is stored in the local cache unit of the vehicle terminal in real time, and the acquisition status information (such as whether the acquisition is normal and the data transmission rate) is synchronously fed back through the vehicle bus (such as the CAN bus) to ensure that the subsequent algorithm processing can obtain continuous and complete original image data to support it.

[0025] As described in step S2 above, for the collected original driving scene image data, firstly, 3-5 image feature layers of different scales can be constructed using Gaussian pyramid downsampling technology. The resolution of each feature layer decreases sequentially by a 1 / 2 gradient to achieve feature coverage of targets of different sizes (such as small traffic signs at a distance and large obstacles at close range). Subsequently, feature weights are assigned based on the edge gradient values ​​of lane line regions. Key lane line regions with edge gradient values ​​higher than a preset threshold are given higher weights to enhance the recognizability of lane line features. At the same time, the Kalman filter algorithm is used to predict the motion trend vectors (including motion direction and speed) of traffic participants such as pedestrians, non-motorized vehicles, and other vehicles, increasing the feature attention of high-risk traffic participant regions pointing to vehicle driving paths. Finally, the multi-scale feature layers are fused, and combined with the lane line weight allocation results and traffic participant motion trend analysis, scene priority labeling is performed on different regions in the image (such as key lane line regions as first-level priority, high-risk traffic participant regions as second-level priority, and ordinary scene regions as third-level priority). Finally, an enhanced feature map that takes into account the integrity of multi-scale features and scene priority guidance is generated, solving the problem of insufficient targeting of traditional feature enhancement.

[0026] As described in step S3 above, the generated enhanced feature map is input in real time to a lightweight Transformer inference model pre-deployed on the vehicle-mounted intelligent terminal. This model has been structurally optimized to adapt to the computing power constraints of the vehicle-mounted hardware, and its core lies in its dynamic adaptation capability: On the one hand, the model collects the hardware computing power status parameters of the vehicle-mounted terminal in real time (including CPU utilization, GPU memory usage, and NPU computing throughput), and dynamically adjusts the number of attention heads according to the computing power load; under low load, 8-12 attention heads are configured to ensure recognition accuracy, under medium load, 4-6 attention heads are configured to balance accuracy and speed, and under high load, 2-3 attention heads are configured to prioritize ensuring real-time performance; on the other hand, combined with the scene priority annotation results of the enhanced feature map, differentiated compression ratios are applied to the feature channels of different priority regions: no compression for the feature channels of the first-priority region, a 1:2 compression ratio for the second-priority region, and a 1:4 compression ratio for the third-priority region, thus reducing the amount of computation while retaining key features. The model performs self-attention calculation on the adjusted feature channels through a hierarchical attention mechanism to strengthen the feature association of high-priority regions. After processing by the classifier and regressor in the output layer, it finally outputs the category label, location coordinates and confidence of key targets in the driving environment (including lane lines, obstacles, traffic signs, pedestrians, etc.), forming a complete recognition result.

[0027] As described in step S4 above, real-time data from vehicle-mounted heterogeneous sensors are collected. These heterogeneous sensors include millimeter-wave radar (providing target distance and velocity data), ultrasonic sensors (providing near-range obstacle detection data), an inertial measurement unit (IMU, providing vehicle attitude angle and acceleration data), and an ambient light sensor (providing illumination intensity data), constructing a multi-source data support system. Subsequently, multi-source data association mapping rules are established. The target position coordinates in the AI ​​visual recognition results are spatially registered with the radar distance data through a coordinate transformation matrix. The target motion state and radar velocity data are time-aligned through timestamp synchronization. The near-range obstacle recognition results and ultrasonic data are verified for coverage by overlapping area ratio. Simultaneously, illumination intensity data is introduced as a visual recognition credibility correction coefficient. Next, a dynamic verification threshold matrix based on scene complexity is constructed, using a gradient boosting tree model. By combining historical driving data, a verification pass rate threshold for each target type is generated in real time, and the matching pass rate between the visual recognition result and the heterogeneous sensor data is calculated. If the matching pass rate is higher than the corresponding threshold, the current algorithm parameters are maintained; if it is lower than the threshold, a closed-loop correction is initiated: through an attention mechanism-guided weight allocation model, the weight of effective feature channels is adaptively increased according to the proportion of the matching pass rate gap, and ineffective feature channels are suppressed through L1 regularization. At the same time, the inference threshold is adjusted by a dual-factor linkage of difference ratio and illumination intensity. Finally, the corrected feature extraction weights and inference thresholds are fed back to the feature enhancement module in step S2 and the lightweight Transformer inference model in step S3 in real time to update the algorithm running parameters, realizing a full closed-loop optimization of acquisition-enhancement-inference-verification-correction, ensuring that the visual perception in driving scenarios can dynamically adapt to complex and changing driving environments, and improving perception accuracy and robustness.

[0028] In one embodiment, before performing multi-scale feature enhancement on the driving scene image data, the method further includes: The sampling frame rate and pixel cropping area of ​​driving scene image data are dynamically adjusted based on the vehicle's dynamic operating parameters to form driving scene image data adapted to the current driving scene.

[0029] In this embodiment, the in-vehicle intelligent terminal acquires vehicle dynamic operating condition parameters in real time via an in-vehicle bus (such as a CAN bus or LIN bus). These dynamic operating condition parameters include vehicle speed, steering angle, road surface roughness level (calculated from acceleration data collected by an inertial measurement unit, IMU), and driving scenario type (determined by preset scenario recognition rules, such as highways, urban roads, and rural roads). Based on preset operating condition-parameter mapping rules, the acquired raw image data is dynamically adapted and adjusted: Regarding sampling frame rate adjustment, when the vehicle speed is ≥80km / h (high-speed conditions), the sampling frame rate is increased to 35-40fps to capture fast-moving targets; when the vehicle speed is ≤30km / h (urban congestion or low-speed conditions), the sampling frame rate is reduced to 20-25fps to reduce invalid data acquisition and computational load; when the road surface roughness level is higher than a preset threshold (such as in severe bumpy scenarios), the sampling frame rate is maintained at a baseline of 30fps to ensure the stability of the image data. Regarding pixel cropping area adjustment, the vehicle trajectory is determined based on the steering angle. When driving in a straight line, the cropped area focuses on the main field of view ahead (horizontal cropping range is 20%-80% of the image width, and vertical cropping range is 30%-90% of the image height), preserving lane lines and distant target areas. When turning left, the cropping ratio of the left field of view is expanded (horizontal cropping range is adjusted to 0%-70% of the image width), enhancing image information of the left lane and obstacles. When turning right, the cropping ratio of the right field of view is expanded (horizontal cropping range is adjusted to 30%-100% of the image width), highlighting details of the right-side driving environment. Simultaneously, the cropping strategy is optimized based on the driving scene type: in urban road scenes, more image areas of nearby pedestrians and non-motorized vehicles are preserved; in highway scenes, the image coverage of distant traffic signs and vehicles ahead is extended. Through the dynamic adjustment of the frame rate and cropping area, redundant image information is filtered out, forming concise and efficient image data adapted to the current driving conditions, reducing computational complexity for subsequent feature enhancement and inference processing.

[0030] In one embodiment, key targets of the driving environment include lane lines, obstacles, and traffic signs.

[0031] In one embodiment, multi-scale feature enhancement is performed on driving scene image data to generate enhanced feature maps with scene priority annotations, including: The driving scene image data is downsampled to obtain multiple image feature layers of different scales, and the resolution of each scale image feature layer decreases by 1 / 2 gradient. Calculate the edge gradient values ​​of lane line regions in the image feature layers at each scale, and assign lane line feature weights based on the edge gradient values. Regions with edge gradient values ​​higher than a preset threshold are assigned a first weight, and regions with edge gradient values ​​lower than a preset threshold are assigned a second weight, with the first weight > the second weight. Predict the motion trend vector of traffic participants in a driving scenario, wherein the motion trend vector includes the direction of motion and the speed of motion; The lane line feature weights of image feature layers at each scale are weighted and fused with the motion trend vectors of traffic participants in the corresponding area; among them, traffic participants whose motion trend vectors point to the vehicle's driving path are assigned a third weight, and traffic participants whose motion trend vectors deviate from the vehicle's driving path are assigned a fourth weight, with the third weight being greater than the fourth weight. Based on the weighted fusion results, feature fusion and pixel-level enhancement are performed on image feature layers at various scales to generate enhanced feature maps with scene priority annotations.

[0032] In this embodiment, for the driving scene image data that has been adapted to the operating conditions, Gaussian pyramid downsampling technology is used to perform multi-scale feature extraction. First, the adjusted image is used as the baseline feature layer (layer 1). The baseline image is smoothed by Gaussian filtering to eliminate high-frequency noise interference. Then, the smoothed image is sampled every other row and column to obtain the second image feature layer (e.g., 960×540) with a resolution of half that of the baseline layer. The above "Gaussian smoothing + every other row and column sampling" process is repeated to generate the third (480×270), fourth (240×135), and fifth (120×67) image feature layers in sequence, finally forming a set of 3-5 feature layers of different scales (the specific number of layers can be dynamically configured according to the computing power and recognition accuracy requirements of the vehicle terminal). The resolution of each feature layer at each scale decreases strictly according to a 1 / 2 gradient, ensuring that the lower feature layers (high resolution) retain the detailed features of near targets, while the higher feature layers (low resolution) cover the global features of far targets. This achieves comprehensive feature coverage of key targets of different sizes and distances in driving scenarios (such as large obstacles at close range, vehicles in front at medium distance, and small traffic signs at long distance), laying a multi-scale feature foundation for subsequent accurate recognition.

[0033] For each scale image feature layer generated, edge information of the lane line region can be extracted separately: First, the feature layer images at each scale are converted to grayscale to simplify the calculation; then, Gaussian filtering is used to further suppress noise and avoid interference from non-lane line edges (such as road cracks and shadows); then, a dual thresholding method (high threshold and low threshold) is used to filter edge pixels, retaining the continuous edge contour of the lane line, and finally, the edge gradient value of each pixel is calculated (the larger the gradient value, the clearer the edge and the more significant the feature). A preset edge gradient threshold (e.g., 0.6, dynamically calibrated based on historical driving data and lane line recognition accuracy requirements) is used to assign weights to pixels in the lane line region at each scale feature layer: regions with edge gradient values ​​higher than the preset threshold (e.g., clear white solid lane lines, yellow dashed lane lines, lane line inflection points, and intersections) are identified as key lane line feature regions and assigned a first weight (e.g., 0.8-1.0); regions with edge gradient values ​​lower than the preset threshold (e.g., lane lines blurred by rain, lane line segments obscured by the vehicle in front, and lane lines in low-light conditions) are identified as weak lane line feature regions and assigned a second weight (e.g., 0.3-0.5), with the first weight strictly ensuring that it is greater than the second weight. This differentiated weight allocation enhances the recognizability of clear lane line features while ensuring that effective information about weak lane line features is not overlooked, solving the problem of insufficient prominence of key regions caused by equal weighting of lane line features in traditional feature processing.

[0034] Next, based on the acquired vehicle dynamic operating parameters (such as vehicle speed and steering angle) and the generated multi-scale image feature layer data, a motion state prediction model for traffic participants (including pedestrians, non-motorized vehicles, and other moving targets) is constructed using the Kalman filter algorithm. First, target detection and tracking are performed on traffic participants in each scale feature layer. Target positions are marked with bounding boxes, and the appearance features (such as size, outline, and color) and position features (such as pixel coordinates) of the targets are extracted. Then, based on the target position changes in consecutive frames (such as the first 5-10 frames), the instantaneous speed (such as pixels / frame, which can be converted to actual speed through coordinate transformation) and direction of movement (such as driving along the lane, crossing the road, parking on the side of the road, etc., quantized by direction vectors) of the traffic participants are calculated. Finally, through the prediction and update equations of the Kalman filter algorithm, the influence of image noise and detection errors on motion state judgment is eliminated, and the motion trend vector of each traffic participant is accurately output. This vector contains the quantified direction of movement (such as an angle of 0° with the vehicle's direction of travel indicating same direction, and 90° indicating lateral crossing) and speed, providing a dynamic basis for subsequent risk level determination and weight allocation.

[0035] For each generated image feature layer at each scale, a mapping is established between lane line feature weights and traffic participant motion trend vectors based on pixel regions: First, the pixel coverage area of ​​each traffic participant in the feature layer is determined, and the lane line feature weights corresponding to that area (i.e., the assigned first or second weights) are used as the base weights; then, based on the predicted traffic participant motion trend vectors, their risk level to the vehicle is determined: if the motion trend vector points to the vehicle's driving path (such as pedestrians crossing the road, vehicles illegally changing lanes from the side or rear into the vehicle's lane, or non-motorized vehicles traveling in the same direction but at significantly lower speeds than the vehicle), they are identified as high-risk traffic participants and assigned a third weight (e.g., 0.7-0.9); if the motion trend vector deviates from the vehicle's driving path (such as pedestrians walking towards the roadside, vehicles traveling in the opposite lane, or vehicles traveling in the same direction at speeds close to the vehicle), they are identified as low-risk traffic participants and assigned a fourth weight (e.g., 0.2-0.4), and it is strictly ensured that the third weight > the fourth weight. A weighted summation formula (e.g., fusion weight = lane line base weight × 0.6 + traffic participant risk weight × 0.4, the weight ratio can be optimized according to the actual scenario) is used to fuse the lane line feature weight of each pixel region with the corresponding traffic participant risk weight to obtain the comprehensive feature weight of each pixel region. This fusion process retains the importance of the core static feature of lane lines while fully considering the risk differences of the dynamic target of traffic participants, making the feature weight allocation more in line with driving safety requirements.

[0036] Finally, an attention fusion algorithm is used to perform cross-scale feature fusion on the obtained image feature layers at various scales: based on the comprehensive feature weight of each pixel region, the feature information of the low-level high-resolution feature layer (highlighting details) and the high-level low-resolution feature layer (highlighting the global picture) are weighted and superimposed. The higher the comprehensive feature weight, the richer the detail information is retained during the fusion process, ensuring that the features of key lane line areas and high-risk traffic participant areas are not lost. Subsequently, pixel-level enhancement processing is performed on the fused feature map: for areas with a comprehensive feature weight higher than the first fusion threshold (i.e., overlapping areas of key lane line areas and high-risk traffic participant areas), histogram equalization is used to improve pixel contrast and enhance feature recognition; for areas with a comprehensive feature weight between the second and first fusion thresholds (i.e., key lane line areas or single high-risk traffic participant areas), brightness and contrast are corrected and optimized; for areas with a comprehensive feature weight lower than the second fusion threshold (i.e., ordinary scene areas), only basic noise reduction processing is performed to avoid excessive enhancement of redundant features. Finally, scene priority labeling is performed on the enhanced feature map based on the comprehensive feature weight: for example, regions with a comprehensive feature weight ≥ 0.8 are labeled as first-priority (key lane lines + high-risk traffic participant regions), regions with a comprehensive feature weight ≤ 0.5 and < 0.8 are labeled as second-priority (single key feature regions), and regions with a comprehensive feature weight < 0.5 are labeled as third-priority (ordinary scene regions). In the end, an enhanced feature map that takes into account the integrity of multi-scale features, the saliency of key features, and the differentiation of scene risks is generated, providing high-quality input for the efficient and accurate inference of the subsequent lightweight Transformer inference model.

[0037] In one embodiment, the enhanced feature map is input into a lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs recognition results including key targets in the driving environment, including: Real-time collection of hardware computing power status parameters of in-vehicle intelligent terminals; Multiple sets of computing power threshold ranges are preset, and the corresponding number of attention heads is dynamically matched according to the computing power threshold range to which the current hardware computing power status parameters belong. Adjusting the feature channel compression ratio based on scene priority annotation of enhanced feature maps; The lightweight Transformer inference model uses a hierarchical attention mechanism to perform self-attention calculations on the adjusted feature channels, strengthens the feature association weights of high-priority regions, and outputs the category labels, location coordinates, and confidence scores of key targets in the driving environment to form a complete recognition result.

[0038] In this embodiment, the in-vehicle intelligent terminal is equipped with a hardware status monitoring module. This module establishes a real-time communication link with the terminal's core computing units such as the CPU, GPU, and NPU, and continuously collects hardware computing power status parameters through the vehicle system's underlying interfaces (such as the Linux kernel's proc file system and the computing power monitoring API provided by the GPU manufacturer). These computing power status parameters include three core dimensions: CPU utilization (the current load ratio of the computing cores, quantized range 0%-100%), GPU memory utilization (the proportion of memory used to store model parameters and intermediate feature data, quantized range 0%-100%), and NPU throughput (the number of tensor operations completed by the NPU per unit time, measured in TOPS). The collection frequency is synchronized with the image data processing frame rate (e.g., 20-40fps) to ensure that the computing power status parameters reflect the current hardware operating load in real time. This provides accurate computing power data support for subsequent dynamic adjustments to the model structure and avoids model adaptation inaccuracies caused by delayed computing power collection.

[0039] Next, based on the hardware performance limits of the in-vehicle intelligent terminal (such as the maximum number of CPU cores, GPU memory capacity, and NPU peak computing power) and the real-time requirements of AI vision algorithms, three sets of computing power threshold ranges are preset, forming a three-level load division system of low-load, medium-load, and high-load. For each set of computing power threshold ranges, a corresponding configuration of the number of attention heads is preset: 8-12 attention heads are configured in the low-load range to strengthen feature association calculation through more attention branches and improve recognition accuracy in complex scenarios; 4-6 attention heads are configured in the medium-load range to streamline some non-critical attention branches and reduce the amount of computation while keeping the accuracy loss controllable; 2-3 attention heads are configured in the high-load range to retain only the core attention branches and ensure that the inference process is not interrupted with minimal computational complexity. When the hardware computing power status parameters cross different threshold ranges, the lightweight Transformer inference model switches the number of attention heads in real time through a dynamic configuration interface. During the switching process, a parameter smooth transition strategy is adopted to avoid abrupt changes in the model output results and ensure recognition stability.

[0040] The lightweight Transformer inference model reads the generated enhanced feature map, analyzes the scene priority annotation information (Level 1 priority: key lane lines + high-risk traffic participant areas; Level 2 priority: single key feature areas; Level 3 priority: ordinary scene areas), and performs differentiated compression processing on the feature channels corresponding to different priority areas based on a preset priority-compression ratio mapping rule. Feature channel compression adopts a combination of depthwise separable convolution and channel pruning: For Level 1 priority areas, the feature channels are compressed at a 1:1 ratio, fully preserving the detailed information of all feature channels (such as lane line edge features and high-risk pedestrian contour features), ensuring that the recognition accuracy of core targets is not lost; for Level 2 priority areas, the feature channels are compressed at a 1:2 ratio, and redundant feature channels (such as non-critical texture feature channels) are removed through channel pruning, retaining only the core feature channels, reducing computational load while maintaining target recognizability; for Level 3 priority areas, the feature channels are compressed at a 1:4 ratio, and multi-channel features are fused into a small number of key channels through depthwise separable convolution, maximizing the simplification of redundant computation, and regularization constraints during compression ensure that feature information loss is controlled within 10%. Feature channel compression for different priority regions is performed independently. Spatial masking technology is used to divide the boundaries of each region to avoid mutual interference between features of different priority regions during the compression process, thereby achieving the differentiated optimization goal of maintaining accuracy in the core region and reducing the load in the ordinary region.

[0041] The encoder module of the lightweight Transformer inference model employs a hierarchical attention mechanism, collaboratively computing the adjusted attention head and compressed feature channels: the bottom attention layer focuses on local feature associations (such as the edges and texture features of the target), the middle attention layer strengthens target-level feature associations (such as the relative positional relationship between lane lines and obstacles), and the top attention layer constructs global scene associations (such as the motion coordination relationship between multiple targets). During self-attention computation, the model dynamically increases the feature association weights of high-priority regions based on scene priority annotations on the enhanced feature maps. The feature association weights of first-priority regions are increased by 20%-30%, the feature association weights of second-priority regions are increased by 10%-15%, and the basic association weights are maintained for third-priority regions, ensuring that the features of core targets are emphasized. Subsequently, the encoded feature vectors are input into the decoder module of the model. The decoder interacts with the encoder output features through a multi-head attention mechanism to accurately locate the feature representations of key targets in the driving environment. Then, the output layer's classifier and regressor work together: the classifier uses the Softmax activation function to map the feature vectors to a target category probability distribution, outputting category labels (such as lane lines, pedestrians, small cars, 60km / h speed limit signs, etc.); the regressor uses a bounding box regression algorithm to output the target's position coordinates in the image (represented by the x / y coordinates of the top-left and bottom-right corners of a rectangular bounding box); simultaneously, a confidence calculation module outputs the confidence score (quantization range 0-1, closer to 1 indicates more reliable recognition) for each recognition result based on feature matching degree and historical recognition accuracy. Finally, the model structurally encapsulates the category labels, position coordinates, and confidence scores to form a complete recognition result, which is transmitted in real-time via the vehicle bus to the subsequent cross-validation module, providing accurate visual recognition data support for closed-loop correction.

[0042] In one embodiment, the hardware computing power status parameters include CPU utilization, GPU memory usage, and NPU computing throughput.

[0043] In one embodiment, cross-validation is performed based on the recognition results and heterogeneous data from vehicle sensors to perform real-time closed-loop correction of the feature extraction weights and inference thresholds of the AI ​​vision algorithm, thereby completing dynamic visual perception of the driving scene, including: Based on the recognition accuracy and scene complexity of each target category in historical driving data, a real-time verification pass rate threshold is dynamically generated. Calculate the matching pass rate between the visual recognition results of each type of target and the heterogeneous data of the vehicle sensor. If the matching pass rate is higher than the real-time verification pass rate threshold, the recognition result is deemed valid and the current algorithm parameters are maintained. If the matching pass rate is lower than the real-time verification pass rate threshold, an intelligent closed-loop correction mechanism is activated: an attention-oriented weight allocation model is introduced, and the effective target features are verified based on heterogeneous data. The weights of the effective feature channels are dynamically allocated, and the weight coefficients of the effective feature channels are adaptively increased according to the proportion of the matching pass rate gap. Among them, the proportion of the matching pass rate gap = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold; the weight coefficients of the invalid feature channels are sparsified and suppressed through regularization constraints. The real-time verification pass rate threshold is dynamically adjusted using the ratio of the difference between the matching pass rate and the real-time verification pass rate threshold as the core factor and the ambient light intensity as the auxiliary factor. When the matching pass rate of a certain type of target is lower than the proportion threshold for multiple consecutive frames, the system automatically calls the preset scene-specific feature template to replace the current feature extraction logic, and feeds the corrected parameters back to the AI ​​vision algorithm in real time to achieve adaptive upgrade of dynamic visual perception in driving scenarios.

[0044] In this embodiment, a historical driving database is first constructed. This database stores massive amounts of driving data under different operating conditions (such as highways, urban roads, and rural roads) and different environments (such as sunny days, rainy / foggy days, and nighttime). This includes visual recognition results for each target category (lane lines, distant obstacles, near obstacles, and traffic signs), heterogeneous data from vehicle sensors, actual road condition annotations, and scene complexity parameters (such as road curvature, congestion level, and light intensity level). Based on this database, a gradient boosting tree model is used for training. The model input is the target category plus the scene complexity parameters, and the output is the optimal validation pass rate benchmark value under this combination. During real-time driving, the onboard intelligent terminal collects the current scene complexity parameters in real time through the scene recognition module (road curvature ≥ 0.05 rad / m is considered high complexity, 0.02-0.05 rad / m is medium complexity, and < 0.02 rad / m is low complexity; congestion levels are divided into no congestion, light congestion, moderate congestion, and heavy congestion based on vehicle density). The current target category plus the real-time scene complexity parameters are input into the trained GBDT model to dynamically generate real-time verification pass rate thresholds for each target category. Scene complexity and verification pass rate thresholds are negatively correlated: for every level increase in scene complexity, the corresponding target's real-time verification pass rate threshold is lowered by 3%-5%, ensuring that the threshold is adapted to the recognition difficulty in complex scenes. For example, in high-complexity scenarios (sharp curves + heavy congestion), the real-time verification pass rate threshold for lane lines is lowered from the baseline of 85% to 75%-80%, avoiding misjudgment of recognition failure due to reasonable matching deviations caused by scene complexity.

[0045] Next, real-time data from vehicle-mounted heterogeneous sensors are collected. These sensors include a millimeter-wave radar (providing target distance and radial velocity data), an ultrasonic sensor (providing near-range obstacle detection data within 3 meters), an inertial measurement unit (IMU, providing vehicle attitude angle and acceleration data), and an ambient light sensor (providing illumination intensity data), forming a multi-source data verification system. A correlation mapping rule between the visual recognition results and the heterogeneous data is established: the image coordinates of the visually recognized target are converted to vehicle coordinates using a coordinate transformation matrix (pre-generated based on sensor calibration parameters), and spatially registered with the target distance data from the millimeter-wave radar. A spatial position deviation ≤ 0.5 meters is considered a spatial match. The motion state of the visually recognized target (vehicle speed and direction calculated based on consecutive frame images) is time-series aligned with the radar radial velocity data using timestamp synchronization technology. A velocity deviation ≤ 5 km / h is considered a motion state match. For near-range obstacles, an overlap area ≥ 60% between the visual recognition results and the ultrasonic detection data is considered a coverage match. Simultaneously, ambient light intensity data is introduced as a matching reliability correction coefficient. When the light intensity is <300 lux (low light), the correction coefficient is 0.8-0.9; when the light intensity is 10000 lux < 50000 lux (strong light), the correction coefficient is 0.9-0.95; and under other lighting conditions, the correction coefficient is 1.0. The matching pass rate is calculated as follows: (Number of targets that match spatially and in motion / coverage ÷ Total number of targets identified in this category) × Confidence correction coefficient × 100%. The calculated matching pass rate is compared with the generated real-time verification pass rate threshold. If the matching pass rate is higher than the threshold, the current visual recognition result is considered reliable. The feature extraction weights and inference thresholds of the AI ​​visual algorithm remain at their current configurations without adjustment, ensuring the consistency of algorithm parameters in stable scenarios.

[0046] When the matching pass rate of a certain type of target is lower than the corresponding real-time verification pass rate threshold, the recognition accuracy of that type of target is deemed insufficient, and an intelligent closed-loop correction mechanism is immediately activated. First, valid target features are verified through heterogeneous data filtering: for example, obstacle features confirmed by millimeter-wave radar but missed by visual recognition, and clear lane line features verified by IMU, are both determined to be valid target features; while false features repeatedly rejected by sensor data (such as road shadows misidentified as obstacles) are determined to be invalid target features. An attention-based weight allocation model is introduced, prioritizing the feature channels corresponding to valid target features for optimization. The matching pass rate gap ratio is calculated (formula: matching pass rate gap ratio = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold). The weight coefficient of the valid feature channel is adaptively increased according to this gap ratio, with an increase of gap ratio × 0.3-0.5. For example, if the real-time verification pass rate threshold is 90% and the actual matching pass rate is 72%, then the gap ratio is 20%. The weight coefficient of the effective feature channel is increased by 20% × 0.3 - 0.5 = 6% - 10%, ensuring that the larger the gap, the stronger the reinforcement of the effective features. For the invalid feature channels, L1 regularization constraint is used for sparsity suppression, with a suppression magnitude of gap ratio × 0.4 - 0.6. By reducing the weight coefficient of the invalid feature channels, their interference with subsequent feature extraction is reduced, avoiding repeated misjudgments. During the weight adjustment process, normalization is used to ensure that the sum of the weight coefficients of all feature channels is 1, maintaining the stability of the algorithm parameters.

[0047] While adjusting the weights, the real-time verification pass rate threshold is dynamically optimized using a dual-factor linkage adjustment strategy of core factor + auxiliary factor. The core factor is the ratio of the difference between the matching pass rate and the real-time verification pass rate threshold, calculated as: Difference Ratio = (Real-time Verification Pass Rate Threshold - Matching Pass Rate) / Real-time Verification Pass Rate Threshold. The auxiliary factor is the light intensity data collected by the ambient light sensor, divided into three levels according to light intensity: low light level (<300 lux), normal light level (300-10000 lux), and strong light level (>10000 lux). The threshold adjustment rule is set as follows: for every 5% increase in the difference ratio, the real-time verification pass rate threshold is reduced by 0.03-0.05; simultaneously, combined with auxiliary factor correction: an additional reduction of 0.02-0.03 for low light level, an additional reduction of 0.01-0.02 for strong light level, and no additional adjustment for normal light level. For example, if the difference ratio is 10% and the current environment is low light, the threshold reduction would be (10% ÷ 5%) × (0.03 - 0.05) + (0.02 - 0.03) = 0.08 - 0.13. Simultaneously, a threshold constraint range is set: the adjusted threshold should not be lower than 0.6 (to avoid a surge in false alarms due to an excessively low threshold) and should not be higher than 0.9 (to avoid overcorrection due to an excessively high threshold). If the adjusted threshold exceeds this range, it will automatically adjust to the nearest constraint boundary to ensure the rationality and safety of the threshold adjustment.

[0048] Then, a preset percentage threshold (e.g., 60%) and a consecutive frame threshold (e.g., 3-5 frames) are used to monitor the matching pass rate of each target category in real time. If the matching pass rate of a certain target is below 60% for 3-5 consecutive frames, it is determined that the current feature extraction logic has failed in this scenario (e.g., lane line feature extraction fails in low-light nighttime scenarios, or obstacle features are obscured in rainy or foggy weather), and an emergency correction mechanism is immediately triggered. The in-vehicle intelligent terminal pre-stores multiple sets of scene-specific feature templates, including lane line templates for low-light nighttime scenarios, obstacle templates for rainy or foggy weather, and traffic sign templates for strong light and backlight. Each template is optimized for the target feature characteristics of a specific scenario (e.g., the nighttime template enhances the brightness and contrast features of lane lines, and the rainy or foggy weather template enhances the contour edge features of obstacles). Based on the current scene type (determined comprehensively through ambient light sensors, IMU, and image data), the corresponding scene-specific feature template is automatically called to replace the original general feature extraction logic, quickly improving the recognition of target features. Simultaneously, the corrected feature extraction weights and adjusted real-time verification pass rate thresholds are fed back to the multi-scale feature enhancement and lightweight Transformer inference model in real time, updating the weight allocation rules in the feature enhancement process and the inference threshold determination criteria of the model. The corrected parameters take effect immediately in subsequent frame image processing, forming a fully closed-loop optimization chain of verification-correction-feedback-iteration. This enables the AI ​​vision algorithm to adapt to changes in different complex scenarios, achieving continuous upgrades in dynamic visual perception of driving scenarios and significantly improving the perception robustness and reliability in extreme scenarios.

[0049] In one embodiment, the method further includes: Based on the typical interference features of different scenarios stored in the scenario meta knowledge base, interference samples matching the current driving scenario are generated, and they are fused with the enhanced feature map to form an anti-interference enhanced feature map. The anti-interference enhancement feature map is input into the lightweight Transformer inference model. The consistency between the heterogeneous data of the vehicle sensor and the recognition result is used as a self-supervised signal. When the matching pass rate is higher than the preset threshold, the corresponding anti-interference enhancement feature map is stored in the high-quality feature pool, and the convolution kernel parameters of the feature extraction network are optimized. The optimized parameters are synchronously updated to the corresponding scene in the scene meta-knowledge base. The similarity between the current driving scene and each scene in the scene meta knowledge base is calculated in real time. When the similarity is lower than the similarity threshold, it is determined to be a new scene. The optimized parameters of the most similar scene in the scene meta knowledge base are transferred. Combined with the anti-interference enhancement feature map of the same scene in the high-quality feature pool, the new scene adaptation is completed after multi-frame fine-tuning. The adapted new scene parameters and anti-interference enhancement feature map are synchronously stored in the scene meta knowledge base and the high-quality feature pool. The output distribution of the feature extraction network is monitored in real time. When the output distribution offset is greater than the threshold, the anti-interference enhancement feature map in the high-quality feature pool and the optimal parameters in the scene meta knowledge base are automatically called to calibrate the feature extraction network online. During the calibration process, a new anti-interference enhancement feature map is generated simultaneously.

[0050] In this embodiment, the scene meta-knowledge base pre-stores massive amounts of driving scene data, covering typical scenarios such as sharp bends in mountainous areas, urban roundabouts, highway sections, rainy and foggy days, and strong light at night. Each scenario is associated with corresponding typical interference features (such as fogging and occlusion features in rainy and foggy scenes, glare features in strong light at night, and blurred target outlines caused by road shadows). During the feature enhancement stage, the in-vehicle intelligent terminal collects key parameters of the current driving scene in real time (including ambient light intensity, weather conditions, road type, and vehicle speed). Using a scene matching algorithm, it retrieves the typical scene with the highest similarity to the current scene from the scene meta-knowledge base and extracts the interference features corresponding to that typical scene. Subsequently, the retrieved typical interference features and the generated original enhanced feature map are input into the lightweight adversarial generative network. The adversarial generative network simulates the real interference patterns in the current scene through the generator (such as adjusting the brightness and range of glare according to real-time light intensity, and adjusting the fog concentration distribution according to weather conditions) to generate interference samples that highly match the current driving scene. Then, the interference samples and the original enhanced feature map are fused at the pixel level through a fusion algorithm to ensure that the original effective features are not covered by interference information. Finally, an anti-interference enhanced feature map with both anti-interference ability and feature integrity is formed, providing more robust input data for the subsequent inference model.

[0051] The generated anti-interference enhanced feature map is input into the lightweight Transformer inference model in real time. After the model outputs the recognition results (category label, position coordinates, confidence score) of key targets in the driving environment, real-time data from vehicle-mounted heterogeneous sensors (millimeter-wave radar, ultrasonic sensors, IMU, etc.) are collected simultaneously to construct a multi-source data verification system. The consistency between the heterogeneous data from vehicle-mounted sensors and the visual recognition results is used as a self-supervised signal, and the matching pass rate between the two is calculated through preset matching rules. For example, the position coordinates of the visually recognized target are spatially registered with the radar distance data, and a position deviation ≤ 0.5 meters is judged as a spatial match; the motion state of the visually recognized target is temporally aligned with the radar speed data, and a speed deviation ≤ 5 km / h is judged as a motion state match. The matching pass rate = (number of targets that are spatially matched and have matching motion states ÷ total number of recognized targets in this category) × 100%. A preset matching pass rate threshold (e.g., 90%) is set. When the matching pass rate of a certain type of target is higher than this threshold, it is determined that the corresponding anti-interference enhancement feature map contains high reliability features. It is automatically stored in the high-quality feature pool of the vehicle terminal, and the convolution kernel parameters of the feature extraction network corresponding to the feature map are optimized. The contrastive learning algorithm is used to iteratively adjust the weight coefficients of the convolution kernel with the anti-interference enhancement features in the high-quality feature pool as positive samples and the historical low matching rate features as negative samples, so that the feature extraction network can more accurately capture anti-interference and high-confidence target features. After optimization, the convolution kernel parameters are updated in real time to the feature extraction-inference parameter set of the corresponding scene in the scene meta knowledge base through the data synchronization interface to realize the dynamic iteration of the knowledge base.

[0052] The in-vehicle intelligent terminal constructs a feature vector for the current driving scene by collecting scene parameters in real time (road type, environmental interference type, traffic participant density, etc.). It then uses a cosine similarity algorithm to calculate the similarity between this feature vector and all typical scene feature vectors in the scene meta-knowledge base. A preset similarity threshold (e.g., 60%) is set. When the calculated similarity is lower than this threshold, the current scene is determined to be a new scene not stored in the knowledge base (e.g., a new type of road with new traffic facilities, or a mixed scene under special weather conditions). A meta-learning adaptation mechanism is then initiated: First, the typical scene with the highest similarity to the current new scene is selected from the scene meta-knowledge base. The optimized convolutional kernel parameters of the feature extraction network and the inference model configuration parameters of this typical scene are then transferred as the initial parameters for the new scene. Subsequently, an anti-interference enhancement feature map consistent with the interference type of the new scene is retrieved from a high-quality feature pool. Combined with the first 3-5 frames of image data of the current new scene, the initial parameters are quickly fine-tuned. Through the gradient update strategy of the meta-learning algorithm, the parameters can be adapted to the feature distribution of the new scene with only a few iterations. After the adaptation is completed, the feature vector of the new scene, the optimized parameter set and the corresponding anti-interference enhancement feature map are simultaneously stored in the scene meta-knowledge base and the high-quality feature pool to realize the real-time supplementation of the new scene data. This ensures that when encountering the same scene again, the adapted parameters can be directly called without repeating the adaptation process.

[0053] The output distribution of the feature extraction network is monitored in real time. The feature output distribution of the current frame is compared with the standard feature distribution of the same scene in the high-quality feature pool, and the distribution offset between the two is calculated. A preset offset threshold is set. When the offset exceeds the threshold, it is determined that the feature extraction network has experienced feature drift due to long-term operation (such as a decrease in feature extraction accuracy caused by hardware aging or changes in environmental interference patterns). An online calibration process is immediately initiated: the anti-interference enhanced feature map with the highest matching pass rate in the same scene is retrieved from the high-quality feature pool, and the optimal parameters for the corresponding scene are extracted from the scene meta-knowledge base. Both are used as calibration samples and calibration benchmarks. The convolution kernel parameters and activation function thresholds of the feature extraction network are fine-tuned through the backpropagation algorithm so that the network output distribution is close to the standard distribution again. During the calibration process, the interference sample generation process of the lightweight adversarial generative network is started simultaneously: based on the network output defects after the current feature drift (such as insensitivity to feature extraction of a certain type of interference), targeted new interference samples are generated (such as strengthening the feature intensity of this type of interference). The new interference samples are fused with the current frame image to form a new anti-interference enhancement feature map. The new feature map is synchronously stored in the high-quality feature pool to feed back into the interference sample generation process, so that the subsequently generated interference samples are more in line with the optimization requirements after network calibration, forming a closed-loop feedback link of calibration-new feature generation-knowledge base update.

[0054] In one embodiment, the method further includes: When the recognition result includes a specific target, the image within the corresponding time period is extracted from the driving scene image data and used as the target image data; Obtain the identification information of the vehicle-mounted intelligent terminal and the version information of the AI ​​vision algorithm; generate a change factor based on the identification information, and change the array elements of the preset key character array to obtain the changed key character array; A curve is generated based on each numeric character in the version information; A first subkey is generated based on the curve and the changed key character array; a second subkey is generated based on the first subkey and the preset key character array. The first subkey and the second subkey are combined to generate a combined key, which is used to independently encrypt and store the target image data.

[0055] In this embodiment, after receiving the key target recognition results of the driving environment output by the lightweight Transformer inference model, the vehicle-mounted intelligent terminal determines the target category of the recognition results in real time. The specific target is a preset high-safety-priority target (such as obstacles at accident scenes, pedestrians illegally crossing the road, new traffic signs, sudden road surface anomalies, etc.). The image data corresponding to this type of target has traceability, analysis, or evidence value and needs to be stored separately with encryption. When the recognition result contains the above-mentioned specific target, the recognition frame time sequence information corresponding to the specific target is automatically extracted (including the frame number of the first appearance of the target and the frame interval that continues to exist), and continuous image frames within the corresponding time period are extracted from the driving scene image data based on the time sequence information. The extraction range is centered on the specific target and extends to a preset pixel range around the target (such as a horizontal extension of 200 pixels and a vertical extension of 150 pixels) and a preset duration to ensure complete inclusion of the target and surrounding related scene information. The extracted continuous image frames are integrated into target image data and temporarily stored in the temporary cache area of ​​the vehicle-mounted terminal, awaiting encryption processing.

[0056] The hardware information reading module of the in-vehicle intelligent terminal obtains the terminal's unique identification information (such as device serial number, MAC address, and built-in identifier of the hardware encryption chip) and the currently running AI vision algorithm version information (including major version number, minor version number, and revision number). The identification information is unique and used to distinguish different in-vehicle terminals, while the version information is used for key synchronization after algorithm iterations. A change factor is generated based on the obtained identification information: a hash operation is performed on the identification information, and the first 16 characters of the hash result are extracted as the base factor. This base factor is then combined with the last 8 digits of the current system timestamp and XORed to generate the final change factor. A fixed-length key character array (e.g., a 64×64 array, with array elements consisting of letters, numbers, and special symbols, pre-stored in the secure storage area of ​​the in-vehicle terminal) is preset, and the generated change factor serves as the rule for changing array elements. The replacement position of the array element is determined by the ASCII code value of the change factor (e.g., the sum of the ASCII code values ​​is modulo 64 to obtain the replacement row number and column number), and the array element at the corresponding position is replaced with the corresponding character of the change factor to generate a change key character array uniquely bound to the current vehicle terminal, ensuring the device specificity of the key basic information.

[0057] Furthermore, each numeric character in the AI ​​vision algorithm version information is extracted, ignoring non-numeric characters in the version number. A two-dimensional coordinate point set is constructed using the order of the numeric characters as the x-axis and the numerical value of the numeric characters as the y-axis. A Bézier curve fitting algorithm is used to simulate a curve on this coordinate point set. During the fitting process, the order of the curve is determined by the number of numeric characters in the version number, and interpolation operations are used to supplement the intermediate nodes of the curve, generating a continuous and smooth two-dimensional curve. The shape of the curve is strongly correlated with the version information, ensuring that different algorithm versions correspond to different curve characteristics.

[0058] Next, a first subkey is generated based on the generated curve and the changed key character array. In one embodiment, the curve is mapped onto a two-dimensional plane of the changed key character array (the horizontal axis corresponds to the array column number, and the vertical axis corresponds to the array row number). Characters at the intersection of the curve and the array, as well as array elements within a preset range (such as adjacent 3×3 areas) around the intersection, are extracted. These characters are arranged in the order of the curve's direction to form a 32-bit string, which is the first subkey. Its generation process is simultaneously associated with the terminal identifier and the algorithm version, exhibiting dual association. Next, a second subkey is generated based on the first subkey and a preset key character array: this can be achieved by performing a bitwise XOR operation (characters participate in the operation using ASCII code values), obtaining the XOR result, and then performing Base64 encoding on the result. The first 32 bits of the encoded result are then used as the second subkey. The second subkey, through association with the original preset key, further enhances the key's complexity and anti-cracking capability.

[0059] Finally, the generated first subkey and second subkey are concatenated in the order of first subkey + second subkey to form a combined key. This combined key integrates the vehicle terminal identifier, AI vision algorithm version, and preset key information, possessing uniqueness, relevance, distinctiveness, and dynamism. This combined key is used as the encryption key to encrypt the target image data in the temporary buffer. During encryption, the target image data is encrypted in blocks (each block is 4KB), and a checksum (such as CRC32 checksum) is added to each block to ensure the integrity of data transmission and storage. The encrypted target image data is stored in the encrypted storage partition of the vehicle terminal (this partition has hardware encryption protection and can only be decrypted using the corresponding combined key). It also records the encryption time, specific target type, and basic information required for generating the combined key (identification information digest, version information) for subsequent traceability and decryption, achieving secure and independent storage of specific target image data.

[0060] In the above embodiments, this application incorporates some existing algorithms and technical features for explanation and description to make the specification more detailed, clear, and complete, thus complying with the provisions of the Patent Law. However, this is not achieved by using a series of complex steps and algorithmic formulas, nor by complicating the technical solution, nor by combining or stacking conventional or simple features. The existing algorithms and technical features listed are for the purpose of disclosing the specific implementation methods of each step of this application (not to limit this application) and to avoid situations where this application cannot be implemented.

[0061] Reference Figure 2 In another embodiment of the present invention, an AI vision algorithm processing device based on an in-vehicle intelligent terminal is also provided, comprising: The acquisition unit is used to acquire real-time image data of driving scenes based on the vehicle-mounted intelligent terminal. The generation unit is used to perform multi-scale feature enhancement on driving scene image data and generate enhanced feature maps with scene priority annotations. The output unit is used to input the enhanced feature map into the AI ​​vision algorithm based on the lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs the recognition results including key targets in the driving environment. The correction unit is used to perform cross-validation based on the recognition results and heterogeneous data from vehicle sensors, and to perform real-time closed-loop correction of the feature extraction weights and inference thresholds of the AI ​​vision algorithm to complete the dynamic visual perception of driving scenarios.

[0062] In this embodiment, the specific implementation of each unit in the above device embodiment is described in the above method embodiment, and will not be repeated here.

[0063] Reference Figure 3 The present invention also provides an electronic device, the internal structure of which can be as follows: Figure 3 As shown, the electronic device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor, designed as a computer, provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the electronic device stores the data corresponding to this embodiment. The network interface of the electronic device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0064] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present invention and do not constitute a limitation on the electronic devices to which the present invention is applied.

[0065] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0066] In summary, the AI ​​vision algorithm processing method and electronic device based on an in-vehicle intelligent terminal provided in this embodiment of the invention include: real-time acquisition of driving scene image data based on the in-vehicle intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotations; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, wherein the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs a recognition result including key targets in the driving environment; performing cross-validation based on the recognition result and heterogeneous data from in-vehicle sensors, and performing real-time closed-loop correction on the feature extraction weights and inference thresholds of the AI ​​vision algorithm to complete the dynamic visual perception of the driving scene. In this invention, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority annotations, dynamically adapting to the driving scene; the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio to balance in-vehicle computing power and recognition accuracy; and real-time closed-loop correction is performed on the AI ​​vision algorithm based on the recognition result and heterogeneous data from in-vehicle sensors to achieve closed-loop optimization of the AI ​​vision algorithm.

[0067] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the present invention and embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0068] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0069] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An AI vision algorithm processing method based on an in-vehicle intelligent terminal, characterized in that, Includes the following steps: Real-time acquisition of driving scene image data based on in-vehicle intelligent terminal; Multi-scale feature enhancement is performed on driving scene image data to generate enhanced feature maps with scene priority annotations; The enhanced feature map is input into an AI vision algorithm based on a lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs recognition results including key targets in the driving environment. Cross-validation is performed based on the recognition results and heterogeneous data from vehicle sensors. The feature extraction weights and inference thresholds of the AI ​​vision algorithm are corrected in real time in a closed loop to achieve dynamic visual perception of driving scenarios.

2. The AI ​​vision algorithm processing method based on an in-vehicle intelligent terminal according to claim 1, characterized in that, Before performing multi-scale feature enhancement on driving scene image data, the following steps are also included: The sampling frame rate and pixel cropping area of ​​driving scene image data are dynamically adjusted based on the vehicle's dynamic operating parameters to form driving scene image data adapted to the current driving scene.

3. The AI ​​vision algorithm processing method based on an in-vehicle intelligent terminal according to claim 1, characterized in that, Key targets of the driving environment include lane markings, obstacles, and traffic signs.

4. The AI ​​vision algorithm processing method based on an in-vehicle intelligent terminal according to claim 1, characterized in that, Multi-scale feature enhancement is performed on driving scene image data to generate enhanced feature maps with scene priority annotations, including: The driving scene image data is downsampled to obtain multiple image feature layers of different scales, and the resolution of each scale image feature layer decreases by 1 / 2 gradient. Calculate the edge gradient values ​​of lane line regions in the image feature layers at each scale, and assign lane line feature weights based on the edge gradient values. Regions with edge gradient values ​​higher than a preset threshold are assigned a first weight, and regions with edge gradient values ​​lower than a preset threshold are assigned a second weight, with the first weight > the second weight. Predict the motion trend vector of traffic participants in a driving scenario, wherein the motion trend vector includes the direction of motion and the speed of motion; The lane line feature weights of image feature layers at each scale are weighted and fused with the motion trend vectors of traffic participants in the corresponding area; among them, traffic participants whose motion trend vectors point to the vehicle's driving path are assigned a third weight, and traffic participants whose motion trend vectors deviate from the vehicle's driving path are assigned a fourth weight, with the third weight being greater than the fourth weight. Based on the weighted fusion results, feature fusion and pixel-level enhancement are performed on image feature layers at various scales to generate enhanced feature maps with scene priority annotations.

5. The AI ​​vision algorithm processing method based on an in-vehicle intelligent terminal according to claim 1, characterized in that, The enhanced feature map is input into the lightweight Transformer inference model. The lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs the recognition results of key targets in the driving environment, including: Real-time collection of hardware computing power status parameters of in-vehicle intelligent terminals; Multiple sets of computing power threshold ranges are preset, and the corresponding number of attention heads is dynamically matched according to the computing power threshold range to which the current hardware computing power status parameters belong. Adjusting the feature channel compression ratio based on scene priority annotation of enhanced feature maps; The lightweight Transformer inference model uses a hierarchical attention mechanism to perform self-attention calculations on the adjusted feature channels, strengthens the feature association weights of high-priority regions, and outputs the category labels, location coordinates, and confidence scores of key targets in the driving environment to form a complete recognition result.

6. The AI ​​vision algorithm processing method based on an in-vehicle intelligent terminal according to claim 5, characterized in that, The hardware computing power status parameters include CPU utilization, GPU memory usage, and NPU computing throughput.

7. The AI ​​vision algorithm processing method based on an in-vehicle intelligent terminal according to claim 1, characterized in that, Cross-validation is performed based on the recognition results and heterogeneous data from vehicle sensors. Real-time closed-loop correction is applied to the feature extraction weights and inference thresholds of the AI ​​vision algorithm to achieve dynamic visual perception in driving scenarios, including: Based on the recognition accuracy and scene complexity of each target category in historical driving data, a real-time verification pass rate threshold is dynamically generated. Calculate the matching pass rate between the visual recognition results of each type of target and the heterogeneous data of the vehicle sensor. If the matching pass rate is higher than the real-time verification pass rate threshold, the recognition result is deemed valid and the current algorithm parameters are maintained. If the matching pass rate is lower than the real-time verification pass rate threshold, an intelligent closed-loop correction mechanism is activated: an attention-oriented weight allocation model is introduced, and the effective target features are verified based on heterogeneous data. The weights of the effective feature channels are dynamically allocated, and the weight coefficients of the effective feature channels are adaptively increased according to the proportion of the matching pass rate gap. Among them, the proportion of the matching pass rate gap = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold; the weight coefficients of the invalid feature channels are sparsified and suppressed through regularization constraints. The real-time verification pass rate threshold is dynamically adjusted using the ratio of the difference between the matching pass rate and the real-time verification pass rate threshold as the core factor and the ambient light intensity as the auxiliary factor. When the matching pass rate of a certain type of target is lower than the proportion threshold for multiple consecutive frames, the system automatically calls the preset scene-specific feature template to replace the current feature extraction logic, and feeds the corrected parameters back to the AI ​​vision algorithm in real time to achieve adaptive upgrade of dynamic visual perception in driving scenarios.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images