AI vision algorithm processing method based on vehicle-mounted intelligent terminal and electronic device

Abstract

The application provides an AI vision algorithm processing method based on a vehicle-mounted intelligent terminal and electronic equipment, and comprises the following steps: collecting real-time driving scene image data based on the vehicle-mounted intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, dynamically adjusting the number of attention heads and the feature channel compression ratio, and outputting a recognition result including a driving environment key target; cross-verification is performed based on the recognition result and heterogeneous data of the vehicle-mounted sensor, real-time closed-loop correction is performed on the feature extraction weight and inference threshold of the AI vision algorithm, and dynamic visual perception of the driving scene is completed. In the application, the defects that the current driving scene cannot be dynamically adapted, the recognition accuracy cannot be balanced, and the visual algorithm cannot be closed-loop optimized are overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an AI vision algorithm processing method based on a vehicle-mounted intelligent terminal and an electronic device. BACKGROUND

[0002] With the rapid development of intelligent networked vehicles and autonomous driving technology, the vehicle-mounted AI vision system as the core module of environmental perception directly affects driving safety in terms of recognition accuracy, real-time performance and robustness. The vehicle-mounted intelligent terminal collects driving scene image data through the camera and realizes the recognition of key targets such as lane lines, obstacles and traffic signs by means of AI vision algorithms, providing data support for autonomous driving decision and control, which has become the core research direction of current vehicle-mounted perception technology.

[0003] However, the existing vehicle-mounted AI vision algorithm processing method still has many technical bottlenecks in actual application, which is difficult to meet the high requirements of complex driving scenes:

[0004] Insufficient scene adaptability and weak feature enhancement pertinence: driving scenes have dynamic variability, such as high-speed driving and urban congestion, strong light in sunny days and weak light at night, dry road surface and rainy and foggy weather, etc. The feature expression of key targets in different scenes is completely different. The existing technology mostly adopts a fixed parameter feature enhancement strategy, without differentiating and strengthening the features of key targets (such as lane lines and close-range obstacles) according to scene priorities, which leads to the problems of insufficient feature extraction and reduced target recognition in complex scenes, thereby affecting the subsequent recognition accuracy.

[0005] It is difficult to balance the computing power and recognition accuracy: the hardware computing power of the vehicle-mounted intelligent terminal is limited by installation space, power consumption budget and other constraints, and cannot be equivalent to the cloud computing power platform. The existing AI vision algorithm mostly adopts standard Transformer or deep learning model, which can ensure recognition accuracy, but has large model parameter quantity and high computational complexity, making it difficult to realize real-time inference on the vehicle-mounted terminal; some lightweight improvement schemes simplify the model by fixing the compression ratio, without dynamically adjusting the model structure (such as the number of attention heads and the number of feature channels) according to the real-time computing power state of the vehicle-mounted hardware, which leads to increased inference delay under high hardware load, or waste of accuracy under low load without fully utilizing the computing power, thus failing to achieve dynamic optimal matching of computing power and accuracy.

[0006] Robustness is insufficient, and there is no closed-loop optimization mechanism: vehicle-mounted visual recognition is susceptible to environmental interference (such as changes in light, occlusion, and road bumps), and the recognition result of single visual data has the risk of misjudgment. The existing technology does not dynamically adjust the verification standard in combination with the scene complexity and environmental factors; and most solutions only stop at the verification level of the recognition result, and do not establish a real-time closed-loop correction mechanism for the core parameters (feature extraction weight, inference threshold) of the AI visual algorithm, which leads to the inability of the algorithm performance to adaptively optimize according to the changes of the actual driving scene, and the recognition failure easily occurs in extreme scenes, affecting the driving safety. At the same time, the driving image data in some specific scenes cannot be saved in time and is easily damaged or has the risk of data leakage, which is not convenient for subsequent inspection.

[0007] Therefore, how to design an AI visual algorithm processing method that can adapt to dynamic driving scenes, balance the recognition accuracy of vehicle-mounted hardware computing power, and realize algorithm closed-loop optimization through multi-source data fusion has become the key to solving the pain points of the existing technology and improving the robustness and real-time performance of the vehicle-mounted visual perception system, and has important significance for promoting the industrial application of automatic driving technology. SUMMARY

[0008] The main purpose of the present application is to provide an AI visual algorithm processing method based on a vehicle-mounted intelligent terminal and an electronic device, which aims to overcome the defects that current cannot dynamically adapt to driving scenes, cannot balance recognition accuracy, and cannot realize visual algorithm closed-loop optimization.

[0009] To achieve the above-mentioned purpose, the present application provides an AI visual algorithm processing method based on a vehicle-mounted intelligent terminal, comprising the following steps:

[0010] Real-time acquisition of driving scene image data based on a vehicle-mounted intelligent terminal;

[0011] Performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation;

[0012] Inputting the enhanced feature map into an AI visual algorithm based on a lightweight Transformer inference model, dynamically adjusting the number of attention heads and the feature channel compression ratio of the lightweight Transformer inference model, and outputting the recognition result including the key target of the driving environment;

[0013] Cross-validation based on the recognition result and the heterogeneous data of the vehicle-mounted sensor, real-time closed-loop correction of the feature extraction weight and the inference threshold of the AI visual algorithm, and completion of the dynamic visual perception of the driving scene.

[0014] Further, before performing multi-scale feature enhancement on the driving scene image data, it further comprises:

[0015] The sampling frame rate and pixel clipping region of the driving scene image data are dynamically adjusted based on vehicle dynamic working condition parameters to form driving scene image data that adapts to the current driving scene.

[0016] Further, the driving environment key target includes a lane line, an obstacle, and a traffic sign.

[0017] Further, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority annotation, including:

[0018] The driving scene image data is down-sampled to obtain multiple image feature layers of different scales, and the resolution of each scale image feature layer decreases by a gradient of 1 / 2;

[0019] The edge gradient value of the lane line region in each scale image feature layer is calculated, and a lane line feature weight is assigned based on the edge gradient value. The region with an edge gradient value higher than a preset threshold is assigned a first weight, and the region with an edge gradient value lower than a preset threshold is assigned a second weight, and the first weight > the second weight;

[0020] The motion trend vector of the traffic participant in the driving scene is predicted, and the motion trend vector includes a motion direction and a motion speed;

[0021] The lane line feature weight of each scale image feature layer and the motion trend vector of the corresponding region traffic participant are weighted and fused; wherein the traffic participant pointing to the vehicle driving path is assigned a third weight, and the traffic participant deviating from the vehicle driving path is assigned a fourth weight, and the third weight > the fourth weight;

[0022] Based on the weighted fusion result, the image feature layers of each scale are fused and pixel-level enhanced to generate an enhanced feature map with scene priority annotation.

[0023] Further, the enhanced feature map is input into a lightweight Transformer inference model, the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs an identification result including the driving environment key target, including:

[0024] The hardware computing power state parameters of the vehicle-mounted intelligent terminal are collected in real time;

[0025] A plurality of groups of computing power threshold intervals are preset, and the corresponding number of attention heads is dynamically matched according to the computing power threshold interval to which the current hardware computing power state parameter belongs;

[0026] The feature channel compression ratio is adjusted based on the scene priority annotation of the enhanced feature map;

[0027] The lightweight Transformer inference model performs self-attention calculation on the adjusted feature channel through a hierarchical attention mechanism, strengthens the feature correlation weight of the high-priority area, and respectively outputs the category label, position coordinates and confidence of the key target in the driving environment, forming a complete recognition result.

[0028] Further, the hardware computing power state parameter includes CPU occupation rate, GPU video memory usage rate and NPU operation throughput.

[0029] Further, cross-validation is performed on the heterogeneous data of the recognition result and the vehicle-mounted sensor, the feature extraction weight and inference threshold of the AI vision algorithm are corrected in real time, and dynamic visual perception of the driving scene is completed, including:

[0030] Based on the recognition accuracy and scene complexity of each target category in the historical driving data, a real-time verification pass rate threshold is dynamically generated;

[0031] The matching pass rate of the visual recognition result of each type of target and the heterogeneous data of the vehicle-mounted sensor is calculated, if the matching pass rate is higher than the real-time verification pass rate threshold, it is determined that the recognition result is effective, and the current algorithm parameter is maintained;

[0032] If the matching pass rate is lower than the real-time verification pass rate threshold, an intelligent closed-loop correction mechanism is started: a weight distribution model guided by an attention mechanism is introduced, the effective target features are verified based on the heterogeneous data, the feature channel weight is dynamically allocated, and the weight coefficient of the effective feature channel is adaptively improved according to the matching pass rate gap ratio; wherein the matching pass rate gap ratio=(real-time verification pass rate threshold-matching pass rate) / real-time verification pass rate threshold; the weight coefficient of the ineffective feature channel is sparsified and inhibited by regularization constraint;

[0033] The difference ratio of the matching pass rate and the real-time verification pass rate threshold is taken as the core factor, and the ambient light intensity is taken as the auxiliary factor, and the real-time verification pass rate threshold is dynamically adjusted;

[0034] When the matching pass rate of a certain type of target is continuously lower than the proportion threshold for multiple frames, a preset scene-specific feature template is automatically called to replace the current feature extraction logic, and the modified parameters are fed back to the AI vision algorithm in real time, realizing adaptive upgrade of dynamic visual perception of the driving scene.

[0035] The application also provides an AI vision algorithm processing device based on a vehicle-mounted intelligent terminal, comprising:

[0036] The acquisition unit is configured to acquire driving scene image data in real time based on the vehicle-mounted intelligent terminal;

[0037] The generation unit is configured to perform multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation.

[0038] an output unit configured to input the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, the lightweight Transformer inference model dynamically adjusting a number of attention heads and a feature channel compression ratio, and output a recognition result including a key target in a driving environment;

[0039] a correction unit configured to cross-verify heterogeneous data of the recognition result and a vehicle-mounted sensor, and perform real-time closed-loop correction on feature extraction weights and inference thresholds of the AI vision algorithm, to complete dynamic visual perception of the driving scene.

[0040] The application further provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method according to any one of the preceding embodiments when executing the computer program.

[0041] The application further provides a computer-readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the method according to any one of the preceding embodiments.

[0042] The AI vision algorithm processing method based on a vehicle-mounted intelligent terminal and the electronic device provided by the application include: collecting driving scene image data in real time based on a vehicle-mounted intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, the lightweight Transformer inference model dynamically adjusting a number of attention heads and a feature channel compression ratio, and outputting a recognition result including a key target in a driving environment; cross-verifying heterogeneous data of the recognition result and a vehicle-mounted sensor, and performing real-time closed-loop correction on feature extraction weights and inference thresholds of the AI vision algorithm, to complete dynamic visual perception of the driving scene. In the application, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority annotation, and the driving scene is dynamically adapted; the lightweight Transformer inference model dynamically adjusts a number of attention heads and a feature channel compression ratio, and balances vehicle-mounted computing power and recognition accuracy; the recognition result and the heterogeneous data of the vehicle-mounted sensor are cross-verified, and the AI vision algorithm is corrected in real time to realize closed-loop optimization of the AI vision algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a step schematic diagram of the AI vision algorithm processing method based on a vehicle-mounted intelligent terminal in an embodiment of the application;

[0044] Figure 2 is a structure block diagram of an AI vision algorithm processing device based on a vehicle-mounted intelligent terminal in an embodiment of the application;

[0045] Figure 3 Fig. 1 is a schematic block diagram of an electronic device according to an embodiment of the present application.

[0046] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] It should be particularly noted that all technical steps, algorithm applications and parameter settings of the technical solutions in the present application have clear technical objectives and application values, and are not used to realize simple functions by means of complicated steps and algorithm formulas. In order to describe each step in detail and avoid unclear problems, some conventional algorithms will be explained and described, but this does not mean that the algorithms and technical features listed herein are the only way to realize the technical solutions of the present application, nor are they used to limit the protection scope of the present application. The present application is not a combination of the listed algorithms and technical features, and the essence is to exemplify the implementation of the present application in order to fully explain and describe the present application. It is not to pursue the complexity of the form and increase the meaningless technical links, nor to deviate from the actual demand of the technical stacking, which conforms to the conventional logic of technical improvement and design.

[0049] With reference to Figure 1 , an AI vision algorithm processing method based on a vehicle-mounted intelligent terminal is provided in an embodiment of the present application, which includes the following steps:

[0050] Step S1, real-time collection of driving scene image data based on the vehicle-mounted intelligent terminal;

[0051] Step S2, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority annotation;

[0052] Step S3, the enhanced feature map is input into an AI vision algorithm based on a lightweight Transformer inference model, the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs the recognition result of the key target of the driving environment;

[0053] Step S4, cross-validation is performed based on the recognition result and the heterogeneous data of the vehicle-mounted sensor, the feature extraction weight and the inference threshold of the AI vision algorithm are real-time closed-loop corrected, and the dynamic visual perception of the driving scene is completed.

[0054] In the embodiment, as described in step S1 above, the vehicle-mounted intelligent terminal is equipped with a high-definition vision acquisition module (such as a CMOS camera), which performs all-around and uninterrupted image data acquisition of the front, side and rear driving scenes of the vehicle in the driving process according to a preset initial sampling standard (such as a default frame rate of 30 fps and a pixel resolution of 1920x1080). During the acquisition process, the camera maintains a continuous working state based on the stable power supply of the vehicle-mounted power supply, and the image data is stored in the local cache unit of the vehicle-mounted terminal in real time, while the acquisition state information (such as whether the acquisition is normal and the data transmission rate) is synchronously fed back through the vehicle-mounted bus (such as the CAN bus), to ensure that the subsequent algorithm processing link can obtain continuous and complete original image data support.

[0055] As described in step S2 above, for the acquired original driving scene image data, 3-5 image feature layers of different scales can be first constructed by Gaussian pyramid downsampling technology, the resolution of each scale feature layer decreases by 1 / 2 gradient, to realize feature coverage of different size targets (such as small traffic signs at a long distance and large obstacles at a short distance); then, based on the edge gradient value distribution of the lane line region, higher weights are given to the key lane line regions with edge gradient values higher than a preset threshold, to strengthen the recognition of lane line features, while the motion trend vector (including motion direction and speed) of the traffic participants such as pedestrians, non-motor vehicles and other vehicles is predicted by Kalman filtering algorithm, to improve the feature attention of the high-risk traffic participant region pointing to the driving path of the vehicle; finally, the multi-scale feature layers are fused, and the lane line weight distribution result and the traffic participant motion trend analysis are combined, to mark the scene priority of different regions in the image (such as the key region of the lane line as the first priority, the high-risk traffic participant region as the second priority, and the ordinary scene region as the third priority), to finally generate an enhanced feature map considering the integrity of multi-scale features and the scene priority orientation, to solve the problem of insufficient targeting of traditional feature enhancement.

[0056] As described in step S3 above, the generated enhanced feature map is input in real time to a lightweight Transformer inference model pre-deployed on the vehicle-mounted intelligent terminal. The model is structurally optimized to adapt to the hardware computing power constraints of the vehicle-mounted terminal. The core lies in its dynamic adaptation capability. On the one hand, the model collects hardware computing power state parameters (including CPU occupancy, GPU memory usage, and NPU operation throughput) of the vehicle-mounted terminal in real time, and dynamically adjusts the number of attention heads according to the computing power load. When the load is low, 8-12 attention heads are configured to ensure recognition accuracy; when the load is medium, 4-6 attention heads are configured to balance accuracy and speed; and when the load is high, 2-3 attention heads are configured to prioritize real-time performance. On the other hand, combined with the scene priority labeling results of the enhanced feature map, different compression ratios are used for feature channels in different priority areas. The feature channels in the first priority area are not compressed, the feature channels in the second priority area are compressed by a ratio of 1:2, and the feature channels in the third priority area are compressed by a ratio of 1:4, thereby reducing the amount of calculation while retaining key features. The model performs self-attention calculation on the adjusted feature channels through a hierarchical attention mechanism, strengthens the feature correlation in high-priority areas, and then processes the results through the classifier and regressor of the output layer. Finally, the class labels, position coordinates, and confidence of the key targets in the driving environment (including lane lines, obstacles, traffic signs, pedestrians, etc.) are output, forming a complete recognition result.

[0057] As described in step S4 above, real-time data of vehicle-mounted heterogeneous sensors is collected, the heterogeneous sensors including millimeter wave radar (providing target distance and speed data), ultrasonic sensor (providing near-distance obstacle detection data), inertial measurement unit (IMU, providing vehicle attitude angle and acceleration data), and ambient light sensor (providing illumination intensity data), a multi-source data support system is constructed; then a multi-source data correlation mapping rule is established, the target position coordinates in the AI visual recognition result and the radar distance data are spatially registered through a coordinate conversion matrix, the target motion state and the radar speed data are time-aligned through time stamp synchronization, the near-distance obstacle recognition result and the ultrasonic data are covered and verified through the overlap area ratio, and the illumination intensity data is introduced as a visual recognition credibility correction coefficient; then a dynamic verification threshold matrix based on scene complexity is constructed, the verification pass rate threshold of each target type is generated in real time through a gradient boosting tree model combined with historical driving data, and the matching pass rate of the visual recognition result and the heterogeneous sensor data is calculated; if the matching pass rate is higher than the corresponding threshold, the current algorithm parameters are maintained; if the matching pass rate is lower than the threshold, a closed-loop correction is started: through a weight distribution model guided by an attention mechanism, the effective feature channel weight is adaptively improved according to the matching pass rate gap ratio, the invalid feature channel is inhibited through L1 regularization, and the inference threshold is adjusted by using the difference ratio and the illumination intensity double-factor linkage; finally, the corrected feature extraction weight and the inference threshold are fed back to the feature enhancement module of step S2 and the lightweight Transformer inference model of step S3 in real time, the algorithm running parameters are updated, the full-closed-loop optimization of collection-enhancement-inference-verification-correction is realized, and it is ensured that the driving scene visual perception can dynamically adapt to the complex and changeable driving environment, and the perception accuracy and robustness are improved.

[0058] In an embodiment, before performing multi-scale feature enhancement on the driving scene image data, the method further comprises:

[0059] Based on the vehicle dynamic working condition parameters, the sampling frame rate and the pixel clipping area of the driving scene image data are dynamically adjusted to form driving scene image data adapted to the current driving scene.

[0060] In the embodiment, the vehicle-mounted intelligent terminal obtains vehicle dynamic working condition parameters in real time through a vehicle-mounted bus (such as a CAN bus or a LIN bus), and the dynamic working condition parameters include a vehicle driving speed, a steering angle, a road surface bumping level (converted from acceleration data collected by an inertial measurement unit (IMU)), and a driving scene type (determined by a preset scene recognition rule, such as a highway, an urban road, or a rural road). Based on a preset working condition-parameter mapping rule, dynamic adaptive adjustment is performed on collected original image data: in terms of sampling frame rate adjustment, when the vehicle driving speed is greater than or equal to 80 km / h (high-speed working condition), the sampling frame rate is increased to 35-40 fps to capture fast-moving targets; when the vehicle driving speed is less than or equal to 30 km / h (urban congestion or low-speed working condition), the sampling frame rate is reduced to 20-25 fps to reduce invalid data collection and calculation load; when the road surface bumping level is higher than a preset threshold (such as a severe bumping scene), the sampling frame rate is maintained at a 30 fps reference value to ensure the stability of image data. In terms of pixel clipping region adjustment, the vehicle driving trajectory is determined according to the steering angle. When driving in a straight line, the clipping region focuses on the front main field of view (the horizontal clipping range is 20%-80% of the image width, and the vertical clipping range is 30%-90% of the image height), and lane lines and long-distance target regions are retained; when turning left, the left field of view clipping proportion is increased (the horizontal clipping range is adjusted to 0%-70% of the image width), and image information of the left lane and obstacles is strengthened; when turning right, the right field of view clipping proportion is increased (the horizontal clipping range is adjusted to 30%-100% of the image width), and right-side driving environment details are highlighted; meanwhile, the clipping strategy is optimized in combination with the driving scene type, more image regions of close-range pedestrians and non-motor vehicles are retained in an urban road scene, and the image coverage range of long-distance traffic signs and preceding vehicles is extended in a highway scene. Through the above dynamic adjustment of the frame rate and the clipping region, redundant image information is filtered, and simplified and efficient image data adapted to the current driving working condition are formed, thereby reducing the calculation complexity for subsequent feature enhancement and inference processing.

[0061] In an embodiment, the driving environment key targets include lane lines, obstacles, and traffic signs.

[0062] In an embodiment, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority labels, including:

[0063] The driving scene image data is down-sampled to obtain a plurality of image feature layers of different scales, and the resolutions of the image feature layers of different scales decrease by a gradient of 1 / 2;

[0064] An edge gradient value of a lane line region in each scale of image feature layer is calculated, and a lane line feature weight is assigned based on the edge gradient value, a first weight is assigned to a region with an edge gradient value higher than a preset threshold, and a second weight is assigned to a region with an edge gradient value lower than the preset threshold, and the first weight is greater than the second weight;

[0065] A motion trend vector of a traffic participant in the driving scene is predicted, and the motion trend vector includes a motion direction and a motion speed;

[0066] The lane line feature weight of each scale of image feature layer is weighted and fused with the motion trend vector of the corresponding region traffic participant; wherein a third weight is assigned to a traffic participant pointing to the driving path of the vehicle, a fourth weight is assigned to a traffic participant deviating from the driving path of the vehicle, and the third weight is greater than the fourth weight;

[0067] Based on the weighted fusion result, feature fusion and pixel-level enhancement are performed on each scale of image feature layer to generate an enhanced feature map with scene priority annotation.

[0068] In this embodiment, for the driving scene image data that has completed the working condition adaptation adjustment, a multi-scale feature extraction operation is performed using a Gaussian pyramid downsampling technology. First, the adjusted image is taken as a reference scale feature layer (layer 1), the reference layer image is smoothed by Gaussian filtering to eliminate high-frequency noise interference, and then the smoothed image is sampled by rows and columns to obtain a second layer image feature layer (such as 960x540) with a resolution of 1 / 2 of the reference layer; the above-mentioned "Gaussian smoothing + row and column sampling" process is repeated to generate a third layer (480x270), a fourth layer (240x135), and a fifth layer (120x67) image feature layer in sequence, and finally form a feature layer set of 3-5 different scales (the specific number of layers can be dynamically configured according to the computing power of the vehicle terminal and the recognition accuracy requirement). The resolution of each scale of feature layer strictly decreases by 1 / 2 gradient, ensuring that the low-level feature layer (high resolution) retains the detailed features of close-range targets, and the high-level feature layer (low resolution) covers the global features of long-distance targets, achieving comprehensive feature coverage of different sizes and distances of key targets (such as close-range large obstacles, medium-distance front vehicles, and long-distance small traffic signs) in the driving scene, and laying a multi-scale feature foundation for subsequent accurate recognition.

[0069] For each generated scale image feature layer, the edge information of the lane line region can be extracted respectively: first, the gray scale of each scale feature layer image is processed to convert it into a single channel gray scale image to simplify the calculation; then, Gaussian filtering is used to further suppress noise and avoid interference from non-lane line edges (such as road cracks and shadows); then, a double threshold method (high threshold and low threshold) is used to screen edge pixels to retain the continuous edge profile of the lane line, and finally the edge gradient value of each pixel point is calculated (the larger the gradient value, the clearer the edge and the more prominent the feature). A preset edge gradient threshold (such as 0.6, which can be dynamically calibrated based on historical driving data and lane line recognition accuracy requirements) is used to assign weights to the pixel points in the lane line region of each scale feature layer: the region with an edge gradient value higher than the preset threshold (such as a clear white solid lane line, a yellow dashed lane line, a lane line inflection point, and an intersection region) is determined as a key feature region of the lane line, and is assigned a first weight (such as 0.8-1.0); the region with an edge gradient value lower than the preset threshold (such as a lane line obscured by rain, a lane line segment obscured by a preceding vehicle, and a lane line in a dimly lit environment) is determined as a weak feature region of the lane line, and is assigned a second weight (such as 0.3-0.5), and it is strictly ensured that the first weight > the second weight. Through this differential weight assignment, the recognition of clear lane line features is strengthened, while the effective information of weak feature lane lines is not missed, solving the problem of key regions not being prominent caused by equal feature weight in traditional feature processing.

[0070] Next, based on the acquired vehicle dynamic working condition parameters (such as vehicle speed and steering angle) and the generated multi-scale image feature layer data, a Kalman filter algorithm is used to construct a motion state prediction model of traffic participants (including pedestrians, non-motor vehicles, and other moving targets). First, the target detection and tracking of traffic participants in each scale feature layer is performed, the target position is marked by a bounding box, and the appearance features (such as size, contour, and color) and position features (such as pixel coordinates) of the target are extracted; then, based on the position changes of the target in consecutive multiple frames of images (such as the previous 5-10 frames), the instantaneous motion speed (such as pixels / frame, which can be converted into actual speed through coordinate conversion) and motion direction (such as driving along the lane, crossing the road, or parking on the side, which is quantified by a direction vector) of the traffic participant are calculated; finally, through the prediction equation and update equation of the Kalman filter algorithm, the influence of image noise and detection errors on the motion state judgment is eliminated, and the motion trend vector of each traffic participant is accurately output, which includes the quantized motion direction (such as 0° with respect to the vehicle driving direction indicating the same direction, and 90° indicating transverse crossing) and motion speed, providing dynamic basis for subsequent risk level judgment and weight assignment.

[0071] For each generated scale image feature layer, an association mapping between lane line feature weight and traffic participant motion trend vector is established by pixel area: first, determine the pixel coverage area of each traffic participant in the feature layer, and take the corresponding lane line feature weight (i.e. the first weight or the second weight assigned) of the area as the basic weight; then, according to the predicted traffic participant motion trend vector, determine the risk level of the vehicle driving: if the motion trend vector points to the vehicle driving path (such as pedestrians crossing the road, vehicles from the rear side illegally changing lanes to the vehicle lane, non-motor vehicles driving in the same direction but with a speed significantly lower than the vehicle), it is determined as a high-risk traffic participant, and a third weight (such as 0.7-0.9) is assigned; if the motion trend vector deviates from the vehicle driving path (such as pedestrians walking to the roadside, vehicles driving to the opposite lane, and vehicles driving in the same direction with a speed close to the vehicle), it is determined as a low-risk traffic participant, and a fourth weight (such as 0.2-0.4) is assigned, and strictly ensure that the third weight> the fourth weight. Using a weighted summation formula (such as fusion weight = lane line basic weight x 0.6 + traffic participant risk weight x 0.4, the weight ratio can be optimized according to the actual scene), the lane line feature weight of each pixel area and the risk weight of the corresponding area traffic participant are calculated to obtain the comprehensive feature weight of each pixel area. This fusion process not only retains the importance of the core static feature of the lane line, but also fully considers the risk difference of the dynamic target of the traffic participant, making the feature weight distribution more suitable for driving safety needs.

[0072] Finally, the attention fusion algorithm is used to fuse the feature layers of each scale image: based on the comprehensive feature weight of each pixel area, the feature information of the low-layer high-resolution feature layer (highlighting details) and the high-layer low-resolution feature layer (highlighting the global) is weighted and superimposed. The more comprehensive feature weight of an area, the more detailed information it retains during the fusion process, ensuring that the features of the key lane line area and the high-risk traffic participant area are not lost. Subsequently, pixel-level enhancement processing is performed on the fused feature map: for areas with a comprehensive feature weight higher than the first fusion threshold (i.e., key lane line areas + high-risk traffic participant overlapping areas), the pixel contrast is improved through histogram equalization to enhance feature recognition; for areas with a comprehensive feature weight between the second fusion threshold and the first fusion threshold (i.e., key lane line areas or high-risk traffic participant single areas), the brightness and contrast are optimized; for areas with a comprehensive feature weight lower than the second fusion threshold (i.e., ordinary scene areas), only basic noise reduction processing is performed to avoid excessive enhancement of redundant features. Finally, based on the comprehensive feature weight, the enhanced feature map is labeled with scene priority: for example, areas with a comprehensive feature weight ≥ 0.8 are labeled as first priority (key lane line + high-risk traffic participant areas), areas with a comprehensive feature weight of 0.5 ≤ comprehensive feature weight < 0.8 are labeled as second priority (single key feature area), and areas with a comprehensive feature weight < 0.5 are labeled as third priority (ordinary scene area), generating an enhanced feature map that takes into account the integrity of multi-scale features, the significance of key features, and the differentiation of scene risks, providing high-quality input for the efficient and accurate inference of the subsequent lightweight Transformer inference model.

[0073] In an embodiment, the enhanced feature map is input into a lightweight Transformer inference model, which dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs the recognition results of key targets in the driving environment, including:

[0074] Real-time acquisition of hardware computing power state parameters of the vehicle intelligent terminal;

[0075] Pre-set multiple groups of computing power threshold intervals, dynamically match the corresponding number of attention heads according to the computing power threshold interval to which the current hardware computing power state parameter belongs;

[0076] Adjust the feature channel compression ratio based on the scene priority labeling of the enhanced feature map;

[0077] The lightweight Transformer inference model performs self-attention calculation on the adjusted feature channel through a hierarchical attention mechanism, strengthens the feature correlation weight of high-priority areas, and outputs the class label, position coordinates, and confidence of the key targets in the driving environment, forming a complete recognition result.

[0078] In this embodiment, the vehicle-mounted intelligent terminal is equipped with a hardware state monitoring module. The module establishes a real-time communication link with the CPU, GPU, NPU, and other core operation units of the terminal. Through the underlying interface of the vehicle-mounted system (such as the proc file system of the Linux kernel and the computing power monitoring API provided by the GPU manufacturer), the hardware computing power state parameters are continuously collected. The computing power state parameters include three major dimensions: CPU occupancy rate (the load proportion of the current operation core, quantization range 0%-100%), GPU memory usage rate (the proportion of the visible memory occupied for storing model parameters and intermediate feature data, quantization range 0%-100%), and NPU operation throughput (the number of tensor operations completed by NPU per unit time, unit: TOPS). The collection frequency is synchronized with the image data processing frame rate (such as 20-40 fps), ensuring that the computing power state parameters can reflect the current hardware running load in real time, providing accurate computing power data support for subsequent model structure dynamic adjustment, and avoiding model adaptation errors caused by lagging computing power collection.

[0079] Next, based on the hardware performance upper limit of the vehicle-mounted intelligent terminal (such as the maximum number of CPU cores, GPU memory capacity, and NPU peak computing power) and the real-time requirements of AI vision algorithms, three groups of computing power threshold intervals are preset, forming a three-level load division system of low load, medium load, and high load. For each group of computing power threshold interval, the corresponding number of attention heads is preset: 8-12 attention heads are configured in the low load interval to strengthen feature association calculation through more attention branches and improve recognition accuracy in complex scenarios; 4-6 attention heads are configured in the medium load interval to simplify some non-critical attention branches and reduce computational complexity under the premise of controllable accuracy loss; 2-3 attention heads are configured in the high load interval to retain only the core attention branches to ensure uninterrupted inference process with minimum computational complexity. When the hardware computing power state parameters cross different threshold intervals, the lightweight Transformer inference model switches the number of attention heads in real time through a dynamic configuration interface. During the switching process, a parameter smoothing transition strategy is adopted to avoid sudden changes in model output results and ensure recognition stability.

[0080] The lightweight Transformer inference model reads the generated enhanced feature map, parses the scene priority annotation information (first priority: lane line key + high-risk traffic participant area; second priority: single key feature area; third priority: ordinary scene area) therein, and performs differential compression processing on the feature channels corresponding to different priority areas based on a preset priority-compression ratio mapping rule. The feature channel compression adopts a technical solution combining deep separable convolution and channel pruning: for the first priority area, the feature channels thereof adopt a 1:1 non-compression ratio, complete details of all feature channels (such as edge features of lane lines and contour features of high-risk pedestrians) are retained, and the recognition accuracy of the core target is ensured not to be lost; for the second priority area, the feature channels thereof adopt a 1:2 compression ratio, redundant feature channels (such as non-key texture feature channels) are removed through channel pruning, and only core feature channels are retained, so as to reduce the amount of calculation while maintaining the target recognition degree; for the third priority area, the feature channels thereof adopt a 1:4 compression ratio, and a plurality of channels of features are fused into a small number of key channels through deep separable convolution, so as to maximize the reduction of redundant calculation, and the feature information loss is ensured to be controlled within 10% through regularization constraint in the compression process. The feature channel compression of different priority areas is independently executed, the boundaries of the areas are divided through a space mask technology, so as to avoid the interference of features of different priority areas with each other in the compression process, and the differential optimization goal of preserving the accuracy of the core area and reducing the load of the ordinary area is achieved.

[0081] The encoder module of the lightweight Transformer inference model adopts a hierarchical attention mechanism to cooperatively calculate the adjusted attention heads and the compressed feature channels: the bottom attention layer focuses on local feature association (such as the edges and texture features of the target), the middle attention layer strengthens the target-level feature association (such as the relative position relationship between the lane line and the obstacle), and the top attention layer constructs the global scene association (such as the motion coordination relationship between multiple targets). During the self-attention calculation process, the model dynamically enhances the feature association weight of the high-priority area based on the scene priority annotation of the enhanced feature map. The feature association weight of the first-priority area is enhanced by 20-30%, the feature association weight of the second-priority area is enhanced by 10-15%, and the feature association weight of the third-priority area is maintained at the basic level to ensure that the features of the core target are highlighted. Subsequently, the encoded feature vector is input into the decoder module of the model, the decoder interacts with the output features of the encoder through the multi-head attention mechanism, and accurately locates the feature representation of the key target in the driving environment; then the classifier and the regressor of the output layer cooperatively process: the classifier uses the Softmax activation function to map the feature vector to the target category probability distribution, and outputs the category label (such as lane line, pedestrian, small car, speed limit 60km / h sign, etc.); the regressor uses the bounding box regression algorithm to output the position coordinates of the target in the image (represented by the left upper corner x / y coordinates and the right lower corner x / y coordinates of the rectangular bounding box); at the same time, through the confidence calculation module, based on the feature matching degree and the historical recognition accuracy, the confidence of each recognition result is output (quantization range 0-1, the closer to 1 indicates that the recognition is more reliable). Finally, the model structurally encapsulates the category label, position coordinates and confidence to form a complete recognition result, which is transmitted in real time to the subsequent cross-validation module through the vehicle bus to provide accurate visual recognition data support for closed-loop correction.

[0082] In an embodiment, the hardware computing power state parameter includes CPU occupancy rate, GPU memory usage rate, and NPU operation throughput.

[0083] In an embodiment, based on the cross-validation of the recognition result and the heterogeneous data of the vehicle-mounted sensor, the feature extraction weight and the inference threshold of the AI visual algorithm are corrected in real time, and the dynamic visual perception of the driving scene is completed, including:

[0084] Based on the recognition accuracy and scene complexity of each target category in the historical driving data, a real-time verification pass rate threshold is dynamically generated;

[0085] The matching pass rate of the visual recognition result of each type of target and the heterogeneous data of the vehicle-mounted sensor is calculated, and if the matching pass rate is higher than the real-time verification pass rate threshold, the recognition result is determined to be valid, and the current algorithm parameter is maintained;

[0086] If the matching pass rate is lower than the real-time verification pass rate threshold, a smart closed-loop correction mechanism is started: a weight distribution model guided by an attention mechanism is introduced, effective target features are verified based on heterogeneous data, feature channel weights are dynamically allocated, and the weight coefficients of effective feature channels are adaptively increased according to the matching pass rate gap ratio; wherein the matching pass rate gap ratio = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold; the weight coefficients of ineffective feature channels are sparsified and inhibited by regularization constraint;

[0087] The difference ratio between the matching pass rate and the real-time verification pass rate threshold is taken as the core factor, and the ambient light intensity is taken as the auxiliary factor to dynamically adjust the real-time verification pass rate threshold.

[0088] When the matching pass rate of a certain type of target is continuously lower than the proportion threshold for multiple frames, the preset scene-specific feature template is automatically called to replace the current feature extraction logic, and the corrected parameters are fed back to the AI vision algorithm in real time, realizing the adaptive upgrade of dynamic visual perception in driving scenes.

[0089] In this embodiment, a historical driving database is first constructed, which stores a large amount of driving data under different working conditions (such as high-speed, urban, and rural roads), different environments (such as sunny, rainy, and foggy, and night), including visual recognition results of each target category (lane lines, long-distance obstacles, short-distance obstacles, and traffic signs), vehicle-mounted sensor heterogeneous data, actual road condition annotation information, and scene complexity parameters (such as road curvature, congestion level, and light intensity level). Based on the database, a gradient boosting tree model is trained, with the input being the target category + scene complexity parameter and the output being the optimal verification pass rate benchmark value under this combination. During real-time driving, the vehicle-mounted intelligent terminal collects the current scene complexity parameters (road curvature ≥ 0.05 rad / m is determined as high complexity, 0.02-0.05 rad / m is medium complexity, and < 0.02 rad / m is low complexity; congestion level is divided into no congestion, light congestion, medium congestion, and heavy congestion according to vehicle density) through the scene recognition module, inputs the current target category + real-time scene complexity parameter into the trained GBDT model, and dynamically generates the real-time verification pass rate threshold of each target category. Among them, the scene complexity and the verification pass rate threshold are negatively correlated: for every one level increase in scene complexity, the real-time verification pass rate threshold of the target is lowered by 3%-5%, ensuring that the threshold adapts to the recognition difficulty in complex scenes. For example, in a high complexity scene (sharp turn + heavy congestion), the real-time verification pass rate threshold of the lane line is lowered from the benchmark value of 85% to 75%-80%, avoiding the misjudgment of reasonable matching deviation as recognition failure due to scene complexity.

[0090] Then, real-time data of vehicle-mounted heterogeneous sensors is collected, the heterogeneous sensors including millimeter wave radar (providing target distance, radial velocity data), ultrasonic sensor (providing 3-meter near-distance obstacle detection data), inertial measurement unit (IMU, providing vehicle attitude angle, acceleration data) and ambient light sensor (providing illumination intensity data), forming a multi-source data verification system. The association mapping rule between visual recognition results and heterogeneous data is established: through the coordinate conversion matrix (pre-generated based on sensor calibration parameters), the image coordinates of the visual recognition target are converted into the vehicle coordinate system coordinates, and are spatially registered with the target distance data of the millimeter wave radar, and the spatial position deviation ≤0.5 meters is determined as spatial matching; through the time stamp synchronization technology, the motion state (speed and direction calculated based on consecutive image frames) of the visual recognition target is time-sequentially aligned with the radar radial velocity data, and the speed deviation ≤5km / h is determined as the motion state matching; for the near-distance obstacle, the overlap area ratio of the visual recognition result and the ultrasonic detection data ≥60% is determined as the coverage matching; meanwhile, the ambient light intensity data is introduced as a matching confidence correction coefficient. When the illumination intensity <300lux (weak light), the correction coefficient is 0.8-0.9, when 10000lux < illumination intensity <50000lux (strong light), the correction coefficient is 0.9-0.95, and under other illumination conditions, the correction coefficient is 1.0. The calculation method of the matching pass rate is: (the number of targets that are spatially matched and motionally matched / coverage matched ÷ the total number of recognized targets of this category) × confidence correction coefficient × 100%. The calculated matching pass rate is compared with the generated real-time verification pass rate threshold, if the matching pass rate is higher than the threshold, it is determined that the current visual recognition result is reliable, the feature extraction weight and inference threshold of the AI vision algorithm are maintained at the current configuration, and there is no need to adjust, ensuring the consistency of the algorithm parameters in the stable scene.

[0091] When the matching pass rate of a certain type of target is lower than the corresponding real-time verification pass rate threshold, it is determined that the recognition accuracy of this type of target is insufficient, and the intelligent closed-loop correction mechanism is immediately started. First, the effective target features are screened and verified through heterogeneous data: for example, the features of obstacles that are confirmed to exist by millimeter wave radar but are missed by visual recognition, and the features of clear lane lines verified by IMU assistance, are all determined to be effective target features; while false features that are repeatedly denied by sensor data (such as road surface shadows that are mistakenly identified as obstacle features) are determined to be invalid target features. A weight distribution model guided by attention mechanism is introduced, the feature channels corresponding to the effective target features are taken as the key optimization objects, the matching pass rate gap ratio (the formula is: matching pass rate gap ratio = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold) is calculated, the weight coefficient of the effective feature channel is adaptively improved according to the gap ratio, and the improvement range is gap ratio x 0.3-0.5. For example, if the real-time verification pass rate threshold is 90%, the actual matching pass rate is 72%, and the gap ratio is 20%, the weight coefficient of the effective feature channel is improved by 20% x 0.3-0.5 = 6%-10%, ensuring that the larger the gap, the stronger the strengthening of the effective features. For invalid feature channels, L1 regularization constraint is used for sparse inhibition, and the inhibition range is gap ratio x 0.4-0.6. By reducing the weight coefficient of the invalid feature channel, the interference of the invalid feature channel on subsequent feature extraction is reduced, and repeated misjudgments are avoided. During the weight adjustment process, normalization processing is used to ensure that the sum of the weight coefficients of all feature channels is 1, maintaining the stability of the algorithm parameters.

[0092] At the same time of weight correction, the real-time verification pass rate threshold is dynamically optimized, and a double-factor linkage adjustment strategy of core factor + auxiliary factor is adopted. The core factor is the difference ratio of matching pass rate and real-time verification pass rate threshold, and the calculation method is: difference ratio = (real-time verification pass rate threshold - matching pass rate) / real-time verification pass rate threshold; the auxiliary factor is the light intensity data collected by the ambient light sensor, which is divided into three levels according to the light intensity: weak light level (<300 lux), normal light level (300-10000 lux), and strong light level (>10000 lux). The threshold adjustment rule is set as follows: the real-time verification pass rate threshold is lowered by 0.03-0.05 based on the difference ratio every 5% increase; at the same time, the auxiliary factor is corrected: the threshold is additionally lowered by 0.02-0.03 in weak light environment, by 0.01-0.02 in strong light environment, and no additional adjustment in normal light environment. For example, the difference ratio is 10%, and it is in weak light environment, then the threshold is lowered by (10% ÷ 5%) × (0.03-0.05) + (0.02-0.03) = 0.08-0.13. At the same time, the threshold constraint range is set: the adjusted threshold is not less than 0.6 (to avoid excessive false positives caused by too low threshold), and not higher than 0.9 (to avoid excessive correction caused by too high threshold); if the adjusted threshold exceeds the range, it is automatically adjusted to the nearest constraint boundary, to ensure the rationality and safety of threshold adjustment.

[0093] Then, a preset proportion threshold (such as 60%) and a continuous frame threshold (such as 3-5 frames) are used to monitor the matching pass rate of each target category in real time: if the matching pass rate of a certain target category is lower than 60% for 3-5 continuous frames, it is determined that the current feature extraction logic is invalid in this scene (such as lane line feature extraction failure in a night weak light scene, obstacle feature being blocked in a rainy and foggy day), and an emergency correction mechanism is triggered immediately. The vehicle-mounted intelligent terminal pre-stores multiple groups of scene-specific feature templates, including night weak light lane line templates, rainy and foggy day obstacle templates, strong light and backlight traffic sign templates, etc., each of which is optimized for the target feature characteristics of a specific scene (such as night templates strengthening the brightness contrast features of lane lines, rainy and foggy day templates strengthening the contour edge features of obstacles). According to the current scene type (determined by the ambient light sensor, IMU and image data), the corresponding scene-specific feature template is automatically called to replace the original general feature extraction logic, quickly improving the recognition of target features. At the same time, the corrected feature extraction weight and the adjusted real-time verification pass rate threshold are fed back to the multi-scale feature enhancement and lightweight Transformer inference model in real time, updating the weight distribution rules and inference threshold determination standards of the model in the feature enhancement process. The corrected parameters take effect immediately in subsequent frame image processing, forming a full-closed-loop optimization link of verification-correction-feedback-iteration, enabling the AI vision algorithm to adapt to changes in different complex scenes, realizing the continuous upgrading of dynamic visual perception in driving scenes, and significantly improving the perception robustness and reliability in extreme scenes.

[0094] In an embodiment, the method further comprises:

[0095] Based on the different scene typical interference features stored in the scene meta-knowledge base, interference samples matching the current driving scene are generated, which are fused with the enhanced feature map to form an anti-interference enhanced feature map;

[0096] The anti-interference enhanced feature map is input into the lightweight Transformer inference model, and the consistency of the vehicle-mounted sensor heterogeneous data and the recognition result is taken as a self-supervised signal. When the matching pass rate is higher than a preset threshold, the corresponding anti-interference enhanced feature map is stored in the high-quality feature pool, and the convolution kernel parameters of the feature extraction network are optimized. The optimized parameters are updated to the corresponding scene in the scene meta-knowledge base synchronously;

[0097] The similarity between the current driving scene and each scene in the scene meta-knowledge base is calculated in real time, and when the similarity is lower than a similarity threshold, it is determined as a new scene. The optimized parameters of the most similar scene in the scene meta-knowledge base are migrated, and the anti-interference enhanced feature map of the same type of scene in the high-quality feature pool is combined. The new scene is adapted through multi-frame fine-tuning, and the adapted new scene parameters and anti-interference enhanced feature map are stored in the scene meta-knowledge base and high-quality feature pool synchronously;

[0098] The output distribution of the real-time monitoring feature extraction network is monitored in real time, and when the output distribution offset is greater than a threshold value, the anti-interference enhanced feature map in the high-quality feature pool and the optimal parameters in the scene meta knowledge base are automatically called to perform online calibration on the feature extraction network. New anti-interference enhanced feature maps are generated synchronously during the calibration process.

[0099] In the present embodiment, the scene meta knowledge base pre-stores massive driving scene data, covering typical scenes such as mountainous area sharp bend, city roundabout, highway section, rainy and foggy day, and night strong light, and each scene is associated with corresponding typical interference features (such as fogging blocking features in rainy and foggy scenes, strong light glare features in night scenes, target contour blurring features caused by road surface shadows, etc.). In the feature enhancement stage, the vehicle-mounted intelligent terminal real-time collects the key parameters of the current driving scene (including environmental light intensity, weather condition, road type, vehicle driving speed), retrieves the typical scene with the highest similarity to the current scene from the scene meta knowledge base through a scene matching algorithm, and extracts the interference features corresponding to the typical scene. Subsequently, the retrieved typical interference features and the generated original enhanced feature map are input into the lightweight generative adversarial network, and the generative adversarial network simulates the real interference pattern under the current scene (such as adjusting the brightness and range of glare according to the real-time light intensity, adjusting the fog concentration distribution according to the weather condition) through the generator, generates interference samples highly matched with the current driving scene; and then the interference samples and the original enhanced feature map are fused at the pixel level through a fusion algorithm to ensure that the original effective features are not covered by interference information, and finally an anti-interference enhanced feature map with both anti-interference ability and feature integrity is formed, providing more robust input data for subsequent reasoning models.

[0100] The generated anti-interference enhanced feature map is input into a lightweight Transformer inference model in real time, and the model outputs the recognition results (category label, position coordinates, and confidence) of the key targets in the driving environment. After that, real-time data of vehicle-mounted heterogeneous sensors (millimeter wave radar, ultrasonic sensor, IMU, etc.) are synchronously collected, and a multi-source data verification system is constructed. The consistency of vehicle-mounted sensor heterogeneous data and visual recognition results is taken as a self-supervised signal, and the matching pass rate of the two is calculated through a pre-set matching rule. For example, the position coordinates of the visual recognition target are spatially registered with the radar distance data, and a position deviation of ≤0.5 meters is determined as spatial matching. The motion state of the visual recognition target is time-aligned with the radar speed data, and a speed deviation of ≤5 km / h is determined as motion state matching. The matching pass rate = (number of targets that are spatially matched and have motion state matching ÷ total number of recognition targets of this category) x 100%. A pre-set matching pass rate threshold (such as 90%) is set. When the matching pass rate of a certain target is higher than the threshold, it is determined that the corresponding anti-interference enhanced feature map contains high-reliability features, and it is automatically stored in the high-quality feature pool of the vehicle terminal. The feature extraction network kernel parameters corresponding to the feature map are optimized: using a contrastive learning algorithm, taking the anti-interference enhanced features in the high-quality feature pool as positive samples and historical low-matching-rate features as negative samples, iteratively adjusting the weight coefficients of the convolution kernel, so that the feature extraction network can more accurately capture anti-interference and high-confidence target features; the optimized convolution kernel parameters are updated to the feature extraction-inference parameter set of the corresponding scene in the scene meta-knowledge base in real time through the data synchronization interface, realizing the dynamic iteration of the knowledge base.

[0101] The vehicle-mounted intelligent terminal constructs a feature vector of the current driving scene by collecting scene parameters (road type, environmental interference type, traffic participant density, etc.) in real time, and calculates the similarity of the feature vector with all typical scene feature vectors in the scene meta-knowledge base using a cosine similarity algorithm. A preset similarity threshold (e.g., 60%) is set, and when the calculated similarity is lower than the threshold, the current scene is determined to be a new scene (e.g., a new type of traffic facility section or a mixed scene under special weather conditions) that is not stored in the knowledge base. The meta-learning adaptation mechanism is started: first, the typical scene with the highest similarity to the current new scene is selected from the scene meta-knowledge base, and the optimized feature extraction network kernel parameters and inference model configuration parameters of the typical scene are transferred as the initial parameters of the new scene; then, the anti-interference enhanced feature map consistent with the interference type of the new scene is retrieved from the high-quality feature pool, and the initial parameters are quickly fine-tuned in combination with the first 3-5 frames of image data of the new scene. Through the gradient update strategy of the meta-learning algorithm, the parameters can be adapted to the feature distribution of the new scene after only a small number of iterations; after the adaptation is completed, the feature vector of the new scene, the optimized parameter set, and the corresponding anti-interference enhanced feature map are stored in the scene meta-knowledge base and the high-quality feature pool, realizing the real-time supplement of the new scene data and ensuring that the same scene can be directly called when it occurs again in the future without repeating the adaptation process.

[0102] The output distribution of the real-time monitoring feature extraction network is compared with the standard feature distribution of the same scene in the high-quality feature pool, and the distribution offset of the two is calculated. A preset offset threshold is set, and when the offset is detected to exceed the threshold, it is determined that the feature extraction network has drifted due to long-term operation (e.g., hardware aging or changes in environmental interference patterns leading to a decrease in feature extraction accuracy), and the online calibration process is immediately started: the anti-interference enhanced feature map with the highest matching pass rate under the same scene is retrieved from the high-quality feature pool, and the optimal parameters of the corresponding scene are extracted from the scene meta-knowledge base, which are used as the calibration sample and calibration benchmark, respectively. The convolution kernel parameters and activation function threshold of the feature extraction network are fine-tuned through the back propagation algorithm to make the network output distribution tend to the standard distribution again. During the calibration process, the interference sample generation process of the lightweight generative adversarial network is started simultaneously: based on the network output defects after feature drift (e.g., insensitivity to feature extraction of a certain type of interference), targeted new interference samples (e.g., enhanced feature intensity of the type of interference) are generated, the new interference samples are fused with the current frame image to form a new anti-interference enhanced feature map; the new feature map is stored in the high-quality feature pool, which in turn feeds back the interference sample generation process, making the subsequently generated interference samples more suitable for the optimization needs after network calibration, forming a closed-loop feedback link of calibration-new feature generation-knowledge base update.

[0103] In an embodiment, the method further comprises:

[0104] When the specific target is included in the recognition result, the image in the corresponding time period in the driving scene image data is intercepted as target image data;

[0105] Obtain the identification information of the vehicle-mounted intelligent terminal and the version information of the AI vision algorithm; generate a change factor based on the identification information, change the array elements of the preset key character array, and obtain a changed key character array;

[0106] Generate a curve based on each digital character in the version information;

[0107] Generate a first sub-key based on the curve and the changed key character array, and generate a second sub-key based on the first sub-key and the preset key character array;

[0108] Combine the first sub-key and the second sub-key to generate a combined key, which is used for independent encryption storage of the target image data.

[0109] In this embodiment, after receiving the driving environment key target recognition result output by the lightweight Transformer inference model, the vehicle-mounted intelligent terminal determines the target category of the recognition result in real time. The specific target is a preset high-safety-priority target (such as an accident scene obstacle, a pedestrian crossing the road in violation of rules, a new traffic sign, a sudden road anomaly, etc.). The image data corresponding to this type of target has tracing, analysis or evidence value and needs to be stored separately. When the recognition result contains the above-mentioned specific target, the identification frame timing information (including the frame sequence number at which the target first appears and the frame interval during which the target exists) corresponding to the specific target is automatically extracted, and the continuous image frames in the corresponding time period in the driving scene image data are intercepted based on the timing information. The interception range is centered on the specific target and is extended to a preset pixel range (such as 200 pixels horizontally and 150 pixels vertically) around the target and a preset time length, ensuring that the target and the surrounding associated scene information are completely included. The continuous image frames after interception are integrated into target image data and temporarily stored in the temporary cache area of the vehicle-mounted terminal, waiting for encryption processing.

[0110] The hardware information reading module of the vehicle-mounted intelligent terminal obtains terminal unique identification information (such as a device serial number, a MAC address, and a hardware encryption chip built-in identification) and current running AI vision algorithm version information (including a major version number, a minor version number, and a revision number), wherein the identification information is unique and is used to distinguish different vehicle-mounted terminals, and the version information is used to adapt to key synchronization after algorithm iteration. A change factor is generated based on the obtained identification information: the identification information is subjected to a hash operation, the first 16 characters of the hash result are extracted as a basic factor, and the last 8 digits of the current system timestamp are combined to generate the final change factor through an XOR operation. A fixed-length key character array (such as a 64x64 array, the array elements are composed of letters, numbers, and special symbols, and are pre-stored in the secure storage area of the vehicle-mounted terminal) is preset, and the generated change factor is used as an array element change rule. The replacement position of the array element is determined according to the ASCII code value of the character of the change factor (for example, the sum of the ASCII code values is taken modulo 64 to obtain the replacement row number and column number), and the array element at the corresponding position is replaced with the corresponding character of the change factor to generate a change key character array that is uniquely bound to the current vehicle-mounted terminal, thereby ensuring the device specificity of the key basic information.

[0111] Furthermore, each numerical character in the AI vision algorithm version information is extracted, and non-numerical characters in the version number are ignored. The order of the numerical characters is used as the horizontal coordinate, and the numerical value of the numerical characters is used as the vertical coordinate to construct a two-dimensional coordinate point set; a Bezier curve fitting algorithm is used to simulate a curve for the coordinate point set, the number of numerical characters in the version number is used to determine the order of the curve during the fitting process, and an interpolation operation is used to supplement the intermediate nodes of the curve to generate a continuous and smooth two-dimensional curve, the shape of the curve is strongly associated with the version information, and different algorithm versions correspond to different curve characteristics.

[0112] Next, a first sub-key is generated based on the generated curve and the change key character array. In one embodiment, the curve is mapped to the two-dimensional plane of the change key character array (the horizontal coordinate corresponds to the array column number, and the vertical coordinate corresponds to the array row number), the characters at the intersection of the curve and the array and the array elements within a predetermined range (such as a 3x3 adjacent area) around the intersection are extracted, these characters are arranged in the order of the curve to form a 32-bit string, and the first sub-key is generated, which is associated with the terminal identification and the algorithm version and has a dual association. Next, a second sub-key is generated based on the first sub-key and the preset key character array: the first sub-key and the preset key character array can be subjected to a bitwise XOR operation (the characters are subjected to the operation in the form of ASCII code values), the XOR result is subjected to Base64 encoding processing, the first 32 characters after the encoding are extracted as the second sub-key, and the second sub-key is further associated with the original preset key to improve the complexity and anti-cracking ability of the key.

[0113] Finally, the generated first sub-key and the second sub-key are spliced in the order of first sub-key + second sub-key to form a combined key, which integrates the vehicle terminal identifier, the AI vision algorithm version, and the preset key three pieces of information, and has uniqueness, correlation, uniqueness, and dynamics. The combined key is used as an encryption key to encrypt the target image data in the temporary buffer area. In the encryption process, the target image data is block encrypted (each block is 4KB in size), and a check code (such as CRC32 check) is added to each data block to ensure the integrity of the data transmission and storage process. The encrypted target image data is stored in the encrypted storage partition of the vehicle terminal (the partition has a hardware encryption protection function and can only be decrypted by the corresponding combined key), and the encryption time, the specific target type, and the basic information (identifier information digest, version information) required for combined key generation are recorded, facilitating subsequent tracing and decryption calling, and realizing safe and independent storage of specific target image data.

[0114] In the above embodiments, in order to make the description more detailed, clear, and complete, some existing algorithms and technical features are combined to explain and describe, so as to meet the requirements of the Patent Law. However, this is not to use a series of complicated steps and algorithm formulas, nor to complicate the technical solution, nor to use conventional or simple features to form a combination or stack. The existing algorithms and technical features listed are to disclose the specific implementation of each step (not limited to the present application) to avoid the present application from being unable to be implemented.

[0115] Reference Figure 2 In another embodiment of the present application, an AI vision algorithm processing device based on a vehicle-mounted intelligent terminal is also provided, which comprises:

[0116] The acquisition unit is configured to acquire real-time driving scene image data based on the vehicle-mounted intelligent terminal.

[0117] The generation unit is configured to perform multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation.

[0118] The output unit is configured to input the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, dynamically adjust the number of attention heads and the feature channel compression ratio of the lightweight Transformer inference model, and output the recognition result of the key target in the driving environment.

[0119] The correction unit is configured to cross-verify the recognition result and the heterogeneous data of the vehicle-mounted sensor, and perform real-time closed-loop correction on the feature extraction weight and the inference threshold of the AI vision algorithm to complete the dynamic visual perception of the driving scene.

[0120] In this embodiment, the specific implementation of each unit in the above device embodiment is described in the above method embodiment, and will not be repeated here.

[0121] Reference Figure 3 This invention also provides an electronic device whose internal structure can be as follows: Figure 3 As shown, the electronic device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor, designed as a computer, provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the electronic device stores the data corresponding to this embodiment. The network interface of the electronic device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0122] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present invention and do not constitute a limitation on the electronic devices to which the present invention is applied.

[0123] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0124] In summary, the AI vision algorithm processing method based on the vehicle-mounted intelligent terminal and the electronic device provided in the embodiments of the present application include: collecting real-time driving scene image data based on the vehicle-mounted intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, dynamically adjusting the number of attention heads and the feature channel compression ratio of the lightweight Transformer inference model, and outputting a recognition result including a driving environment key target; cross-verifying the recognition result and heterogeneous data of the vehicle-mounted sensor, and performing real-time closed-loop correction on the feature extraction weight and inference threshold of the AI vision algorithm to complete dynamic visual perception of the driving scene. In the present application, multi-scale feature enhancement is performed on the driving scene image data to generate an enhanced feature map with scene priority annotation, which dynamically adapts to the driving scene; the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio to balance the vehicle-mounted computing power and recognition accuracy; the recognition result and the heterogeneous data of the vehicle-mounted sensor are cross-verified to perform real-time closed-loop correction on the AI vision algorithm, thereby realizing closed-loop optimization of the AI vision algorithm.

[0125] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, databases, or other media in this application and in the embodiments refers to both non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0126] It is to be understood that the terminology "including", "comprising", or any other variation thereof, is intended to cover a non-exclusive inclusion such that process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0127] The above description is merely the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made according to the content of the present application specification and drawings, or directly or indirectly applied to other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An AI vision algorithm processing method based on a vehicle-mounted intelligent terminal, characterized in that, The method comprises the following steps: real-time acquisition of driving scene image data based on a vehicle-mounted intelligent terminal; performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation; inputting the enhanced feature map into an AI vision algorithm based on a lightweight Transformer inference model, dynamically adjusting the number of attention heads and the feature channel compression ratio of the lightweight Transformer inference model, and outputting a recognition result including key targets in the driving environment; cross-verification based on the recognition result and the heterogeneous data of the vehicle-mounted sensor, real-time closed-loop correction of the feature extraction weight and inference threshold of the AI vision algorithm, and completion of dynamic visual perception of the driving scene; including: dynamically generating a real-time verification pass rate threshold based on the recognition accuracy of each target category and the scene complexity in the historical driving data; including: in the real-time driving process, the vehicle-mounted intelligent terminal acquires the current scene complexity parameter in real time through the scene recognition module, inputs the current target category and the current scene complexity parameter into the trained GBDT model, and dynamically generates the real-time verification pass rate threshold of each target category; calculating the matching pass rate of the visual recognition result of each type of target and the heterogeneous data of the vehicle-mounted sensor, if the matching pass rate is higher than the real-time verification pass rate threshold, determining that the recognition result is valid, and maintaining the current algorithm parameters; if the matching pass rate is lower than the real-time verification pass rate threshold, starting the intelligent closed-loop correction mechanism: introducing a weight distribution model guided by an attention mechanism, verifying effective target features based on heterogeneous data, dynamically distributing feature channel weights, and adaptively improving the weight coefficients of effective feature channels according to the matching pass rate gap proportion; wherein the matching pass rate gap proportion=(real-time verification pass rate threshold-matching pass rate) / real-time verification pass rate threshold; the weight coefficients of ineffective feature channels are sparsely suppressed through regularization constraint; dynamically adjusting the real-time verification pass rate threshold with the difference ratio of the matching pass rate and the real-time verification pass rate threshold as the core factor and the ambient light intensity as the auxiliary factor; when the matching pass rate of a certain type of target is continuously lower than the proportion threshold for multiple frames, automatically calling a preset scene-specific feature template to replace the current feature extraction logic, and simultaneously feeding back the corrected parameters to the AI vision algorithm in real time, realizing adaptive upgrading of dynamic visual perception of the driving scene. 2.The AI vision algorithm processing method based on the intelligent terminal in the vehicle according to claim 1, characterized in that, Before performing multi-scale feature enhancement on the driving scene image data, the method further comprises: dynamically adjusting the sampling frame rate and pixel clipping area of the driving scene image data based on the dynamic working condition parameters of the vehicle, to form driving scene image data adapted to the current driving scene. 3.The AI vision algorithm processing method based on the intelligent terminal in the vehicle according to claim 1, characterized in that, The key targets in the driving environment include lane lines, obstacles, and traffic signs. 4.The AI vision algorithm processing method based on the intelligent terminal in a vehicle according to claim 1, characterized in that, Performing multi-scale feature enhancement on the driving scene image data to generate an enhanced feature map with scene priority annotation comprises: down-sampling the driving scene image data to obtain multiple image feature layers of different scales, and the resolution of each scale image feature layer decreases by 1 / 2 gradient; The edge gradient values of the lane line region in the image feature layer of each scale are calculated, and lane line feature weights are assigned based on the edge gradient values, a first weight being assigned to a region with an edge gradient value higher than a preset threshold, and a second weight being assigned to a region with an edge gradient value lower than the preset threshold, and the first weight > the second weight; A motion trend vector of a traffic participant in a driving scene is predicted, the motion trend vector including a motion direction and a motion speed; The lane line feature weights of the image feature layer of each scale are weighted and fused with the motion trend vector of the corresponding region traffic participant; wherein a third weight is assigned to a traffic participant pointing to the driving path of the vehicle, and a fourth weight is assigned to a traffic participant deviating from the driving path of the vehicle, and the third weight > the fourth weight; Based on the weighted fusion result, the image feature layers of each scale are feature-fused and pixel-level enhanced to generate an enhanced feature map with scene priority labeling. 5.The AI vision algorithm processing method based on the intelligent terminal in the vehicle according to claim 1, characterized in that, The enhanced feature map is input into a lightweight Transformer inference model, the lightweight Transformer inference model dynamically adjusts the number of attention heads and the feature channel compression ratio, and outputs an identification result including key targets in the driving environment, including: Real-time acquisition of hardware computing power state parameters of the vehicle-mounted intelligent terminal; A plurality of groups of computing power threshold intervals are preset, and the corresponding number of attention heads is dynamically matched according to the computing power threshold interval to which the current hardware computing power state parameter belongs; The scene priority labeling of the enhanced feature map is used to adjust the feature channel compression ratio; The lightweight Transformer inference model performs self-attention calculation on the adjusted feature channel through a hierarchical attention mechanism, strengthens the feature correlation weight of the high-priority region, and respectively outputs the class label, position coordinates and confidence of the key target in the driving environment to form a complete identification result. 6.The AI vision algorithm processing method based on the intelligent terminal in the vehicle according to claim 5, characterized in that, The hardware computing power state parameters include CPU occupancy, GPU memory usage and NPU operation throughput. 7.An electronic device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 6. The processor executes the computer program to realize the steps of the method of any one of claims 1 to 6.

Images