Intelligent automobile night vision target detection method and system

Abstract

The invention discloses an intelligent automobile night vision target detection method and system, and belongs to the technical field of automatic driving. The system comprises an infrared image acquisition module, a feature extraction network, an improved FPN, an ASF module, an OIOD module, a double-current sensing module and a detection output module. The method comprises the following steps: acquiring an infrared video stream, enhancing feature expression by utilizing an improved FPN, realizing target adaptive identification in an open environment through an OIOD module, processing static semantic and dynamic trend information at the same time by utilizing a double-flow sensing module, and finally outputting a high-precision target detection result and motion trend prediction thereof. The problems of low target detection precision, weak dynamic sensing capability and poor model adaptability at night and in a severe environment are effectively solved, the all-weather environment sensing capability and the driving safety of the intelligent automobile are remarkably improved, and vehicle-mounted deployment is easy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, autonomous driving and automotive electronics, specifically to a method for night vision target detection in intelligent vehicles, and also to a system for night vision target detection in intelligent vehicles. Background Technology

[0002] With the development of intelligent driving technology, significant progress has been made in target detection under visible light conditions. However, in adverse environments such as nighttime, no light, or rain and fog, the perception capability of visible light cameras drops sharply, resulting in problems such as low detection accuracy, high false negative rate, and inability to effectively predict the movement trend of dynamic targets, which seriously restricts the realization of all-weather autonomous driving.

[0003] Existing infrared night vision target detection schemes are mostly based on traditional single-frame image processing or improved deep learning models, but they still have the following inherent drawbacks:

[0004] (1) Poor environmental adaptability: The model relies heavily on a large amount of labeled data, has a weak ability to identify unknown or unlabeled obstacles (such as temporary roadblocks and strange vehicle models), and has poor adaptability;

[0005] (2) Lack of dynamic perception: Most methods are based on static image analysis and lack the ability to predict the movement trend of the target. They cannot cope with sudden situations such as "ghost peek" and emergency lane change, and cannot meet the needs of autonomous driving for time-series information processing.

[0006] (3) Insufficient information utilization: Due to the characteristics of infrared images such as simple texture, low resolution and blurred contours, traditional structures such as Feature Pyramid Network (FPN) are prone to semantic information loss during feature fusion, which affects the detection accuracy of small targets and blurred targets;

[0007] (4) Low system integration: The algorithm is too complex, making it difficult to achieve low latency and high efficiency real-time processing on the vehicle embedded platform, and making it difficult to implement in engineering.

[0008] Therefore, there is an urgent need for an infrared night vision detection solution that can simultaneously achieve high precision, high real-time performance, strong adaptability, and effectively predict the movement trend of targets. Summary of the Invention

[0009] The purpose of this invention is to provide a method and system for night vision target detection in intelligent vehicles, so as to improve the detection accuracy and recall rate of road targets (vehicles, pedestrians, obstacles, etc.) in intelligent vehicles at night and in adverse weather conditions, and reduce false negatives and false negatives; to realize intelligent prediction and perception of the movement trend of dynamic targets, providing forward-looking information for autonomous driving decision-making systems; to enhance the model's open recognition and adaptive learning capabilities for unknown target categories; to optimize the network structure, reduce computational complexity and memory consumption, and ensure that the algorithm can run efficiently and in real time on the vehicle computing platform.

[0010] To achieve the above functions, this invention designs an intelligent vehicle night vision target detection method, which executes the following steps S1-S7 to complete vehicle night vision target detection:

[0011] Step S1: Deploy an infrared camera at a preset position on the vehicle body, collect ambient infrared video streams in real time through the infrared camera, and perform noise reduction preprocessing;

[0012] Step S2: Extract the current frame and historical frames from the continuous video stream obtained in step S1, and input the current frame and historical frames into two parallel feature extraction backbone networks to obtain multi-level features of the current frame and historical frames respectively.

[0013] Step S3: Input the multi-level features of the current frame and the historical frame obtained in step S2 into two parallel improved feature pyramid networks. The improved feature pyramid network introduces a spatial adaptive module based on the FPN network. The spatial adaptive module performs semantic compensation and feature enhancement on the multi-level features to generate multi-scale feature maps of the current frame and the historical frame respectively.

[0014] Step S4: Input the multi-scale feature map obtained in step S3 into the open infrared target monitoring module, use the energy probability distribution to perform target analysis and open recognition, distinguish between known and unknown targets and learn step by step;

[0015] Step S5: Input the multi-scale feature maps of the continuous multi-frames obtained in step S4 into the dual-stream perception module, process the static stream and dynamic stream respectively, obtain the semantic and trend features of the target, and output the dual-stream fusion features that fuse the target's current state and future motion trend.

[0016] Step S6: Send the output of the dual-stream sensing module into the detection head and output module to complete the classification, localization and motion trend prediction of the target, and generate the final detection result;

[0017] Step S7: Output the detection results to the autonomous driving decision-making system for planning and control, as well as for visualization, to complete the vehicle night vision target detection.

[0018] As a preferred technical solution of the present invention: the feature extraction backbone network in step S2 adopts the Darknet-53 network as the backbone to extract multi-level features from the input infrared image sequence.

[0019] As a preferred technical solution of the present invention: the improved feature pyramid network in step S3 introduces a spatial adaptive module based on the FPN network, wherein the FPN network includes multiple convolutional layers from bottom to top, and a spatial adaptive module is introduced in the highest convolutional layer; the spatial adaptive module first inputs multi-level features x(0), x(1), ... x(n-1), and after connection, it is passed through a Conv 1x1 convolutional layer to obtain a feature map with C channels, height h, and width w. After obtaining this channel, a Conv 3x3 convolutional layer is used, and the feature map at this time is After outputting, the sigmoid activation function is used to obtain the final result. Feature map.

[0020] As a preferred technical solution of the present invention: the spatial adaptive module in the improved feature pyramid network consists of an upsampled feature input layer, an adaptive weight generation layer and a feature weighted fusion layer, wherein the adaptive weight generation layer includes a convolution operation with a kernel size of 3x3.

[0021] As a preferred technical solution of the present invention: in step S4, the input dimension of the open infrared target detection module is the feature map dimension, and the output dimension is the category probability value; the training of the energy probability distribution model adopts a semi-supervised learning strategy, initially using K-Means pre-clustering to generate seed categories, and subsequently updating the discriminator parameters through adversarial learning; the energy probability distribution model adopts an energy-based classifier, specifically as follows:

[0022] ;

[0023] Where f represents the feature of the hidden space F, l is the corresponding label, and T is its corresponding temperature parameter; the relationship between the network output after the softmax layer and the Gibbs distribution of the category-specific energy value is expressed as follows:

[0024] ;

[0025] In the formula, p(l|f) represents the probability density of label l given feature f. The freeness of the classification model is defined by its log odds as follows:

[0026] ;

[0027] In the formula, g(f) represents the probability of the number of arbitrary class pairs output by the classification head g(·).

[0028] As a preferred technical solution of the present invention: In step S5, the dual-stream sensing module extracts semantic features from the multi-scale feature maps of the input continuous multi-frames for both static and dynamic streams; for the static stream, for the current frame, it extracts semantic features from the single-frame multi-scale feature map, including target localization and classification; and outputs a semantic feature map, including the target's bounding box and class probability; for the dynamic stream, it fuses features from adjacent frames, calculates motion features between multi-frame multi-scale feature maps using optical flow or frame difference, including displacement and velocity, and outputs a motion feature map, including the target's position offset, velocity scalar, and movement trend; and fuses the static and dynamic streams into a dual-stream fusion feature, including the target's current position and predicted position, as well as the target type, movement trend, and velocity.

[0029] As a preferred embodiment of the present invention: In the dual-stream sensing module, the fully connected layer is used for classification tasks, and the convolutional layer is used for target bounding box regression and localization tasks. The loss function is expressed as:

[0030] ;

[0031] In the above formula, For the loss weights of the fully connected layer structure, The loss weights of the convolutional layer structure, For the loss function of the fully connected layer structure, The loss function of the convolutional layer structure, To extract the loss function generated from the candidate boxes, the SmoothL1Loss function and the cross-entropy loss function were used for regression and classification, respectively.

[0032] As a preferred technical solution of the present invention: In the dual-stream sensing module, the semantic features of the single-frame multi-scale feature map are extracted using the ResNet-18 network for the static stream, and the motion features between the multi-frame multi-scale feature maps are calculated using the TCN network for the dynamic stream.

[0033] As a preferred technical solution of the present invention: the detection head and output module in step S6 includes a classification head, a regression head and an object head, inputs the dual-stream fusion features output by the dual-stream perception module, generates the final bounding box, class confidence and predicted offset direction; after non-maximum suppression, the output obtains the visualized detection result, including the current detection box and the predicted motion trajectory.

[0034] This invention also designs an intelligent vehicle night vision target detection system, including an infrared image acquisition module, a feature extraction backbone network module, an improved feature pyramid network module, an open infrared target monitoring module, a dual-stream sensing module, a detection head and an output module, to realize the aforementioned intelligent vehicle night vision target detection method:

[0035] The infrared image acquisition module acquires ambient infrared video streams in real time through an infrared camera and performs noise reduction preprocessing.

[0036] The feature extraction backbone network module extracts the current frame and historical frames from the continuous video stream. The current frame and historical frames are then input into two parallel feature extraction backbone networks to obtain multi-level features of the current frame and historical frames, respectively.

[0037] The multi-level features of the current frame and historical frames are respectively input into two parallel improved feature pyramid network modules. The improved feature pyramid network modules introduce a spatial adaptive module based on the FPN network. The spatial adaptive module performs semantic compensation and feature enhancement on the multi-level features to generate multi-scale feature maps of the current frame and historical frames respectively.

[0038] The multi-scale feature map is input into the open infrared target monitoring module, and the energy probability distribution is used for target analysis and open recognition to distinguish between known and unknown targets and learn step by step.

[0039] Multi-scale feature maps from multiple consecutive frames are input into the dual-stream perception module, which processes the static and dynamic streams respectively to obtain the semantic and trend features of the target and outputs a dual-stream fusion feature that integrates the target's current state and future motion trend.

[0040] The output of the dual-stream sensing module is sent to the detection head and output module to complete the classification, localization and motion trend prediction of the target, and generate the final detection result.

[0041] The detection results are output to the autonomous driving decision-making system for planning and control, as well as for visualization, to complete the detection of targets in the vehicle's night vision.

[0042] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0043] 1. High detection accuracy: The improved FPN structure (including the ASF module) effectively reduces the loss of semantic information in infrared image feature fusion, and the OIOD module enhances the model's adaptability to unknown targets, thereby significantly improving the detection accuracy (up to 90%) and recall rate in complex night vision environments.

[0044] 2. Strong dynamic perception capability: The innovative dual-stream perception (DFP) module breaks through the limitations of single-frame image processing, and can effectively predict the target's motion trend and future position, providing valuable early warning time for vehicles to deal with emergencies and greatly improving safety;

[0045] 3. Adaptive and Open Recognition: The OIOD mechanism enables the system to no longer rely entirely on a predefined closed category set, and has the ability to discover and gradually learn novel targets on actual open roads, making it more robust;

[0046] 4. Good real-time performance and easy deployment: The network structure is carefully designed and optimized, with high computing efficiency and relatively small memory footprint (detection speed ≥30fps, latency <1s), which can meet the real-time requirements of the vehicle embedded platform and is easy to integrate into the existing intelligent vehicle system to achieve integrated deployment;

[0047] 5. High cost-effectiveness: It mainly relies on software algorithm innovation to improve performance, and has relatively simple requirements for hardware sensors (mainly relying on infrared cameras), which has good cost-effectiveness and promotion prospects. Attached Figure Description

[0048] Figure 1 This is an overall architecture diagram of an intelligent vehicle night vision target detection method and system provided according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of an improved feature pyramid network structure provided according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram illustrating the working principle of the open infrared target monitoring module provided in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the structure and data processing flow of the dual-stream sensing module provided in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram comparing the detection results of the conventional method provided by the present invention with the detection results of the method of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0054] This invention provides an intelligent vehicle night vision target detection method, referring to... Figure 1 Perform the following steps S1-S7 to complete the vehicle night vision target detection:

[0055] Step S1: Deploy an infrared camera at a preset position on the vehicle body, collect ambient infrared video streams in real time through the infrared camera, and perform noise reduction preprocessing;

[0056] In this embodiment, the hardware platform adopts an in-vehicle embedded system, equipped with an infrared camera and a GPU (such as NVIDIA Jetson AGX Orin); the software is developed based on the PyTorch framework and uses infrared datasets such as FLIR ADAS for training and validation.

[0057] The infrared image acquisition module uses an automotive-grade far-infrared camera with a frame rate of no less than 30fps.

[0058] Step S2: Extract the current frame and historical frames from the continuous video stream obtained in step S1, and input the current frame and historical frames into two parallel feature extraction backbone networks to obtain multi-level features of the current frame and historical frames respectively.

[0059] The feature extraction backbone network uses the Darknet-53 network as its backbone, and its pre-trained weights can be obtained from ImageNet. The feature extraction backbone network extracts multi-level features from the input infrared image sequence.

[0060] Step S3: Input the multi-level features of the current frame and the historical frame obtained in step S2 into two parallel improved feature pyramid networks (Enhanced FPN). The improved feature pyramid network introduces a spatial adaptation (ASF) module based on the FPN network. The spatial adaptation module performs semantic compensation and feature enhancement on the multi-level features to generate multi-scale feature maps of the current frame and the historical frame respectively.

[0061] Improved Feature Pyramid Network Structure Reference Figure 2 In the figure, C2-C5 are the feature maps corresponding to the convolutional layers from bottom to top, M2-M5 are the ROI feature layers, P2~P5 are the corresponding feature fusion layers, RPN is the Region Proposal Network, ROI Align is the alignment of the region of interest, class is the classification, and box is the bounding box.

[0062] The improved feature pyramid network introduces a spatial adaptive module based on the FPN network. The FPN network includes multiple convolutional layers from bottom to top, and the spatial adaptive module is introduced in the highest convolutional layer. This module compensates for the semantic information lost due to upsampling and convolution operations by generating an adaptive weight map, and then fuses the weighted features with the features of the lower layers to form an enhanced multi-scale feature map, which significantly improves the feature representation ability of small targets and blurred targets in infrared images.

[0063] The spatial adaptive module first inputs multi-level features x(0), x(1), ... x(n-1), which are then concatenated and passed through a Conv1x1 convolutional layer to obtain a feature map with C channels, h height, and w width. After obtaining this feature map, a Conv3x3 convolutional layer is applied, resulting in the following feature map: After outputting, the sigmoid activation function is used to obtain the final result. Feature map.

[0064] The spatial adaptive module in the improved feature pyramid network consists of an upsampled feature input layer, an adaptive weight generation layer, and a feature weighted fusion layer. The adaptive weight generation layer includes a convolution operation with a kernel size of 3x3. The weight map is optimized using the L1 loss function to compensate for the loss of high-dimensional feature information.

[0065] Reference Figure 2 The specific structure of the spatial adaptive module is as follows: the upsampled features are used as input, and a weight value is generated for each feature. The generated weight value is used to obtain the feature layer M6. Finally, the M5 layer is added to the M6 ​​layer through a 3x3 convolution (the purpose of which is mainly to eliminate the aliasing effect caused by upsampling) and extended downward as the P5 layer to solve the problem of information loss.

[0066] Step S4: Input the multi-scale feature map obtained in step S3 into the Open Infrared Target Detection (OIOD) module, use the energy probability distribution to perform target analysis and open recognition, distinguish between known and unknown targets and learn step by step;

[0067] Reference Figure 3 The open-source infrared target detection module employs an energy probability distribution model combined with a clustering algorithm to perform online analysis and classification of potential targets in the feature map. For "Unknown" targets (unknown to the model), it can initially identify and cluster them based on their energy distribution characteristics, and gradually transform them into known "Known" categories through subsequent learning, endowing the system with adaptive recognition and continuous learning capabilities in an open environment. The input dimension of the open-source infrared target detection module is the feature map dimension, and the output dimension is the category probability value. The energy probability distribution model adopts a logistic regression neural network structure, and the training uses a semi-supervised learning strategy. The clustering algorithm uses the DBSCAN density clustering method. In the initial stage, K-Means pre-clustering is used to generate seed categories, and subsequent adversarial learning is used to update the discriminator parameters.

[0068] The energy probability distribution model employs an energy-based classifier that learns the degree of matching between input features and labels to identify unknown targets. Given features and labels, an energy function is learned to obtain a scalar describing the degree of matching between features and labels. The sum of results for all labels is calculated using the Helmholtz free energy formula, as shown below:

[0069] ;

[0070] Where f represents the feature of the hidden space F, l is the corresponding label, and T is its corresponding temperature parameter; the relationship between the network output after the softmax layer and the Gibbs distribution of the category-specific energy value is expressed as follows:

[0071] ;

[0072] In the formula, p(l|f) represents the probability density of label l given feature f. The freeness of the classification model is defined by its log odds as follows:

[0073] ;

[0074] In the formula, g(f) represents the odds of the rank-1 logarithm output by the classification head g(·). Based on this trend, a set of offset Weiber distributions is used to model the energy distribution of the energy values ​​of the known and unknown classes. Compared with the gamma, exponential, and normal distributions, this distribution can fit the energy data of a small reserved validation set (containing samples of known and unknown classes) very well. If the known energy value is less than the unknown energy value, the predicted result can be labeled as the unknown class using the learned distribution.

[0075] Step S5: Input the multi-scale feature maps of the consecutive frames obtained in step S4 into the dual-stream perception (DFP) module, process the static stream and dynamic stream respectively, obtain the semantic and trend features of the target, and output the dual-stream fusion features that fuse the target's current state and future motion trend.

[0076] Reference Figure 4 Historical frame A consists of one or more frames from frame T to frame T+n-1, used to provide historical feature information. These frames contain information such as the scene and target state at past moments. Based on this, the data input layer separates historical frame A and current frame B from the continuous video frame sequence and inputs them into two parallel sub-networks: a "feature extraction backbone network + FPN network". Features are extracted from historical frame A and current frame B to generate a multi-scale feature pyramid. Then, cross-frame feature fusion and interaction are performed to achieve the perception of information such as motion trends and semantic consistency.

[0077] The dual-stream perception module extracts semantic features from the multi-scale feature maps of multiple consecutive input frames for both static and dynamic streams. For the static stream, it extracts semantic features from the single-frame multi-scale feature map for the current frame (e.g., frame T+n), including target localization and classification, and outputs a semantic feature map including the target's bounding box and class probability. For the dynamic stream, it fuses features from adjacent frames, calculates motion features (e.g., displacement and velocity) between multi-frame multi-scale feature maps (e.g., from frame T to T+n) using optical flow or frame difference, and outputs a motion feature map including the target's position offset, velocity scalar, and movement trend (e.g., "fast movement" and "stationary"). The static and dynamic streams are then fused into a dual-stream fusion feature, which includes the target's current and predicted positions, target type, movement trend, and velocity.

[0078] To better ensure the performance of object regression and object localization tasks, we adopt the Double-Head structure, treating the fully connected layers as classification tasks and the convolutional layers as object bounding box regression and localization tasks. Therefore, the loss function after adopting the Double-Head structure is expressed as:

[0079] ;

[0080] In the above formula, For the loss weights of the fully connected layer structure, The loss weights of the convolutional layer structure, For the loss function of the fully connected layer structure, The loss function of the convolutional layer structure, To extract the loss function generated by the candidate boxes, the SmoothL1Loss function and the cross-entropy loss function are used for regression and classification, respectively. Therefore, by utilizing the advantages of fully connected layer structure and convolutional layer structure, the problems of low accuracy and missed detection in infrared target detection can be effectively solved.

[0081] In the dual-stream sensing module, the ResNet-18 network is used to extract semantic features of single-frame multi-scale feature maps for static streams, and the TCN (Temporal Convolutional Network) network is used to calculate motion features between multi-frame multi-scale feature maps for dynamic streams.

[0082] The loss function for dynamic flow includes a motion prediction loss term (MSE Loss) and a spatiotemporal consistency loss term (Charbonnier Loss), with weighting coefficients of 0.6 and 0.4, respectively.

[0083] A two-stage strategy was employed during training: first, the basic detection network was trained, and then some parameters were frozen for fine-tuning of the DFP module. Testing showed that this embodiment significantly improved mAP on the FLIR dataset. Figure 5 As shown in Table 1, and in actual road tests, it can effectively predict the movement intentions of pedestrians and vehicles, verifying the effectiveness and advancement of the present invention.

[0084] Table 1. Experimental Data Results

[0085]

[0086] Step S6: Send the output of the dual-stream sensing module into the detection head and output module to complete the classification, localization and motion trend prediction of the target, and generate the final detection result;

[0087] The detection head and output module in step S6 includes a classification head, a regression head, and an object head. It inputs the dual-stream fusion features output by the dual-stream perception module to generate the final bounding box, class confidence, and predicted offset direction. After non-maximum suppression (NMS), the output is visualized, including the current detection box and the predicted motion trajectory.

[0088] Step S7: Output the detection results to the autonomous driving decision-making system for planning and control, as well as for visualization, to complete the vehicle night vision target detection;

[0089] In this embodiment, the detection results containing a confidence threshold ≥ 0.9 are output to the autonomous driving decision-making system.

[0090] This invention also provides an intelligent vehicle night vision target detection system, including an infrared image acquisition module, a feature extraction backbone network module, an improved feature pyramid network module, an open infrared target monitoring module, a dual-stream sensing module, a detection head, and an output module, to implement the aforementioned intelligent vehicle night vision target detection method.

[0091] The infrared image acquisition module acquires ambient infrared video streams in real time through an infrared camera and performs noise reduction preprocessing.

[0092] The feature extraction backbone network module extracts the current frame and historical frames from the continuous video stream. The current frame and historical frames are then input into two parallel feature extraction backbone networks to obtain multi-level features of the current frame and historical frames, respectively.

[0093] The multi-level features of the current frame and historical frames are respectively input into two parallel improved feature pyramid network modules. The improved feature pyramid network modules introduce a spatial adaptive module based on the FPN network. The spatial adaptive module performs semantic compensation and feature enhancement on the multi-level features to generate multi-scale feature maps of the current frame and historical frames respectively.

[0094] The multi-scale feature map is input into the open infrared target monitoring module, and the energy probability distribution is used for target analysis and open recognition to distinguish between known and unknown targets and learn step by step.

[0095] Multi-scale feature maps from multiple consecutive frames are input into the dual-stream perception module, which processes the static and dynamic streams respectively to obtain the semantic and trend features of the target and outputs a dual-stream fusion feature that integrates the target's current state and future motion trend.

[0096] The output of the dual-stream sensing module is sent to the detection head and output module to complete the classification, localization and motion trend prediction of the target, and generate the final detection result.

[0097] The detection results are output to the autonomous driving decision-making system for planning and control, as well as for visualization, to complete the detection of targets in the vehicle's night vision.

[0098] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for detecting a target at night by an intelligent vehicle, characterized in that, The following steps S1-S7 are performed to complete the automobile night vision target detection: Step S1: deploy an infrared camera at a predetermined position of the vehicle body, collect an infrared video stream of the environment in real time through the infrared camera, and perform denoising preprocessing; Step S2: extract a current frame and a historical frame from the continuous video stream obtained in step S1, and input the current frame and the historical frame into two parallel feature extraction backbone networks respectively to obtain multi-level features of the current frame and the historical frame respectively; Step S3: input the multi-level features of the current frame and the historical frame obtained in step S2 into two parallel improved feature pyramid networks respectively, introduce a spatial adaptive module based on the FPN network into the improved feature pyramid network, perform semantic compensation and feature enhancement on the multi-level features through the spatial adaptive module, and generate multi-scale feature maps of the current frame and the historical frame respectively; Step S4: input the multi-scale feature maps obtained in step S3 into an open infrared target monitoring module, perform target analysis and open recognition using an energy probability distribution, distinguish known and unknown targets, and gradually learn; Step S5: input the multi-scale feature maps of the continuous multiple frames obtained in step S4 into a dual-stream perception module, process static streams and dynamic streams respectively, obtain semantic and trend features of the target, and output dual-stream fusion features fused with the instant state and future motion trend of the target; Step S6: send the output of the dual-stream perception module into a detection head and an output module, complete target classification, positioning and motion trend prediction, and generate a final detection result; Step S7: output the detection result to an automatic driving decision system for planning and control, and perform visual display, thereby completing the automobile night vision target detection. 2.The intelligent automobile night vision target detection method of claim 1, wherein, The feature extraction backbone network in step S2 adopts a Darknet-53 network as the backbone to extract multi-level features from the input infrared image sequence. 3.The intelligent automobile night vision target detection method of claim 1, wherein, The improved feature pyramid network in step S3 introduces a spatial adaptive module based on the FPN network, wherein the FPN network comprises a plurality of convolutional layers from bottom to top, and the spatial adaptive module is introduced at the highest convolutional layer; the spatial adaptive module first inputs the multi-level features x(0), x(1), … x(n-1), is connected through a Conv 1x1 convolutional layer, and the obtained feature map has a channel number C, a height h and a width w; after the channel is obtained, a Conv 3x3 convolutional layer is used, at this time, the feature map is , and after output, a sigmoid activation function is used to finally obtain a feature map . 4.The intelligent automobile night vision target detection method of claim 3, wherein, The spatial adaptive module in the improved feature pyramid network is composed of an up-sampling feature input layer, an adaptive weight generation layer and a feature weighted fusion layer, wherein the adaptive weight generation layer includes a convolution operation with a convolution kernel size of 3x3.

5. The intelligent automobile night vision target detection method of claim 1, wherein, The input dimension of the open infrared target monitoring module in step S4 is the feature map dimension, and the output dimension is the class probability value; the training of the energy probability distribution model adopts a semi-supervised learning strategy, uses K-Means pre-clustering to generate seed classes in the initial stage, and updates the discriminator parameters through adversarial learning in the subsequent stage; the energy probability distribution model adopts an energy-based classifier, which is specifically as follows: ； wherein f is the feature of the hidden space F, l is the corresponding label, and T is the corresponding temperature parameter; the expression between the network output after the softmax layer processing and the Gibbs distribution of the class-specific energy value is as follows: ； wherein p (l|f) represents the probability density of the label l under the condition of the feature f, and the freedom of the classification model is defined as the logarithmic ratio as follows: ； wherein g(f) represents the logarithmic ratio of any classification output by the classification head g(·). 6.The intelligent automobile night vision target detection method of claim 1, wherein, The double-flow perception module in step S5 extracts the static flow and the dynamic flow respectively for the input multi-scale feature maps of continuous multiple frames; wherein for the static flow, the semantic features of the single-frame multi-scale feature map are extracted for the current frame, including the positioning and classification of the target; and the semantic feature map is output, including the positioning frame and the category probability of the target; for the dynamic flow, the features of adjacent frames are fused, and the motion features between the multi-frame multi-scale feature maps are calculated through the optical flow or the frame difference, including the displacement and the speed, and the motion feature map is output, including the position offset, the speed scalar and the moving trend of the target; the static flow and the dynamic flow are fused into the double-flow fusion features, including the current position and the predicted position of the target, and the target type, the moving trend and the speed.

7. The intelligent automobile night vision target detection method according to claim 6, characterized in that, In the double-flow perception module, the fully connected layer is used as the classification task, and the convolutional layer is used as the target frame regression positioning task, and the loss function is represented as: ； In the above formula, is the loss weight of the full connection layer structure, is the loss weight of the convolution layer structure, is the loss function of the full connection layer structure, is the loss function of the convolution layer structure, is the loss function generated by extracting the candidate frame, and the regression and classification respectively adopt the SmoothL1Loss function and the cross-entropy loss function. 8.The intelligent automobile night vision target detection method of claim 6, wherein, In the double-flow perception module, the ResNet-18 network is used to extract the semantic features of the single-frame multi-scale feature map for the static flow, and the TCN network is used to calculate the motion features between the multi-frame multi-scale feature maps for the dynamic flow. 9.The intelligent automobile night vision target detection method of claim 1, wherein, The detection head and the output module in step S6 include the classification head, the regression head and the object head, input the double-flow fusion features output by the double-flow perception module, and generate the final bounding box, the category confidence and the predicted offset direction; after the non-maximum suppression, the visual detection results are obtained, including the current detection frame and the predicted motion trajectory.

10. An intelligent automotive night vision target detection system, characterized in that, The infrared image acquisition module, the feature extraction backbone network module, the improved feature pyramid network module, the open infrared target monitoring module, the double-flow perception module and the detection head and the output module are included to realize the intelligent automobile night vision target detection method of any one of claims 1-9: The infrared image acquisition module acquires the environmental infrared video stream in real time through the infrared camera and performs denoising preprocessing; The feature extraction backbone network module extracts the current frame and the historical frame in the continuous video stream, and inputs the current frame and the historical frame into two parallel feature extraction backbone networks respectively to obtain the multi-level features of the current frame and the historical frame respectively; The multi-level features of the current frame and the historical frame are input into two parallel improved feature pyramid network modules respectively, the improved feature pyramid network module introduces a spatial adaptive module based on the FPN network, performs semantic compensation and feature enhancement on the multi-level features through the spatial adaptive module, and generates the multi-scale feature maps of the current frame and the historical frame respectively; The multi-scale feature maps are input into the open infrared target monitoring module, target analysis and open recognition are performed using the energy probability distribution, known and unknown targets are distinguished and gradually learned; The multi-scale feature maps of continuous multiple frames are input into the double-flow perception module, and the static flow and the dynamic flow are processed respectively to obtain the semantic and trend features of the target, and the double-flow fusion features fused with the instant state and the future motion trend of the target are output; The output of the double-flow perception module is sent to the detection head and the output module to complete the classification, positioning and motion trend prediction of the target, and the final detection result is generated; The detection result is output to the automatic driving decision system for planning and control, and visual display is performed to complete the automobile night vision target detection.

Images