Infrared and visible light fusion pedestrian detection method based on weight autonomous optimization

Abstract

The invention belongs to the field of machine learning and target detection, and discloses an infrared and visible light fusion pedestrian detection method based on weight autonomous optimization. According to the method, feature extraction is carried out on infrared and visible light images through a convolutional neural network; and through a learnable proportionality coefficient, the extracted features are superposed back to the original image according to respective proportions. In the feature fusion stage, the fusion proportion of the infrared and visible light modes is dynamically adjusted through a learnable fusion weight coefficient. In the training stage, end-to-end optimization is carried out by using a large-scale annotation data set, so that the superposition proportion and the fusion weight alpha of each mode can be adaptively adjusted to adapt to different illumination conditions and scene changes, and the generalization ability of the model is remarkably improved. In a robot vision system for night pedestrian detection, an infrared and visible light modal fusion method based on machine learning weight optimization is adopted, and the accuracy and robustness of pedestrian detection are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning and target detection, and relates to a pedestrian detection method based on infrared and visible light dual-modal fusion, which is particularly suitable for improving the analysis of night or low-light scenes. The method adaptively fuses infrared features and visible light features through a learnable weight autonomous optimization mechanism to improve the accuracy and robustness of pedestrian detection. Background Art

[0002] With the rapid development of machine learning technology, deep learning models have made significant progress in image recognition and target detection tasks. Especially in the field of pedestrian detection, detection algorithms based on convolutional neural networks (CNN) have been widely used in various scenarios, including monitoring, autonomous driving, and robot navigation. However, traditional pedestrian detection methods mostly rely on single-modal visible light image features, which show high detection accuracy in daytime or well-lit environments. However, when the environment changes to nighttime or low-light conditions, visible light images are limited by insufficient light, which significantly reduces detection accuracy and reduces system robustness.

[0003] In order to solve the challenges of accuracy and robustness of pedestrian detection at night and in low light, infrared images are gradually introduced due to their unique characteristics that are different from visible light, and are used for pedestrian detection in combination with visible light images for dual-modal fusion. However, most of the current dual-modal fusion methods only directly superimpose infrared and visible light features when performing modal fusion. For example, a recent study introduced a cross-modal attention feature fusion module (Yang Y, Xu K, Wang K. Cascaded formation enhancement and cross-modal attention feature fusion for multispectral pedestrian detection [J]. Frontiers in Physics, 2023, 11: 1121311.). This type of method ignores the difference in information contained in the two modalities themselves and the difference in importance in forming decisions, resulting in most dual-modal fusion methods being unable to give full play to the complementarity of the two modalities in complex environments. In addition, if the weight distribution of each modality in feature fusion for decision-making can be adaptively adjusted according to the environmental conditions in the scene, a flexible modal fusion mechanism can be formed to improve the detection effect and robustness of the dual-modal fusion method.

[0004] Furthermore, most of the existing dual-modal fusion technologies use fixed fusion weights to simply weight the two modalities, which lacks flexibility and fails to dynamically adjust the modal weights according to the lighting or weather conditions of the actual environment. This fixed-weight fusion strategy often cannot adapt to changing lighting conditions, resulting in uneven detection performance in different scenarios. The latest research results introduce a method called Multispectral Pedestrian Detection Transformer (MS-DETR) (Zhao C, Sun Y, Wang W, et al. MS-DETR: Efficient DETR Training with Mixed Supervision [C]. 2024: 17027-17036.), which uses two parallel modality-specific CNN backbones to extract multi-scale feature maps and fuses these features through a multi-modal transformer decoder. There is a lack of targeted enhancement strategies in the processing of infrared and visible light images, which makes the infrared modality, which can play a unique role, not fully play its advantages in low-light scenes.

[0005] In view of the above problems, the innovation of the present invention is embodied in:

[0006] 1. Data enhancement method for infrared and visible light modalities: An innovative dual-modal data enhancement technology is proposed to superimpose infrared and visible light feature maps in a specific proportion, thereby improving the pedestrian feature representation capabilities of each modality in different scenarios and enhancing the model's sensitivity to pedestrian features. (This innovation is added in the front that most people have not considered this issue)

[0007] 2. Adaptive weight fusion mechanism: In the process of modal fusion, learnable weight coefficients are introduced. By adaptively adjusting the fusion ratio of infrared and visible light modalities, the model can dynamically optimize the modal weights according to scene requirements, thereby improving the feature fusion efficiency and adaptability of the model. Summary of the invention

[0008] The present invention aims at the key issues of the effectiveness of pedestrian feature characterization and modal information fusion in the dual-modal fusion of infrared and visible light, and proposes an innovative fusion method to improve the accuracy and robustness of pedestrian detection at night or in low light conditions. The first innovation of this method is to introduce a new data enhancement technology for each modality, superimpose the extracted feature map on the original image at a certain ratio, so as to enhance the characterization ability of pedestrian features. The second innovation is to introduce a learnable weight coefficient in the modal fusion process, which is used to dynamically adjust the fusion ratio of infrared and visible light modalities, so that the model can adaptively optimize the modal weights according to different scenarios, and achieve more efficient feature fusion.

[0009] Specifically, the present invention first extracts features from infrared images and visible light images separately through independent convolutional neural networks to obtain unique feature representations for each modality. Next, the extracted features are superimposed back to the original image at their respective proportions through learnable scale coefficients. The superposition ratio of each modality is adaptive and independently learned, ensuring that different modalities can fully retain their respective key information during the feature enhancement process and further improve the sensitivity to pedestrian features.

[0010] In the feature fusion stage, the fusion ratio of infrared and visible light modalities is dynamically adjusted through the learnable fusion weight coefficient α, making the weight configuration of different modalities more reasonable in diverse scenarios, thereby generating richer and more accurate fusion feature representation, further improving the accuracy of pedestrian detection.

[0011] In the training phase, the method uses a large-scale annotated data set for end-to-end optimization, so that the superposition ratio and fusion weight α of each modality can be adaptively adjusted to adapt to different lighting conditions and scene changes, thereby significantly improving the generalization ability of the model. In the testing phase, the present invention can give full play to the complementary characteristics of infrared and visible light modalities in complex nighttime or low-light environments to ensure the efficiency and accuracy of the pedestrian detection system.

[0012] The technical solution of the present invention:

[0013] A pedestrian detection method based on infrared and visible light fusion with autonomous weight optimization includes the following steps:

[0014] Step 1: Feature Extraction

[0015] The corresponding convolutional neural networks are used to extract features from infrared images and visible light images respectively. This process aims to make full use of the spectral characteristics of different modalities to obtain diversified feature representations and lay the foundation for subsequent fusion.

[0016] 1.1 Input preprocessing

[0017] The original infrared image and visible light image are preprocessed, including image normalization, cropping and enhancement, to obtain the processed infrared image I infrared and visible light image I visible ;

[0018] The processed visible light image and infrared image are input into two independent convolutional neural networks for feature extraction. Both convolutional neural networks use the YOLOv5 backbone network structure; the two convolutional neural networks are set as F visible and F infrared , which correspond to the feature extraction of visible light images and infrared images, respectively, are expressed as follows:

[0019] Fvisible =φ(I visible )

[0020] F infrared =φ(I infrared )

[0021] Among them, φ represents the forward propagation process of the convolutional neural network, including convolution operations, activation functions (such as ReLU), and pooling operations.

[0022] 1.2 Feature map generation

[0023] After feature extraction, the feature map output by the convolutional neural network can be expressed as F v and F i :

[0024] F v =F visible (I visible )=[f v1 ,f v2 ,…,f vn ]

[0025] F i =F infrared (I infrared )=[f i1 ,f i2 ,…,f in ]

[0026] Among them, f v1 ,f v2 ,…,f vn is the feature of the visible light image, f i1 ,f i2 ,…,f in is the infrared image feature, and n is the number of features of each modality.

[0027] Step 2: Data Augmentation

[0028] A new data enhancement method is introduced for each modality. The superposition ratio of feature maps is dynamically adjusted through a learnable gain coefficient. The feature maps of the extracted infrared image and visible light image are correspondingly weighted and superimposed on the original infrared image and visible light image to form enhanced infrared images and enhanced visible light images.

[0029] 2.1 Introduction of learnable gain coefficient

[0030] In the data enhancement process, a learnable gain coefficient is introduced for each modality (visible light and infrared). The gain coefficients of the visible light modality and the infrared modality are defined as g v and g iThese gain coefficient feature maps are dynamically generated to ensure that they can dynamically adjust the superposition ratio of feature maps according to the characteristics of different scenes; the formula is as follows:

[0031] g v =σ(f v (F v ))

[0032] g i =σ(f i (F i ))

[0033] Among them, σ represents the activation function (such as Sigmoid), which is used to limit the gain coefficient to a certain range; f v and f i is a mapping function used to generate gain coefficients, which is implemented by the fully connected layer; F v and F i Characteristic maps representing visible and infrared modes, respectively;

[0034] 2.2 Dynamic Superposition of Feature Maps

[0035] After feature extraction, the feature maps of the infrared image and the visible light image are weighted and superimposed with the original infrared image and the visible light image through the learnable gain coefficient; the enhanced infrared feature F i ′ and visible light feature F′ v The generation process is as follows:

[0036] I′ v =I v +g v ·F v

[0037] I′ i =I i +g i ·F i

[0038] Among them, I v and I i They are the original infrared image and visible light image respectively;

[0039] Step 3: Fusion weight calculation

[0040] The fusion weight coefficient w is introduced to optimize the fusion process of infrared features and visible light features. The fusion weight coefficient w is learned through a self-supervised learning method during the training phase to ensure that the feature fusion model automatically adapts to different scenarios, thereby generating a more representative feature representation.

[0041] The fusion weight coefficient is set to w, the purpose of which is to perform weighted fusion of infrared features and visible light features. The fusion weight can be defined as:

[0042] w=σ(f(F′ v ,F′ i ))

[0043] Among them, σ is an activation function (such as Sigmoid), which is used to limit the weight coefficient to a certain range (0, 1), f is a mapping function used to generate the weight coefficient, which is implemented by the fully connected layer; I′ i and I′ v represents enhanced infrared and visible light features;

[0044] Step 4: Feature Fusion

[0045] Feature fusion is to generate a comprehensive feature representation by weighted summing the gain-adjusted infrared features and visible light features and combining them with the fusion weight coefficients; this process ensures that the key information of each modality is fully retained during the fusion process, thereby improving the accuracy and robustness of pedestrian detection.

[0046] In order to ensure that the key information of each modality can be fully retained during the fusion process, a feature reweighting mechanism is adopted; by calculating the attention weight of each feature, the process is expressed as:

[0047] F fused =Attention(F′ v )·(w·F′ v )+Attention(F′ i )·((1-w)·F′ i )

[0048] Among them, Attention (F) is a multi-head attention module, which takes the features of two modalities as input and outputs the gain of the corresponding modality;

[0049] Step 5: Model training

[0050] End-to-end joint training is performed using a large-scale annotated dataset to adaptively learn the gain coefficient and fusion weight coefficient. By providing feedback on different lighting conditions and scene changes, the feature extraction and fusion strategies are optimized, significantly improving the generalization ability of the model.

[0051] 5.1 Data Preparation

[0052] First, infrared and visible light images under various lighting conditions and scenes are obtained from a large-scale annotated dataset to construct a training sample set D = {(I i ,I v ,y)}, where I i and I v represent infrared and visible light images respectively, and y is the corresponding label.

[0053] 5.2 Objective Function

[0054] In order to adaptively learn the gain coefficient and the fusion weight coefficient, a joint loss function L is defined, which consists of multiple parts, including prediction loss and regularization loss; it is expressed as:

[0055] L=L pred +λ1L gain +λ2L fusion

[0056] Among them, L pred is the prediction loss, which measures the gap between the output of the pedestrian detection model and the true label; L gain is the regularization loss of the gain coefficient to prevent overfitting; L fusion To regularize the loss of the fusion weight and ensure that it is within a reasonable range, λ1 and λ2 are used as hyperparameters, which are the gain loss weight and fusion loss weight, respectively, and the default values ​​are 0.1 and 0.2;

[0057] 5.3 End-to-end training

[0058] The parameters of the joint loss function L are optimized through the back-propagation algorithm, including the learnable gain coefficient g i and g v And the fusion weight coefficient w, the update process is expressed as:

[0059]

[0060] Among them, θ is the parameter set of the pedestrian detection model; η is the learning rate; is the gradient of the joint loss function L with respect to the parameters of the pedestrian detection model.

[0061] Step 6: Test and Verify

[0062] Apply trained models in complex nighttime or low-light environments to perform pedestrian detection tasks using the complementary strengths of infrared and visible light. Evaluate system performance in real time to ensure efficiency and accuracy in a variety of environments.

[0063] 6.1 Test data preparation

[0064] Select a challenging test dataset that includes infrared and visible light images under various lighting conditions. These images should cover a variety of complex scenes to fully evaluate the performance of the model. Test sample set T = {(I i ,I v )} include infrared images and visible light samples.

[0065] 6.2 Model Inference

[0066] Use the trained model to detect pedestrians. The model processes each pair of infrared and visible light images to obtain the pedestrian detection result P. The process can be expressed as:

[0067] P = f(I i ,I v ;θ)

[0068] Among them, f is the inference function of the model and θ is the parameter of the model.

[0069] 6.3 Performance Evaluation

[0070] In order to evaluate the performance of the model in different scenarios, standard evaluation indicators are introduced, including accuracy, recall, F1-score and miss rate. These indicators are calculated as follows:

[0071]

[0072]

[0073] in:

[0074] TP (True Positive): The number of correctly detected pedestrians

[0075] TN (True Negative): The number of correctly undetected background

[0076] FP (False Positive): The number of background images that are incorrectly detected as pedestrians.

[0077] FN (False Negative): The number of pedestrians not detected

[0078] 6.4. Complementary Advantages Assessment

[0079] By comparing the detection results of pedestrians using infrared and visible light images alone and after dual-modal fusion, the effectiveness of complementary advantages is evaluated. The comparative experimental results show that:

[0080]

[0081] Among them, Performance fused is the detection performance after fusion. single It is the detection performance of a single mode.

[0082] Beneficial effects of the invention: In the robot vision system for nighttime pedestrian detection, the invention adopts an infrared and visible light modality fusion method based on machine learning weight optimization, which effectively improves the accuracy and robustness of pedestrian detection. Through adaptive learning gain coefficients and fusion weight coefficients, the invention ensures that the model can dynamically adjust the feature fusion strategy under different lighting conditions and scene changes, thereby achieving efficient pedestrian detection in complex environments.

[0083] First, the present invention uses convolutional neural networks (CNNs) to extract features from infrared and visible light images, ensuring that diverse feature representations related to pedestrian detection are extracted. This process improves the quality of feature extraction and sample efficiency by performing necessary preprocessing and data enhancement on the input images, laying a solid foundation for subsequent feature fusion.

[0084] Secondly, by introducing learnable gain coefficients and fusion weight coefficients, the present invention achieves weighted summation of infrared and visible light features in the feature fusion stage. This process ensures that pedestrian features are effectively retained during the fusion process by focusing on key information of different modalities, thereby significantly improving detection performance. During the training process, the model uses a self-supervised learning strategy to ensure the rationalization of fusion weights, thereby improving the ability to extract features related to shared tasks.

[0085] Finally, after end-to-end training on a large-scale annotated dataset, the present invention realizes an efficient pedestrian detection system that performs well at night or in low-light environments. Through comparative experiments with traditional single-modal detection methods, the complementary advantages of infrared and visible light modal fusion are verified, and the generalization ability and adaptability of the model are improved, so that it can still maintain high accuracy and low miss rate in complex dynamic scenes. These innovative designs and methods provide a new solution for nighttime pedestrian detection and have important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is the overall flow chart of a nighttime pedestrian detection method that combines infrared and visible light modalities with machine learning weight optimization. DETAILED DESCRIPTION

[0087] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0088] like Figure 1 As shown, the present invention provides a method for nighttime pedestrian detection based on infrared and visible light modality fusion and machine learning weight optimization, and the specific implementation process is as follows:

[0089] 1. Feature map generation

[0090] In the present invention, infrared and visible light images are subjected to feature extraction through a specially designed convolutional neural network (CNN) to fully utilize the spectral characteristics of the two modalities and obtain diversified feature representations, laying the foundation for feature fusion. First, the visible light image I visible and infrared image I infrared Preprocessing, including normalization, cropping, and enhancement operations, is performed to improve the quality and consistency of the input data. The normalized image can be represented as Where I′ is the normalized image, μ and σ are the mean and standard deviation of the original image respectively. The preprocessed infrared and visible light images are input into the convolutional neural network F visible and F infrared Through the forward propagation operation, including the convolution, activation and pooling layers, the feature map F is generated. v =F visible (I′ visible ) and F i =F infrared (I′ infrared ).

[0091] 2. Data Augmentation

[0092] The present invention introduces an innovative data enhancement method in step 2 to further improve the sensitivity to pedestrian features. Specifically, the feature fusion effect of infrared and visible light modalities is optimized by dynamically adjusting the superposition ratio of feature maps. First, a learnable gain coefficient g is introduced for infrared and visible light modalities respectively. v and g i , these gain coefficients can be learned through optimization algorithms. Based on this, according to the characteristics of different scenes, the superposition ratio of feature maps is dynamically adjusted to ensure the best fusion effect. Subsequently, the feature map extracted in step 1 is dynamically superimposed with the original image through the gain coefficient to generate the enhanced visible light feature F′ v =I v +g v ·F v and infrared signature F i ′=I i +g i ·F i .

[0093] 3. Fusion weight calculation

[0094] In the present invention, based on the infrared and visible light features extracted in step 1, the weighted fusion of the two modal features is achieved by introducing a fusion weight coefficient w, so as to further optimize the feature representation and enhance the adaptability of the model to different scenarios. The weight coefficient w is learned by a self-supervised learning method and is defined as w = σ(f(F′ v ,F′ i)). Under the optimization of self-supervised learning, the fusion weight w can effectively adapt to different scene characteristics, thereby improving the detection effect of the model.

[0095] 4. Feature Fusion

[0096] In the fourth step of the present invention, the visible light and infrared feature maps after gain adjustment in step 2 are used, and combined with the weight fusion coefficient learned in step 3, the two modalities are weighted summed to generate a comprehensive feature and expressed as F fused To further optimize feature fusion, a feature reweighting mechanism is adopted. The attention weight of each modal feature is calculated through the attention mechanism Attention (F), thereby achieving weighted adjustment of the input features. The final fusion feature is represented by F fused =Attention(F′ v )·(w·F′ v )+Attention(F′ i )·((1-w)·F′ i ).

[0097] 5. Model training

[0098] In the model training phase of the present invention, the gain coefficient g is optimized by the end-to-end joint training method in step 2 v and g i And the fusion weight w obtained in step 3, so as to adapt to different lighting conditions and scene changes to improve the robustness of the model. In the data preparation process, we first constructed a training sample set D = {(I i ,I v ,y)}. The objective function L consists of prediction loss, regularization loss and fusion loss to ensure the correct learning of the gain coefficient g in step 2 v and g i In the process of maintaining a reasonable range, while optimizing the weight fusion coefficient w in step 3. Finally, all model parameter sets θ are updated through back propagation, and gradient descent updates are performed with a learning rate η. Achieve global optimization of feature extraction and fusion strategies.

[0099] 6. Testing and Verification

[0100] In the last step of the present invention, the trained model is tested and verified in complex night and low-light environments to comprehensively evaluate the pedestrian detection performance after the fusion of infrared and visible light modalities. In the test data preparation stage, a test dataset T = {(I i ,I v)}, ensuring that all kinds of scene features are covered. In the model inference process, the model parameters θ optimized in the first five steps are used to infer the function P = f(I i ,I v ; θ) Detect each pair of visible light and infrared images, generate pedestrian detection results, and calculate relevant indicators. At the same time, by comparing the difference between the fused performance and the single-modal performance, calculate the detection performance improvement rate, and thus confirm the improvement effect of the machine learning weight fusion strategy on the accuracy and robustness of pedestrian detection.

Claims

1. A pedestrian detection method based on infrared and visible light fusion with autonomous weight optimization, characterized in that: The following steps are involved: Step 1: Feature Extraction The infrared image and visible light image are respectively subjected to feature extraction through corresponding convolutional neural networks; 1.1 Input preprocessing The original infrared image and visible light image are preprocessed, including image normalization, cropping and enhancement, to obtain the processed infrared image I infrared and visible light image I visible ; The processed visible light image and infrared image are respectively input into two independent convolutional neural networks for feature extraction; both convolutional neural networks adopt the backbone network structure of YOLOv5; the two convolutional neural networks are set as F visible and F infrared , which correspond to the feature extraction of visible light images and infrared images, respectively, are expressed as follows: F visible =φ(I visible ) F infrared =φ(I infrared ) Among them, φ represents the forward propagation process of the convolutional neural network, including convolution operation, activation function and pooling operation; 1.2 Feature map generation After feature extraction, the feature map output by the convolutional neural network is represented as F v and F i : F v =F visible (I visible )=[f v1 ,f v2 ,…,f vn ] F i =F infrared (I infrared )=[f i1 ,f i2 ,…,f in ] Among them, f v1 ,f v2 ,…,f vn is the feature of the visible light image, f i1 ,f i2 ,…,f in is the infrared image feature, n is the number of features of each modality; Step 2: Data Augmentation A new data enhancement method is introduced for each modality. The superposition ratio of feature maps is dynamically adjusted through a learnable gain coefficient. The feature maps of the extracted infrared image and visible light image are correspondingly weighted and superimposed on the original infrared image and visible light image to form an enhanced infrared image and an enhanced visible light image. 2.

1. Introducing a learnable gain coefficient The gain coefficients of the visible light mode and infrared mode are defined as g v and g i ; These gain coefficients are dynamically generated by the feature map to ensure that they can dynamically adjust the superposition ratio of the feature map according to the characteristics of different scenes; the formula is as follows: g v =σ(f v (F v )) g i =σ(f i (F i ) Among them, σ represents the activation function, which is used to limit the gain coefficient to a certain range, f v and f i is a mapping function used to generate gain coefficients, which is implemented by the fully connected layer; F v and F i Characteristic maps representing visible and infrared modes, respectively; 2.2 Dynamic Superposition of Feature Maps The feature maps of the infrared image and the visible light image are compared with the original infrared image through a learnable gain coefficient. Weighted superposition of visible light image; enhanced infrared feature F i ′ and visible light feature F v The generation process of ′ is as follows: F v ′=I v +g v ·F v F i ′=I i +g i ·F i Among them, I v and I i They are the original infrared image and visible light image respectively; Step 3: Fusion weight calculation The fusion weight coefficient w is introduced to optimize the fusion process of infrared features and visible light features. The fusion weight coefficient w is learned through a self-supervised learning method during the training phase to ensure that the feature fusion model automatically adapts to different scenarios, thereby generating a more representative feature representation. Definition of fusion weights Set the fusion weight coefficient w as shown in the following formula: w=σ(f(F′ v ,F′ i ) Among them, σ represents the activation function, which is used to limit the weight coefficient within a certain range, and f is the mapping function used to generate the weight coefficient, which is implemented by the fully connected layer; I i ′ and I v ′ represents the enhanced infrared and visible light features; Step 4: Feature Fusion Feature fusion is to generate a comprehensive feature representation by weighted summing the gain-adjusted infrared features and visible light features and combining the fusion weight coefficients; Weighted Sum In order to ensure that the key information of each modality can be fully retained during the fusion process, a feature reweighting mechanism is adopted; by calculating the attention weight of each feature, the process is expressed as: F fused =Attention(F v ′ )·(w·F v ′ )+Attention(F i ′ )·((1-w)·F i ′ ) Among them, Attention (F) is a multi-head attention module, which takes the features of two modalities as input and outputs the gain of the corresponding modality; Step 5: Pedestrian detection model training End-to-end joint training using publicly available large-scale annotated datasets; 5.1 Data Preparation First, we construct a training sample set D = {(I i ,I v ,y)}, where I i and I v are the original infrared image and visible light image respectively, and y is the corresponding target detection label; 5.2 Objective Function In order to adaptively learn the gain coefficient and the fusion weight coefficient, a joint loss function L is defined, which consists of multiple parts, including prediction loss and regularization loss; it is expressed as: L=L pred +λ1L gain +λ2L fusion Among them, L pred is the prediction loss, which measures the gap between the output of the pedestrian detection model and the true label; L gain is the regularization loss of the gain coefficient to prevent overfitting; L fusion To regularize the loss of the fusion weight and ensure that it is within a reasonable range, λ1 and λ2 are used as hyperparameters, which are the gain loss weight and fusion loss weight, respectively, and the default values ​​are 0.1 and 0.2; 5.3 End-to-end training The parameters of the joint loss function L are optimized through the back-propagation algorithm, including the learnable gain coefficient g i and g v And the fusion weight coefficient w, the update process is expressed as: Among them, θ is the parameter set of the pedestrian detection model; η is the learning rate; is the gradient of the joint loss function L with respect to the parameters of the pedestrian detection model.

Images