Image non-motor vehicle detection and identification method based on hierarchical loss function

Abstract

The invention relates to an image non-motor vehicle detection and identification method based on a hierarchical loss function, and the method comprises the steps: obtaining an input non-motor vehicle image, carrying out the feature extraction of the non-motor vehicle image through a multi-scale feature extraction network, and obtaining a multi-scale feature map, detecting a non-uniform illumination region in the multi-scale feature map and performing illumination standardization processing to obtain an illumination standardization feature map, classifying the low-dimensional feature vectors through a preset classification model to obtain a classification result of the non-motor vehicle image, obtaining an environment complexity level, and obtaining a non-motor vehicle image classification result. And dynamically adjusting a weight coefficient of the hierarchical loss function according to the environmental complexity level, training a target detection branch and an environmental adaptation branch at the same time through a multi-task learning framework, generating an environmental adaptive feature representation, obtaining a final fusion feature, calculating a comprehensive loss value according to the final fusion feature, and obtaining a non-motor vehicle identification result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image non-motorized vehicle detection and recognition method based on a hierarchical loss function. Background Technology

[0002] Current mainstream non-motorized vehicle recognition methods generally use a single loss function for model training. This approach shows a significant lack of adaptability when dealing with complex traffic scenarios. Traditional detection algorithms often rely on fixed training datasets, which are difficult to cover various changes in the real environment, resulting in a significant drop in recognition accuracy when the model faces new scenarios.

[0003] Factors such as lighting changes, weather conditions, and occlusion in real-world traffic environments significantly affect the visual characteristics of non-motorized vehicles. Existing training data often fails to adequately cover these complex combinations of environmental changes. This gap between training data and actual application scenarios further exacerbates the inconsistency in model performance under different environmental conditions. When a model encounters scene combinations not seen during training, its recognition ability exhibits a significant degradation. This lack of generalization ability directly restricts the reliability of the detection system in dynamically changing environments. Traditional loss function designs cannot effectively guide the model to learn sufficiently robust feature representations, making it difficult for the model to maintain stable recognition performance when facing environmental changes.

[0004] A hierarchical loss function that can dynamically adapt to various environmental changes, enabling it to effectively integrate scene information of different complexities during training and guide the model to learn feature representations that are highly adaptable to environmental changes, thereby improving the generalization ability of non-motorized vehicle recognition systems in unknown scenarios, has become a key problem that urgently needs to be solved in the current field of image detection. Summary of the Invention

[0005] To address the problems existing in the prior art, this application aims to provide an image non-motorized vehicle detection and recognition method based on a hierarchical loss function.

[0006] The image non-motorized vehicle detection and recognition method based on hierarchical loss function described in this application includes:

[0007] Step S101: The input non-motorized vehicle image is preprocessed using a multi-scale feature extraction network. Image features at different resolutions are extracted by a convolutional neural network. If an uneven illumination region is detected in the image, histogram equalization is performed on the region to obtain a feature map after illumination standardization.

[0008] Step S102: Establish an environment change detection module based on the acquired feature map. By analyzing the brightness distribution, contrast change and edge sharpness parameters of the image, determine the environmental complexity level of the current scene. If the brightness variance is greater than the preset threshold T1 and the contrast is less than the threshold T2, it is determined to be a complex lighting environment.

[0009] Step S103: An adaptive weight allocation algorithm is used to dynamically adjust the weight coefficients of each sub-loss term in the hierarchical loss function according to the environmental complexity level, using formula W. i =α i ×e -βi×C Calculate the weight of the i-th loss value Li, where α i β represents the basic weighting coefficient. i Let represent the decay factor of the i-th loss function, and C represent the environmental complexity value;

[0010] Step S104: Train the target detection branch and the environment adaptation branch simultaneously through a multi-task learning framework. The target detection branch is responsible for identifying the location and category of non-motorized vehicles, while the environment adaptation branch generates a feature enhancement vector for the current environment based on dynamic weight coefficients, thus obtaining an environment-adaptive feature representation.

[0011] Step S105: After obtaining the feature enhancement vector, establish a feature fusion mechanism. If the confidence of the output of the environment adaptation branch is greater than the threshold T3, the enhancement vector and the original feature are weighted and fused. The fusion formula is F_final=λ×F_original+(1-λ)×F_enhanced, where λ represents the fusion weight parameter.

[0012] Step S106: Calculate the comprehensive loss value based on the fused feature representation. The comprehensive loss function L_total consists of three parts: classification loss L_cls, localization loss L_loc, and environment adaptation loss L_env. Update the network parameters according to the backpropagation algorithm based on the hierarchical loss value to obtain the model weights that adapt to multiple environment changes.

[0013] Step S107: The trained model is continuously optimized using an online learning strategy. When a new environmental change pattern is detected, the model parameters are updated using an incremental learning method. If the recognition accuracy of the new scene is lower than the preset threshold T4, a local update mechanism for the model parameters is triggered.

[0014] Step S108: Multiple sub-models trained for different environmental conditions are fused using model ensemble technology. The most suitable combination of sub-models is automatically selected based on the environmental characteristics of the current scene to obtain the final non-motorized vehicle recognition result and confidence score.

[0015] Preferably, in step S101, the step of extracting features from the non-motorized vehicle image using a multi-scale feature extraction network to obtain a multi-scale feature map includes:

[0016] The input non-motorized vehicle image is processed by multi-scale convolution through a convolutional neural network to obtain feature maps of different resolutions and generate a multi-scale feature map spectrum.

[0017] If uneven illumination regions are detected in the multi-scale feature map, histogram equalization is used to process the uneven illumination regions to obtain an illumination-normalized feature map.

[0018] Preferably, in step S102, obtaining the environmental complexity level includes:

[0019] The feature map is obtained from the input image, and the brightness distribution and brightness variance are calculated to obtain the first analysis result.

[0020] If the brightness variance in the first analysis result is greater than the preset threshold T1, then the feature map is processed by histogram equalization to obtain the second analysis result;

[0021] Based on the second analysis result, extract the contrast change parameters, calculate the global contrast value, and obtain the contrast distribution characteristics;

[0022] If the proportion of regions with a contrast distribution feature smaller than a preset threshold T2 is greater than a preset ratio, then the edge sharpness is extracted by the Laplacian operator to obtain the edge feature set.

[0023] The edge point density and edge intensity are calculated using the edge feature set to obtain the environment complexity parameter. The decision tree algorithm is then used to determine whether the current scene is a complex lighting environment based on the environment complexity parameter, and an environment classification result is generated.

[0024] Preferably, in step S103, the step of dynamically adjusting the weight coefficients of the hierarchical loss function according to the environmental complexity level includes:

[0025] The environmental complexity value C is obtained by analyzing the environmental characteristics through sensor data or a preset model.

[0026] Based on the environmental complexity value C, the weight coefficients of each sub-loss term are calculated using the exponential decay formula W_i=α_i×exp(-β×C) to obtain the weight value W_i, where α_i is the preset basic weight coefficient and β is the preset decay factor. The hierarchical loss function is updated by the weighted summation method to obtain the adjusted loss function value.

[0027] If the adjusted loss function value exceeds a preset threshold, the model parameters are optimized using the gradient descent algorithm to obtain an optimized set of model parameters.

[0028] The optimized model parameter set is used to update the adaptive weight allocation algorithm to obtain a new set of weight coefficients;

[0029] The hierarchical loss function is recalculated based on the new set of weight coefficients to obtain the updated loss function value.

[0030] Preferably, in step S104, the step of simultaneously training the target detection branch and the environment adaptation branch through a multi-task learning framework to generate an environment-adaptive feature representation includes:

[0031] The initial features of the input image are extracted using a convolutional neural network to obtain the first feature set;

[0032] A fast region convolutional neural network is used to detect the non-motorized vehicle location and category of the first feature set to obtain first location information and first category information.

[0033] Dynamic weight coefficients are calculated from the first feature set based on an attention mechanism to generate a first feature enhancement vector;

[0034] The first feature enhancement vector is fused with the first feature set through matrix multiplication to generate an environment-adaptive second feature set.

[0035] If the feature distribution of the second feature set does not match the preset environmental feature threshold, the dynamic weight coefficients are adjusted through gradient backpropagation to obtain an optimized second feature enhancement vector.

[0036] A third feature set is generated by feature concatenation. The target detection branch parameters are updated using the third feature set, and non-motorized vehicles are detected again to obtain the final location information and category information.

[0037] Preferably, in step S105, the use of a feature fusion mechanism to fuse the original features and enhanced features to obtain the final fused features includes:

[0038] The input data is processed by a convolutional neural network to extract features, resulting in the original feature vector F_original.

[0039] An enhanced feature vector F_enhanced is generated using a generative adversarial network through an environment adaptation branch;

[0040] The confidence output is obtained from the environmental adaptation branch. If the confidence output is greater than the preset threshold T3, the weighted fusion formula F_final=λ×F_original+(1-λ)×F_enhanced is used to perform feature fusion to obtain the final fused feature F_final, where λ is the fusion weight parameter.

[0041] The final fused feature F_final is classified using a support vector machine to obtain the classification result;

[0042] The gradient descent method is used to optimize the fusion weight parameter λ, resulting in the updated fusion weight parameter.

[0043] Based on the updated fusion weight parameters, feature fusion is re-executed to obtain the optimized final fused features.

[0044] Preferably, in step S106, calculating the comprehensive loss value based on the final fusion features and updating the model parameters includes:

[0045] The feature vector is obtained by extracting the fused feature representation through a convolutional neural network;

[0046] The classification loss L_cls is calculated using the cross-entropy loss function to obtain the classification error. The localization loss L_loc is then calculated using the bounding box regression algorithm to obtain the localization error.

[0047] If the positioning error exceeds a preset threshold, environmental features are extracted through the environmental perception module, and the environmental adaptation loss L_env is calculated to obtain the environmental error.

[0048] The comprehensive loss value L_total is calculated using a weighted summation method based on the classification error, positioning error, and environmental error.

[0049] The network parameters are updated using the backpropagation algorithm based on the comprehensive loss value L_total to obtain the optimized parameters;

[0050] The optimized parameters are used to update the model weights, resulting in model weights that adapt to changes in multiple environments.

[0051] Preferably, in step S108, obtaining the non-motorized vehicle identification result includes:

[0052] New environmental data is acquired from the data stream through an online learning strategy, environmental change patterns are detected, a set of change patterns is obtained, and the parameters of the trained model are adjusted using an incremental learning method to obtain updated model parameters.

[0053] Perform a recognition task on new scene data, calculate the recognition accuracy, and if the recognition accuracy is lower than a preset threshold T4, trigger a local update mechanism to adjust the parameters of a specific layer of the model to obtain an optimized model.

[0054] The optimized model is used to re-identify the new scene data to obtain a new recognition accuracy.

[0055] The model ensemble technique is used to perform weighted fusion of sub-models trained for different environmental conditions. Candidate sub-model groups are determined based on the environmental feature set of the current scene. The weight of each sub-model is calculated using an ensemble learning algorithm to obtain the fused model. The final non-motorized vehicle recognition result and confidence score are then output.

[0056] The method for detecting and recognizing non-motorized vehicles based on a hierarchical loss function described in this application has the following advantages: It preprocesses the input image using a multi-scale feature extraction network, extracts features at different resolutions using a convolutional neural network, performs histogram equalization on unevenly lit areas to obtain a standardized feature map, establishes an environment change detection module, analyzes parameters such as image brightness distribution to determine scene complexity, dynamically adjusts the weights of the loss function using an adaptive weight allocation algorithm, simultaneously trains the target detection and environment adaptation branches through a multi-task learning framework, establishes a feature fusion mechanism, fuses the enhanced vectors generated by the environment adaptation branch with the original features, continuously optimizes the model using an online learning strategy, and integrates multiple sub-models through model ensemble technology. This invention can effectively improve the recognition accuracy of non-motorized vehicles in complex lighting and other variable environments, and has strong environmental adaptability. Attached Figure Description

[0057] Figure 1 This application describes a method for detecting and recognizing non-motorized vehicles based on a hierarchical loss function. Figure 1 ;

[0058] Figure 2 This application describes a method for detecting and recognizing non-motorized vehicles based on a hierarchical loss function. Figure 2 . Detailed Implementation

[0059] like Figures 1-2 As shown, the image non-motorized vehicle detection and recognition method based on hierarchical loss function described in this application includes:

[0060] Step S101: The input non-motorized vehicle image is preprocessed using a multi-scale feature extraction network. Image features at different resolutions are extracted by a convolutional neural network. If an uneven illumination region is detected in the image, histogram equalization is performed on the region to obtain a feature map after illumination standardization.

[0061] Step S102: Establish an environment change detection module based on the acquired feature map. By analyzing the brightness distribution, contrast change and edge sharpness parameters of the image, determine the environmental complexity level of the current scene. If the brightness variance is greater than the preset threshold T1 and the contrast is less than the threshold T2, it is determined to be a complex lighting environment.

[0062] Step S103: An adaptive weight allocation algorithm is used to dynamically adjust the weight coefficients of each sub-loss term in the hierarchical loss function according to the environmental complexity level, using formula W. i =α i ×e -βi×C Calculate the weight of the i-th loss value Li, where α i β represents the basic weighting coefficient. i Let represent the decay factor of the i-th loss function, and C represent the environmental complexity value;

[0063] Step S104: Train the target detection branch and the environment adaptation branch simultaneously through a multi-task learning framework. The target detection branch is responsible for identifying the location and category of non-motorized vehicles, while the environment adaptation branch generates a feature enhancement vector for the current environment based on dynamic weight coefficients, thus obtaining an environment-adaptive feature representation.

[0064] Step S105: After obtaining the feature enhancement vector, establish a feature fusion mechanism. If the confidence of the output of the environment adaptation branch is greater than the threshold T3, the enhancement vector and the original feature are weighted and fused. The fusion formula is F_final=λ×F_original+(1-λ)×F_enhanced, where λ represents the fusion weight parameter.

[0065] Step S106: Calculate the comprehensive loss value based on the fused feature representation. The comprehensive loss function L_total consists of three parts: classification loss L_cls, localization loss L_loc, and environmental adaptation loss L_env. Update the network parameters through the backpropagation algorithm to obtain the model weights that adapt to multiple environmental changes.

[0066] Step S107: The trained model is continuously optimized using an online learning strategy. When a new environmental change pattern is detected, the model parameters are updated using an incremental learning method. If the recognition accuracy of the new scene is lower than the preset threshold T4, a local update mechanism for the model parameters is triggered.

[0067] Step S108: Multiple sub-models trained for different environmental conditions are fused using model ensemble technology. The most suitable combination of sub-models is automatically selected based on the environmental characteristics of the current scene to obtain the final non-motorized vehicle recognition result and confidence score.

[0068] like Figures 1-2 As shown, in step S101, a multi-scale feature extraction network is used to preprocess the input non-motorized vehicle image. Image features at different resolutions are extracted by a convolutional neural network. If an uneven illumination region is detected in the image, histogram equalization is performed on the region to obtain a feature map after illumination standardization.

[0069] Furthermore, in step S101, the input non-motorized vehicle image is preprocessed by a multi-scale feature extraction network, and image features at different resolutions are extracted by a convolutional neural network to obtain a multi-scale feature map.

[0070] If an uneven illumination region is detected in the multi-scale feature map, histogram equalization is performed on the region to obtain the illumination-normalized feature map.

[0071] Based on the feature map after illumination standardization, deep feature extraction is performed through a convolutional neural network to obtain a high-dimensional feature vector and thus a deep feature representation.

[0072] If there is noise interference in the deep feature representation, Gaussian filtering is used to smooth the high-dimensional feature vector to obtain a denoised feature vector.

[0073] Based on the denoised feature vectors, the principal component analysis algorithm is used to reduce the feature dimension to obtain low-dimensional feature vectors and determine the optimized feature representation.

[0074] The optimized feature representation is processed by a pre-defined classification model to obtain the classification results of non-motorized vehicle images, thus obtaining preliminary recognition results.

[0075] If the confidence score of the preliminary identification result is lower than the preset threshold, the classification model weights are adjusted through the environmental feature set, and the classification process is repeated to obtain the final identification result.

[0076] Based on the final recognition results, non-motorized vehicle category labels and corresponding confidence scores are generated, and structured recognition data is output.

[0077] Specifically, in step S101, the image multi-scale feature extraction branch uses a multi-scale feature extraction network to preprocess the input non-motorized vehicle image, and uses a ResNet architecture containing 3 convolutional layers to extract image features at different resolutions. The first convolutional kernel size is 7×7 with a stride of 2, and the second and third layers use 3×3 convolutional kernels with a stride of 1, outputting a multi-scale feature map.

[0078] If the standard deviation of pixel values ​​in a certain region of the multi-scale feature map exceeds 50, it is determined to be an area of ​​uneven illumination. The CLAHE algorithm is used to perform histogram equalization on this region, and the contrast parameter is limited to 2 to obtain the multi-scale feature map after illumination standardization.

[0079] Deep feature extraction is performed on the feature map at each scale using a pre-trained VGG16 network to obtain the deep feature representation at each scale.

[0080] In the depth feature representation at each scale, if the noise energy accounts for more than 15% of the depth feature representation, Gaussian filtering is used to smooth the high-dimensional feature vector. The filter size is set to 5×5 and the standard deviation is 1 to obtain the denoised feature vector.

[0081] Based on the denoised feature vector, principal component analysis algorithm is used for dimensionality reduction, retaining 95% of the variance contribution rate, reducing the feature dimension to 256 dimensions, and determining the optimized feature representation;

[0082] For the optimized feature representation obtained at the i-th scale, the Softmax function is used to generate the predicted score of the non-motorized vehicle category label, and the cross-entropy loss function is used to calculate the non-motorized vehicle classification loss value Li at the i-th scale.

[0083] like Figures 1-2 As shown, in step S102, in the environment detection branch, an environment change detection module is established based on the acquired feature map. By analyzing the brightness distribution, contrast change and edge sharpness parameters of the image, the environmental complexity level of the current scene is determined. If the brightness variance is greater than the preset threshold T1 and the contrast is less than the threshold T2, it is determined to be a complex lighting environment.

[0084] Further, in step S102, the brightness distribution and brightness variance are calculated using the acquired feature map to obtain the first analysis result;

[0085] If the brightness variance in the first analysis result is greater than the preset threshold T1, then the feature map is processed by histogram equalization to obtain the second analysis result.

[0086] Based on the second analysis results, the contrast variation parameters are extracted, the global contrast value is calculated, and the contrast distribution characteristics are obtained.

[0087] If the proportion of regions with contrast distribution features smaller than the preset threshold T2 is greater than the preset ratio, then the edge sharpness is extracted by the Laplacian operator to obtain the edge feature set;

[0088] By using an edge feature set, edge point density and edge intensity are calculated to obtain environmental complexity parameters;

[0089] By using the environmental complexity parameter and applying the decision tree algorithm, we can determine whether the current scene is a complex lighting environment and obtain the environmental classification result.

[0090] Based on the environmental classification results, a detection marker for complex lighting environments is generated, and the final judgment result is output.

[0091] Specifically, in step S102, during each iteration, the brightness distribution is calculated using the grayscale histogram statistical method based on the acquired feature map, and the variance is calculated based on the pixel brightness value. If the variance exceeds the preset threshold T1 (T1 = 50), the image is subjected to histogram equalization processing to enhance the uniformity of brightness distribution.

[0092] On the equalized image, the global contrast calculation formula (Michelson contrast) is used to extract the contrast change parameters. If the proportion of regions with contrast values ​​lower than the threshold T2 (T2 = 0.3) exceeds 50%, the Laplacian operator (3×3 convolution kernel) is used to extract edge information and generate an edge feature set.

[0093] Based on the edge feature set, the number of edge points per unit area is counted (edge ​​point density per 100×100 pixel area > 50) and the average gradient magnitude is calculated (mean intensity of Sobel operator calculation > 0.5), and the environmental complexity parameter is obtained by combining the results.

[0094] Input this parameter into the pre-trained C4.5 decision tree model, and determine the scene classification based on node splitting rules (such as information gain > 0.1), and output the "complex lighting environment" label;

[0095] like Figures 1-2 As shown, in step S103, in the multi-scale loss function weight calculation, an adaptive weight allocation algorithm is used to dynamically adjust the weight coefficients of the loss value Li at each scale in the hierarchical loss function according to the environmental complexity level, using formula W. i =α i ×e -βi×C Calculate the weight of the i-th loss value Li, where α i β represents the basic weighting coefficient. i Let represent the decay factor of the i-th loss function, and C represent the environmental complexity value.

[0096] Further, in step S103, the environmental characteristics are analyzed through sensor data or a preset model, and the environmental complexity value C is calculated to obtain the environmental complexity value.

[0097] Based on the environmental complexity value C, the exponential decay formula W is used. i =α i ×e -βi×C Calculate the weight coefficient of the i-th loss term and determine the weight value W_i of the i-th loss term;

[0098] In the iterative training of the algorithm, the parameter E (E=200) is set as the number of iterations for model training. In each iteration, the image features at various scales and the classification loss value L are calculated. i A weighted summation method is used, combined with the calculated weight value W.i Calculate the total loss function L of the algorithm;

[0099] Based on the adjusted loss function value, the model parameters are optimized using the gradient descent algorithm to obtain the optimized model parameter set;

[0100] Based on the optimized model parameter set, update the weight coefficients of the adaptive weight allocation algorithm to obtain a new set of weight coefficients, and the current iteration calculation ends.

[0101] After the iteration calculation of parameter E is completed, the model's prediction output is adjusted using the newly obtained set of weight coefficients to obtain the final model prediction result, which includes structured identification data of category ID and confidence value.

[0102] Obtain the feature enhancement vector, combine it with the final model prediction results, establish a feature fusion mechanism, and obtain the fused feature vector;

[0103] If the confidence level of the environmental adaptation branch output is greater than the threshold T3, then the weighted fusion features are calculated using the fusion formula F_final=λ×F_original+(1-λ)×F_enhanced to obtain the final fusion feature vector.

[0104] Specifically, in step S103, environmental data is collected by LiDAR and camera, features are extracted by convolutional neural network, and the environmental complexity value C is calculated. For example, in the autonomous driving scenario, C = 0.8 indicates high complexity.

[0105] Based on C = 0.8, set the basic weight α_i = [0.3, 0.5, 0.2] and the attenuation factor β = 0.5. Substitute these values ​​into the formula W_i = α_i × exp(-0.5 × 0.8) to obtain the weight value W_i = [0.20, 0.34, 0.13].

[0106] Using a weighted summation method, W_i is multiplied by the classification loss, regression loss, and regularization loss, and then summed to obtain the adjusted loss value L = 1.25;

[0107] If L exceeds the threshold of 1.0, the Adam optimizer is used to update the model parameters θ with a learning rate of 0.001. After 5 iterations, θ_new is obtained.

[0108] Based on θ_new, α_i = [0.28, 0.48, 0.24] is recalculated, and a new weight set W_i' = [0.19, 0.32, 0.16] is generated;

[0109] The loss value L' = 1.18 was recalculated using W_i', and the model output bounding box coordinates and class probabilities were adjusted.

[0110] The feature enhancement vector F_enhanced = [0.7, 0.3, 0.9] is extracted from the attention module and concatenated with the original feature F_original = [0.5, 0.6, 0.8].

[0111] When the confidence level of the environmental branch is 0.85 > T3 = 0.8, we take λ = 0.6 and calculate the final feature [0.58, 0.48, 0.84] according to F_final = 0.6 × [0.5, 0.6, 0.8] + 0.4 × [0.7, 0.3, 0.9].

[0112] like Figures 1-2 As shown, in step S104, the target detection branch and the environment adaptation branch are trained simultaneously through a multi-task learning framework. The target detection branch is responsible for identifying the location and category of non-motorized vehicles, while the environment adaptation branch generates a feature enhancement vector for the current environment based on dynamic weight coefficients, thus obtaining an environment-adaptive feature representation.

[0113] Further, in step S104, parallel training is initialized through a multi-task learning framework, and initial features of the input image are extracted using a convolutional neural network to obtain the first feature set;

[0114] The input to the target detection branch is obtained from the first feature set, and a fast region convolutional neural network is used to detect the position and category of non-motorized vehicles to determine the first position information and the first category information.

[0115] The input to the environment adaptation branch is obtained from the first feature set, and the dynamic weight coefficients are calculated based on the attention mechanism to generate the first feature enhancement vector.

[0116] By fusing the first feature enhancement vector with the first feature set, a second feature set that is adaptive to the environment is generated using matrix multiplication.

[0117] If the feature distribution of the second feature set does not match the preset environmental feature threshold, the dynamic weight coefficients are adjusted through gradient backpropagation to obtain an optimized second feature enhancement vector.

[0118] Enhanced features are obtained from the optimized second feature enhancement vector, and combined with the first position information and the first category information to generate a third feature set through feature concatenation.

[0119] The parameters of the target detection branch are updated using the third feature set, and non-motorized vehicles are re-detected through a fast region convolutional neural network to obtain the final location and category information;

[0120] The image data of the current scene is denoised and standardized using preprocessing techniques to obtain the first image data.

[0121] Environmental feature vectors are extracted from the first image data, and the complexity of lighting, weather, and background is analyzed using feature extraction algorithms to obtain an environmental feature set.

[0122] Specifically, in step S104, parallel training is initialized through a multi-task learning framework, and the initial features of the input image are extracted using a ResNet-50 convolutional neural network, outputting a 1024-dimensional feature vector as the first feature set.

[0123] Input the Faster R-CNN object detection branch from the first feature set, set the anchor point size to [32, 64, 128], fine-tune the weights based on the pre-trained weights in the COCO dataset, and output the bounding box coordinates and class probability distribution of non-motorized vehicles as the first location information and the first class information;

[0124] Simultaneously, the environment adaptation branch is input from the first feature set, and the dynamic weight coefficients are calculated using a multi-head attention mechanism. The number of attention heads is set to 8, and after normalization by the Softmax function, a 128-dimensional first feature enhancement vector is generated.

[0125] Perform matrix multiplication between the first feature enhancement vector and the first feature set, and generate a 2048-dimensional second feature set after aligning the feature dimensions.

[0126] If the KL divergence value of the second feature set deviates from the preset environmental feature threshold of 0.05 by more than 15%, the attention layer parameters are adjusted by backpropagation through the Adam optimizer, and the optimized 256-dimensional second feature enhancement vector is obtained after 3 iterations.

[0127] The optimized second feature enhancement vector is concatenated with the first position information and the first category information, and a fully connected layer is used for dimension matching to generate a 3072-dimensional third feature set;

[0128] Update the ROI pooling layer parameters of Faster R-CNN with the third feature set, set the learning rate to 0.001, and output the corrected bounding box coordinates and class confidence after re-detection;

[0129] The input image is denoised by Gaussian filtering (kernel size 5×5, σ=1.5), and then the mean-variance normalization of ImageNet is performed to obtain the first image data after preprocessing.

[0130] The pre-trained VGG16 network was used to extract the conv5 layer features of the first image data. The dimensionality was reduced to 64 dimensions by PCA, and the light intensity (threshold > 150 lux), rain and fog index (threshold < 0.3) and background entropy value (threshold > 6.5) were used to calculate the environmental feature set.

[0131] like Figures 1-2As shown, in step S105, after obtaining the feature enhancement vector, a feature fusion mechanism is established. If the confidence level of the output of the environmental adaptation branch is greater than the threshold T3, the enhancement vector and the original feature are weighted and fused. The fusion formula is F_final=λ×F_original+(1-λ)×F_enhanced, where λ represents the fusion weight parameter.

[0132] Further, in step S105, the original feature vector F_original and the enhanced feature vector F_enhanced are obtained, and two sets of feature vectors are obtained through sensor data processing and feature extraction algorithms;

[0133] Calculate the confidence score of the environmental adaptation branch, analyze the environmental data using a pre-defined classification model, and obtain the confidence score value P.

[0134] Determine whether the confidence value P is greater than the threshold T3. If it is greater than the threshold T3, then initiate the feature fusion mechanism and proceed to the weighted fusion process.

[0135] The weighted fusion formula F_final=λ×F_original+(1-λ)×F_enhanced is used to fuse the original feature vector F_original and the enhanced feature vector F_enhanced to obtain the final fused feature F_final;

[0136] Obtain the environmental complexity value, analyze environmental characteristics through sensor data or preset models, and obtain the environmental complexity value C;

[0137] Based on the environmental complexity value C, the weight coefficients of each sub-loss term are calculated using the exponential decay formula W_i=α_i×exp(-β×C), and the weight value W_i is obtained.

[0138] The hierarchical loss function is updated by using a weighted summation method and combining the weight values ​​W_i to obtain the adjusted loss function value L;

[0139] Determine whether the adjusted loss function value L exceeds the preset threshold. If it exceeds the threshold, optimize the model parameters using the gradient descent algorithm to obtain the optimized model parameter set.

[0140] Based on the optimized model parameter set, the weight coefficients of the adaptive weight allocation algorithm are updated to obtain a new set of weight coefficients.

[0141] Specifically, in step S105, raw environmental data is collected by a multimodal sensor, a 128-dimensional original feature vector F_original is extracted using a ResNet-50 network, and a 256-dimensional enhanced feature vector F_enhanced is generated using a self-attention mechanism.

[0142] The pre-trained VGG-16 classification model performs probabilistic analysis on environmental factors such as illumination and noise, outputting a confidence value P = 0.92. The fusion condition is triggered when the threshold T3 = 0.85 is set.

[0143] Using a fusion weight of λ = 0.3, a linear weighting operation F_final = 0.3 × F_original + 0.7 × F_enhanced is performed to generate fusion features;

[0144] Using YOLOv5 to detect the density and trajectory of objects in the environment, the computational complexity is C = 1.8. When α_i = 0.5 and β = 0.2, the sub-loss weights are obtained by W_i = 0.5 × exp(-0.2 × 1.8) ≈ 0.35.

[0145] Weight coefficients are introduced into the cross-entropy loss function, and the SGD optimizer is used to update the fully connected layer parameters with a learning rate of 0.01. When the loss value L = 1.2 exceeds the threshold of 1.0, parameter updates are triggered.

[0146] The coefficients of α_i are adjusted according to the gradient direction output by the Adam optimizer, and a new weight set {0.38, 0.42, 0.20} is generated for the next round of training.

[0147] like Figures 1-2 As shown, in step S106, the comprehensive loss value is calculated based on the fused feature representation. The comprehensive loss function L_total consists of three parts: classification loss L_cls, localization loss L_loc, and environmental adaptation loss L_env. The network parameters are updated through the backpropagation algorithm to obtain the model weights that adapt to multiple environmental changes.

[0148] Further, in step S106, input data is acquired, and fused feature representations are extracted through a convolutional neural network to obtain feature vectors;

[0149] Based on the feature vector, the classification loss L_cls is calculated using the cross-entropy loss function to obtain the classification error;

[0150] To address the classification error, the localization loss L_loc is calculated using the bounding box regression algorithm to obtain the localization error.

[0151] If the positioning error exceeds the preset threshold, the environmental features are extracted through the environmental perception module, the environmental adaptation loss L_env is calculated, and the environmental error is obtained.

[0152] Based on the classification error, positioning error, and environmental error, the weighted summation method is used to calculate the comprehensive loss value L_total, thus obtaining the comprehensive loss.

[0153] The optimized parameters are obtained by updating the network parameters based on the comprehensive loss using the backpropagation algorithm.

[0154] The model weights are updated by optimizing the parameters to obtain model weights that adapt to changes in multiple environments;

[0155] Obtain the environmental complexity value, analyze environmental characteristics through sensor data or preset models, and obtain the environmental complexity value C;

[0156] Based on the environmental complexity value C, the weight coefficients of each sub-loss term are calculated using the exponential decay formula W_i=α_i×exp(-β×C), resulting in a new set of weight coefficients.

[0157] Specifically, in step S106, input data is acquired, and fused feature representations are extracted through a convolutional neural network to obtain feature vectors. For example, if the size of the input RGB image is 224×224×3, a 1024-dimensional feature vector is extracted through a ResNet50 network.

[0158] Based on the feature vector, the classification loss L_cls is calculated using the cross-entropy loss function to obtain the classification error. Specifically, the class probability distribution is calculated using the Softmax function. When the true class is the 3rd class and the predicted probability is 0.2, the classification error is 1.609.

[0159] To address the classification error, the localization loss L_loc is calculated using the bounding box regression algorithm to obtain the localization error. The Smooth L1 loss function is adopted. When the predicted box coordinates are (50, 60, 120, 150) and the true box coordinates are (55, 65, 115, 145), the localization error is 0.8.

[0160] If the positioning error exceeds the preset threshold of 0.5, the environmental features are extracted through the environmental perception module, the environmental adaptation loss L_env is calculated, and the environmental error is obtained. For example, when the light intensity is 300 lux, the environmental error is 0.3.

[0161] Based on the classification error of 1.609, the positioning error of 0.8, and the environmental error of 0.3, the weighted summation method is used to calculate the comprehensive loss value L_total, with weights of 0.6, 0.3, and 0.1, respectively, resulting in a comprehensive loss of 1.227.

[0162] The network parameters are updated based on the comprehensive loss using the backpropagation algorithm, and the convolution kernel weights are adjusted with a learning rate of 0.001 using the Adam optimizer to obtain the optimized parameters.

[0163] The model weights are updated by optimizing the parameters to obtain model weights that adapt to changes in multiple environments. For example, after the update, the L2 norm of the 5th convolutional kernel changes from 1.2 to 1.15.

[0164] Obtain environmental complexity values ​​and analyze environmental characteristics using sensor data or preset models. For example, if a PM2.5 concentration of 80 μg / m³ is detected.3 The environmental complexity value was obtained as C = 0.7;

[0165] Based on the environmental complexity value C = 0.7, the weight coefficients of each sub-loss term are calculated using the exponential decay formula W_i = α_i × exp(-β × C). Let α_cls = 0.6, α_loc = 0.3, and β = 0.5, to obtain a new set of weight coefficients W_cls = 0.42 and W_loc = 0.21.

[0166] like Figures 1-2 As shown, in step S107, an online learning strategy is used to continuously optimize the trained model. When a new environmental change pattern is detected, the model parameters are updated through an incremental learning method. If the recognition accuracy of the new scene is lower than the preset threshold T4, a local update mechanism for the model parameters is triggered.

[0167] Furthermore, in step S107, the real-time data stream of the current scene is acquired, and the data is segmented and preprocessed using stream processing technology to obtain a standardized data stream;

[0168] Based on standardized data streams, environmental change detection algorithms are used to analyze data characteristics and obtain a set of change patterns.

[0169] The system matches the set of change patterns with a preset environmental feature library. If a match is found, the corresponding environmental change pattern is determined.

[0170] If there is no match, the default environment mode will be used to obtain the environment change mode label;

[0171] Based on the environmental change pattern labels, the parameters of the trained model are adjusted using an incremental learning algorithm to obtain the updated model parameters;

[0172] The updated model parameters are used to perform a recognition task on the new scene data, the recognition accuracy is calculated, and the current recognition accuracy value is obtained.

[0173] If the current recognition accuracy is lower than the preset threshold T4, a local update mechanism is triggered, and the parameters of a specific layer of the model are adjusted through a gradient optimization algorithm to obtain the optimized model parameters.

[0174] If the value is higher than or equal to the threshold T4, the optimized model parameters are output directly.

[0175] Based on the optimized model parameters, the recognition task is re-executed on the new scene data to calculate the new recognition accuracy and obtain the updated accuracy value.

[0176] If the updated accuracy value is lower than the preset threshold T4, then incremental learning and local update mechanisms are executed iteratively to adjust the model parameters and obtain the final optimized model parameters.

[0177] If the value is higher than or equal to the threshold T4, then the final optimized model parameters are determined.

[0178] By optimizing the model parameters, target detection is performed on the new scene data to generate structured recognition results and confidence scores, resulting in the final output data.

[0179] Specifically, in step S107, the Kafka streaming framework is used to obtain real-time video data streams, the window size is set to 256 frames, and the data is preprocessed by Gaussian filtering and Z-score normalization to output a normalized data stream.

[0180] The KL divergence-based environmental change detection algorithm is used to analyze the standardized data stream, calculate the difference in feature distribution between consecutive frames, and record the change point when the difference value exceeds 0.15, generating a set of change patterns including sudden changes in illumination and increased fog.

[0181] The set of change patterns is matched with the pre-built Elasticsearch environment feature library for similarity. The cosine similarity threshold is set to 0.8. When the match is successful, the pattern label such as "strong backlight" is returned; otherwise, it is marked as "unknown pattern".

[0182] The Online Random Forest incremental learning algorithm is adopted, and the model parameters are adjusted according to the pattern label. The learning rate is set to 0.01, and the weights of the fully connected layer are updated.

[0183] The updated model was evaluated on the test set (500 frames of new scene data), and the mAP value was calculated as an accuracy metric.

[0184] If mAP is lower than the threshold T4 (set to 0.85), the LAMB optimizer is used to locally fine-tune the third convolutional layer of ResNet34 for 5 epochs.

[0185] Re-evaluate the fine-tuned model. If the mAP is still below the threshold, repeat the incremental learning process and adjust the learning rate to 0.005.

[0186] The final model uses NMS post-processing to output non-motorized vehicle detection boxes and confidence scores, retains prediction results with scores higher than 0.7, and generates structured JSON data containing entries such as "electric vehicle: 0.92".

[0187] like Figures 1-2 As shown, in step S108, multiple sub-models trained for different environmental conditions are fused using model ensemble technology. The most suitable combination of sub-models is automatically selected based on the environmental characteristics of the current scene to obtain the final non-motorized vehicle recognition result and confidence score.

[0188] Further, in step S108, image data of the current scene is acquired, and the image is denoised and standardized using preprocessing techniques to obtain the first image data;

[0189] Environmental feature vectors are extracted from the first image data, and the complexity of lighting, weather, and background is analyzed using feature extraction algorithms to obtain an environmental feature set.

[0190] If the environmental feature set matches the pre-established environmental condition library, the corresponding sub-model set is determined based on the matching result;

[0191] If there is no match, the default set of sub-models is used to obtain the candidate sub-model group;

[0192] The candidate sub-model groups are weighted and fused using model ensemble techniques, and the weights of each sub-model are calculated using ensemble learning algorithms to obtain the fused model.

[0193] Based on the fusion model, non-motorized vehicle targets are detected in the first image data, and preliminary recognition results and confidence scores are output.

[0194] If the confidence score of the preliminary identification result is lower than the preset threshold, the weight of the fusion model is adjusted by the environmental feature set, and the target detection is performed again to obtain the final identification result.

[0195] Based on the final recognition results, generate non-motorized vehicle category labels and corresponding confidence scores, and output structured recognition data;

[0196] New environmental data is acquired from the data stream through an online learning strategy, environmental change patterns are detected, and a set of change patterns is obtained.

[0197] Based on the set of change patterns, incremental learning is used to adjust the parameters of the trained model to obtain updated model parameters.

[0198] Specifically, in step S108, image data of the current scene is acquired, and Gaussian filtering (σ = 1.5) and histogram equalization are used for denoising and standardization to obtain first image data with a size normalized to 640×480.

[0199] The illumination intensity (threshold range 0-100 lux), SIFT feature point density (≥200 key points per frame), and background complexity score (0-1 interval) of semantic segmentation output in the HSV color space are extracted from the first image data to form a 128-dimensional environment feature vector;

[0200] If the environmental feature vector matches the pre-built KNN environmental condition library (Euclidean distance ≤ 0.3), then the YOLOv5s and Faster R-CNN sub-models for the corresponding lighting conditions (such as cloudy or rainy weather) are called.

[0201] Otherwise, the default ResNet50+RetinaNet combination is loaded to form a candidate group containing 3 sub-models;

[0202] The AdaBoost ensemble algorithm was used to calculate the weight coefficients (0.5 / 0.3 / 0.2) based on the mAP (0.85 / 0.78 / 0.72) of the sub-models on the validation set, and then construct the fusion model.

[0203] The fusion model is used to perform non-motorized vehicle detection on the first image, and the detection box and initial confidence score (threshold 0.6) are output.

[0204] If the confidence level of the detected electric bicycle is 0.55, the YOLOv5s weights are dynamically increased (from 0.5 to 0.7) through the feature vector, and the corrected result (confidence level 0.65) is obtained after re-detection.

[0205] The structured output contains JSON data with the format "bicycle: 0.92, tricycle: 0.88".

[0206] Real-time acquisition of new environmental images based on data stream (5 frames per second), detection of sudden changes in illumination through LSTM network (Δlux≥20 for 10 consecutive frames), and recording as a set of change patterns;

[0207] The online random forest algorithm was used to update the parameters of the fully connected layers of the model with an incremental learning rate of 0.01. When the mAP dropped to 0.68 on the new scene test set, the backbone network was frozen and only the FPN layer was fine-tuned (lr=0.001). After 3 iterations, the mAP recovered to 0.75.

[0208] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this application.

Claims

1. A hierarchical loss function-based image non-motor vehicle detection and recognition method, characterized in that, The method comprises the following steps: acquiring an input non-motor vehicle image, performing feature extraction on the non-motor vehicle image through a multi-scale feature extraction network to obtain a multi-scale feature map; detecting a non-uniform illumination area in the multi-scale feature map and performing illumination standardization processing to obtain an illumination standardized feature map; generating a high-dimensional feature vector according to the illumination standardized feature map through deep feature extraction, and obtaining a low-dimensional feature vector through dimension reduction processing; classifying the low-dimensional feature vector through a preset classification model to obtain a classification result of the non-motor vehicle image and acquire an environment complexity level; dynamically adjusting a weight coefficient of a hierarchical loss function according to the environment complexity level, simultaneously training a target detection branch and an environment adaptation branch through a multi-task learning framework, generating an environment self-adaptive feature representation, fusing original features and enhanced features through a feature fusion mechanism, and obtaining final fusion features; calculating a comprehensive loss value according to the final fusion features, updating model parameters, and obtaining a non-motor vehicle recognition result.

2. The image non-motor vehicle detection and recognition method based on hierarchical loss function according to claim 1, characterized in that, The method further comprises the following steps: classifying the low-dimensional feature vector through a preset classification model to obtain a classification result of the non-motor vehicle image and calculate a loss function of a current level; the loss function of the current level is composed of a classification loss L cls, a positioning loss L loc and an environment adaptation loss L_env. acquiring an environment complexity level of each scale according to the multi-scale feature map; dynamically adjusting a weight coefficient of a hierarchical loss function according to the environment complexity level, and calculating a final weighted loss value; simultaneously training a target detection branch and an environment adaptation branch through a multi-task learning framework, updating model parameters by using the weighted loss value, and obtaining a non-motor vehicle recognition result. 3.The image non-motor vehicle detection and recognition method based on hierarchical loss function according to claim 1, characterized in that, The illumination standardization processing adopts histogram equalization, specifically comprising the following steps: if it is detected that a standard deviation of pixel values in a region of the multi-scale feature map exceeds a preset threshold, a CLAHE algorithm is used for histogram equalization, and a limited contrast parameter is set to 2.

4. The image non-motor vehicle detection and recognition method based on hierarchical loss function according to claim 1, characterized in that, The method for acquiring an environment complexity level comprises the following steps: calculating a brightness distribution and a brightness variance of the feature map, if the brightness variance is greater than a preset threshold T1, performing histogram equalization; extracting a contrast change parameter of the equalized image, if a proportion of regions with a global contrast value less than a threshold T2 exceeds a preset proportion, extracting edge sharpness through a Laplace operator; calculating an environment complexity parameter based on edge point density and edge intensity, and determining whether it is a complex illumination environment through a decision tree algorithm.

5. The image non-motor vehicle detection and recognition method based on hierarchical loss function according to claim 1, characterized in that, An adaptive weight allocation algorithm is used to dynamically adjust the weight coefficients of each sub-loss term in the hierarchical loss function according to the level of environmental complexity, using the formula: W i =α i ×e -βi×C Calculate the weight of the i-th loss value Li, where α i β represents the basic weighting coefficient. i Let represent the decay factor of the i-th loss function, and C represent the environmental complexity value.

6. The image non-motor vehicle detection and recognition method based on hierarchical loss function according to claim 1, characterized in that, The loss value calculated according to the hierarchical loss function is used to update network parameters through a back propagation algorithm, and a model weight suitable for multiple environment changes is obtained.

Images