Intelligent detection method and device for road surface of logistics transport vehicle at night

Abstract

The invention relates to the technical field of driving pavement detection, and discloses a night road pavement intelligent detection method for a logistics transport vehicle. Collecting a night image sample monitored by a front camera of the vehicle, and marking the night image sample to generate a first image sample; performing automatic direction correction on the first image sample by using Harris corner detection and a nearest neighbor matching algorithm to generate a third image sample; configuring a data file of the YOLOv9; establishing a model training environment, taking a third image sample subjected to labeling and automatic direction correction as a data set, modifying a configuration file of YOLOv9, starting a training process by using a provided training script, and in the training process, regularly monitoring a loss function and mAP of a training and verification set to complete model establishment; adjusting and optimizing the model, and exporting and deploying the optimized model; acquiring a test set, and evaluating the model by using the test set; the method has the advantages of being convenient to deploy and improving the model learning effect.

Description

Technical Field

[0001] The present invention relates to the technical field of road surface detection, and in particular to an intelligent detection method and device for a road surface of a logistics transport vehicle at night. Background Art

[0002] At this stage, online shopping has been integrated into people's daily lives, which has also generated an increasing demand for express delivery, and road transportation occupies an important position in express delivery. Due to cracks or other damage on the road, especially when it is not repaired in time, there are certain safety hazards for transportation vehicles. With the continuous increase in nighttime road transportation, the research and development of nighttime road surface detection has become particularly important. However, the challenges faced by nighttime road surface detection are far more complex than those during the day, mainly because the lighting conditions at night are poor and the visibility of the road surface is low, which greatly increases the difficulty of detection.

[0003] Therefore, a method and device for intelligent detection of road pavement at night for logistics transport vehicles are provided to solve the above problems. Summary of the invention

[0004] The main purpose of the present invention is to solve the problem in the prior art that due to cracks or other damage on the road, especially when maintenance is not done in time, there are certain safety hazards for transport vehicles. With the continuous increase of nighttime road transportation, the research and development of nighttime road surface detection has become particularly important. However, the challenges faced by nighttime road surface detection are far more complex than those during the day, mainly because the lighting conditions at night are poor and the visibility of the road surface is low, which greatly increases the difficulty of detection.

[0005] A first aspect of the present invention provides a method for intelligently detecting a road surface of a logistics transport vehicle at night, the method comprising:

[0006] Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample;

[0007] Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample;

[0008] Configure YOLOv9 data file;

[0009] Build a model training environment, use the annotated and automatically oriented third image samples as the data set, modify the YOLOv9 configuration file, and use the provided training script to start the training process. During the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0010] Adjust and optimize the model, export and deploy the optimized model;

[0011] Get the test set and use it to evaluate the model.

[0012] Optionally, the collecting night image samples monitored by the front camera of the vehicle and annotating the night image samples to generate the first image samples includes:

[0013] Collect night image samples monitored by the vehicle's front camera;

[0014] Import the night image samples into LabelImg software;

[0015] Select the label and color of the marker box, and adjust the size and position of the marker box;

[0016] After annotation is completed, save and export the first image sample.

[0017] Optionally, the automatic direction correction of the first image sample by using Harris corner detection and nearest neighbor matching algorithm includes:

[0018] Determine whether the labeled first image sample is a grayscale image, and if not, convert it into a grayscale image to generate a second image sample;

[0019] calculating horizontal and vertical gradients of the second image samples using an edge detection operator;

[0020] Apply the Harris corner detection algorithm, calculate the Harris response value of each pixel, set a threshold, filter out pixels with response values ​​higher than the threshold as potential corners, apply the non-maximum suppression algorithm to remove non-maximum response values ​​in adjacent pixels, and determine the corner position;

[0021] Extract feature descriptors around the detected corners;

[0022] Matching the corner features of the second image sample with the corner features of the reference image using a nearest neighbor matching algorithm;

[0023] Using the matched corner point pairs, a transformation matrix is ​​estimated using the RANSAC (Random Sampling Consensus) algorithm;

[0024] Using the transformation matrix, rotating and translating the second image sample;

[0025] The blank areas generated by the rotation and translation are filled by an interpolation algorithm to generate a third sample image.

[0026] Optionally, modifying the configuration file of YOLOv9 includes: modifying batch size information, learning rate information, and number of iterations information configured by YOLOv9.

[0027] Optionally, the adjusting and optimizing the model includes:

[0028] According to the training curve, the learning rate is adjusted using the cosine annealing strategy or in stages;

[0029] Enhance model data by rotating, flipping, scaling, cropping, and color jittering;

[0030] After training, the model is pruned, unimportant weights are removed or quantized, and floating-point operations are converted to integer operations.

[0031] Based on the basic model, fine-tune it on a custom dataset using a smaller learning rate for specific application scenarios or goals;

[0032] Weighted fusion of model prediction results at different training stages or with different hyperparameter settings;

[0033] Evaluate model performance using a validation set or test set that was not used in training.

[0034] Optionally, it also includes: using visualization tools to compare the model's prediction results on the test set with the actual annotations.

[0035] The second aspect of the present invention provides an intelligent detection device for a nighttime road surface of a logistics transport vehicle, comprising:

[0036] A night image sample labeling module is used to collect night image samples monitored by the front camera of the vehicle, and label the night image samples to generate a first image sample;

[0037] An automatic direction correction module, used to perform automatic direction correction on the first image sample by using Harris corner point detection and nearest neighbor matching algorithm to generate a third image sample;

[0038] Data file configuration module, used to configure YOLOv9 data files;

[0039] The model training module is used to build the model training environment, use the third image sample that has been labeled and automatically oriented as the data set, modify the YOLOv9 configuration file, and use the provided training script to start the training process. During the training process, the loss function and mAP (mean Average Precision) of the training and validation sets are regularly monitored to complete the model establishment.

[0040] Model adjustment and optimization module, used to adjust and optimize the model, export and deploy the optimized model;

[0041] The test set model evaluation module is used to obtain the test set and use the test set to evaluate the model.

[0042] A third aspect of the present invention provides an electronic device, the electronic device comprising a memory and at least one processor, the memory storing instructions;

[0043] The at least one processor calls the instructions in the memory so that the electronic device executes the various steps of the intelligent detection method for nighttime road surfaces of logistics transport vehicles as described above.

[0044] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of the method for intelligent detection of road pavement at night for logistics transport vehicles as described above.

[0045] The technical solution of the present invention provides an intelligent detection algorithm for nighttime road pavement of logistics transport vehicles. The algorithm can eliminate the differences introduced by different photographic equipment and shooting conditions, so that the model can focus more on the nighttime road pavement detection task itself; improve the learning effect of the model; and facilitate deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A first flow chart of a method for intelligently detecting a road surface at night for a logistics transport vehicle provided by an embodiment of the present invention;

[0047] Figure 2 A second flow chart of a method for intelligently detecting a road surface at night for a logistics transport vehicle provided by an embodiment of the present invention;

[0048] Figure 3 A third flow chart of a method for intelligently detecting a road surface at night for a logistics transport vehicle provided by an embodiment of the present invention;

[0049] Figure 4 A fourth flow chart of a method for intelligently detecting a road surface at night for a logistics transport vehicle provided by an embodiment of the present invention;

[0050] Figure 5 A fifth flow chart of a method for intelligently detecting a road surface at night for a logistics transport vehicle provided by an embodiment of the present invention;

[0051] Figure 6 A schematic diagram of the structure of an intelligent detection device for nighttime road pavement of a logistics transport vehicle provided by an embodiment of the present invention;

[0052] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The embodiment of the present invention provides an intelligent detection method for nighttime road pavement of a logistics transport vehicle, including collecting nighttime image samples monitored by a front camera of the vehicle, annotating the nighttime image samples to generate a first image sample; using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample; configuring a YOLOv9 data file; building a model training environment, using the annotated and automatically corrected third image sample as a data set, modifying the configuration file of YOLOv9, using the provided training script to start the training process, during the training process, regularly monitoring the loss function and mAP (mean Average Precision) of the training and validation sets, and completing the model establishment; adjusting and optimizing the model, exporting and deploying the optimized model; obtaining a test set, and using the test set to evaluate the model. The present invention solves the problem in the prior art that due to cracks or other damage on the road, especially when it is not maintained in time, there are certain safety hazards in the transport vehicle. With the continuous increase in nighttime road transportation, the research and development of nighttime road surface detection has become particularly important. However, the challenges faced by nighttime road surface detection are far more complex than those during the day, mainly because the lighting conditions at night are poor and the visibility of the road surface is low, which greatly increases the difficulty of detection.

[0054] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0055] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the intelligent detection method of the road surface of a logistics transport vehicle at night in the embodiment of the present invention includes:

[0056] Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample;

[0057] Specifically, they include:

[0058] Collect night image samples monitored by the vehicle's front camera;

[0059] Import the night image samples into LabelImg software;

[0060] Select the label and color of the marker box, and adjust the size and position of the marker box;

[0061] After annotation is completed, save and export the first image sample.

[0062] Night image samples include: video or image data recorded by a high-definition camera installed on the front of the vehicle during driving in different night scenes, such as urban streets, suburban roads, highways, etc. These data should cover various weather conditions (such as clear, rainy, foggy, snowy), different road lighting conditions (street lighting, no lighting, reflective signs) and roadside conditions (water accumulation, snow accumulation, cracks, potholes).

[0063] Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample;

[0064] Configure YOLOv9 data file;

[0065] Build a model training environment, use the annotated and automatically oriented third image samples as the data set, modify the YOLOv9 configuration file, and use the provided training script to start the training process. During the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0066] Adjust and optimize the model, export and deploy the optimized model;

[0067] Get the test set and use it to evaluate the model.

[0068] See also Figure 2 The second embodiment of the intelligent detection method of the road surface of the logistics transport vehicle at night in the embodiment of the present invention includes:

[0069] Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample;

[0070] Specifically, they include:

[0071] Collect night image samples monitored by the vehicle's front camera;

[0072] Import the night image samples into LabelImg software;

[0073] Select the label and color of the marker box, and adjust the size and position of the marker box;

[0074] After annotation is completed, save and export the first image sample.

[0075] Night image samples include: video or image data recorded by a high-definition camera installed on the front of the vehicle during driving in different night scenes, such as urban streets, suburban roads, highways, etc. These data should cover various weather conditions (such as clear, rainy, foggy, snowy), different road lighting conditions (street lighting, no lighting, reflective signs) and roadside conditions (water accumulation, snow accumulation, cracks, potholes).

[0076] Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample;

[0077] Specifically, they include:

[0078] Determine whether the labeled first image sample is a grayscale image. If not, convert it into a grayscale image to generate a second image sample. If it is already a grayscale image, directly proceed to the next stage.

[0079] calculating horizontal and vertical gradients of the second image samples using an edge detection operator;

[0080] Apply the Harris corner detection algorithm, calculate the Harris response value of each pixel, set a threshold, filter out pixels with response values ​​higher than the threshold as potential corners, apply the non-maximum suppression algorithm to remove non-maximum response values ​​in adjacent pixels, and determine the corner position. This step can accurately determine the corner position;

[0081] Extract feature descriptors around the detected corners;

[0082] Matching the corner features of the second image sample with the corner features of the reference image using a nearest neighbor matching algorithm;

[0083] Using the matched corner point pairs, a transformation matrix is ​​estimated using the RANSAC (Random Sampling Consensus) algorithm;

[0084] Using the transformation matrix, rotating and translating the second image sample;

[0085] The blank areas generated by the rotation and translation are filled by an interpolation algorithm to generate a third sample image.

[0086] Configure YOLOv9 data file;

[0087] Build a model training environment, use the annotated and automatically oriented third image samples as the data set, modify the YOLOv9 configuration file, and use the provided training script to start the training process. During the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0088] Adjust and optimize the model, export and deploy the optimized model;

[0089] Get the test set and use it to evaluate the model.

[0090] See also Figure 3 The third embodiment of the intelligent detection method of the road surface of the logistics transport vehicle at night in the embodiment of the present invention includes:

[0091] Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample;

[0092] Specifically, they include:

[0093] Collect night image samples monitored by the vehicle's front camera;

[0094] Import the night image samples into LabelImg software;

[0095] Select the label and color of the marker box, and adjust the size and position of the marker box;

[0096] After annotation is completed, save and export the first image sample.

[0097] Night image samples include: video or image data recorded by a high-definition camera installed on the front of the vehicle during driving in different night scenes, such as urban streets, suburban roads, highways, etc. These data should cover various weather conditions (such as clear, rainy, foggy, snowy), different road lighting conditions (street lighting, no lighting, reflective signs) and roadside conditions (water accumulation, snow accumulation, cracks, potholes).

[0098] Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample;

[0099] Specifically, they include:

[0100] Determine whether the labeled first image sample is a grayscale image. If not, convert it into a grayscale image to generate a second image sample. If it is already a grayscale image, directly proceed to the next stage.

[0101] calculating horizontal and vertical gradients of the second image samples using an edge detection operator;

[0102] Apply the Harris corner detection algorithm, calculate the Harris response value of each pixel, set a threshold, filter out pixels with response values ​​higher than the threshold as potential corners, apply the non-maximum suppression algorithm to remove non-maximum response values ​​in adjacent pixels, and determine the corner position. This step can accurately determine the corner position;

[0103] Extract feature descriptors around the detected corners;

[0104] Matching the corner features of the second image sample with the corner features of the reference image using a nearest neighbor matching algorithm;

[0105] Using the matched corner point pairs, a transformation matrix is ​​estimated using the RANSAC (Random Sampling Consensus) algorithm;

[0106] Using the transformation matrix, rotating and translating the second image sample;

[0107] The blank areas generated by the rotation and translation are filled by an interpolation algorithm to generate a third sample image.

[0108] Configure YOLOv9 data file;

[0109] YOLOv9 has achieved significant performance improvements, surpassing previous models based on convolutional neural networks (CNN) and Transformer. The following are several key features and innovations of the YOLOv9 model:

[0110] Programmable Gradient Information (PGI): This is a novel training strategy that allows programmable control of gradient information during training to guide model learning more effectively. This mechanism can dynamically adjust the gradient of the loss function and optimize the model's learning of samples of different difficulty levels, thereby improving the model's generalization ability and detection accuracy.

[0111] Generalized Efficient Layer Aggregation Network (GELAN): GELAN is a new network architecture design that achieves a balance between the depth and width of the model while maintaining computational efficiency by integrating multiple efficient network layers and structures. This network structure helps improve feature expression capabilities, allowing the model to better capture target features at different scales.

[0112] Simplified model structure: YOLOv9 is based on models / detect / yolov9.yaml in structure, and compared with previous versions, it reduces auxiliary branches, making the model more concise. In inference mode, although the model structure is similar to the previous generation of YOLO, it emphasizes the application of re-parameter technology and GELAN blocks. These improvements help improve inference speed and detection efficiency.

[0113] Improved detection head: YOLOv9 achieves the final output of target detection through its Detect Head, which includes not only traditional operations such as non-maximum suppression (NMS), but also other optimization measures to ensure that the model can accurately and efficiently output detection boxes and category predictions.

[0114] Wide applicability: YOLOv9 is not limited to image recognition, but performs well in the field of computer vision. The deep learning principles behind it are also applicable to fields such as language processing and speech recognition, demonstrating the potential for cross-domain applications.

[0115] Build a model training environment, use the third image sample that has been labeled and automatically oriented as a data set, modify the YOLOv9 configuration file, specifically including: modify the batch size information, learning rate information, and number of iterations of the YOLOv9 configuration, start the training process using the provided training script, and during the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0116] Adjust and optimize the model, export and deploy the optimized model;

[0117] Get the test set and use it to evaluate the model.

[0118] See also Figure 4 The fourth embodiment of the intelligent detection method of the road surface of the logistics transport vehicle at night in the embodiment of the present invention includes:

[0119] Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample;

[0120] Specifically, they include:

[0121] Collect night image samples monitored by the vehicle's front camera;

[0122] Import the night image samples into LabelImg software;

[0123] Select the label and color of the marker box, and adjust the size and position of the marker box;

[0124] After annotation is completed, save and export the first image sample.

[0125] Night image samples include: video or image data recorded by a high-definition camera installed on the front of the vehicle during driving in different night scenes, such as urban streets, suburban roads, highways, etc. These data should cover various weather conditions (such as clear, rainy, foggy, snowy), different road lighting conditions (street lighting, no lighting, reflective signs) and roadside conditions (water accumulation, snow accumulation, cracks, potholes).

[0126] Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample;

[0127] Specifically, they include:

[0128] Determine whether the labeled first image sample is a grayscale image. If not, convert it into a grayscale image to generate a second image sample. If it is already a grayscale image, directly proceed to the next stage.

[0129] calculating horizontal and vertical gradients of the second image samples using an edge detection operator;

[0130] Apply the Harris corner detection algorithm, calculate the Harris response value of each pixel, set a threshold, filter out pixels with response values ​​higher than the threshold as potential corners, apply the non-maximum suppression algorithm to remove non-maximum response values ​​in adjacent pixels, and determine the corner position. This step can accurately determine the corner position;

[0131] Extract feature descriptors around the detected corners;

[0132] Matching the corner features of the second image sample with the corner features of the reference image using a nearest neighbor matching algorithm;

[0133] Using the matched corner point pairs, a transformation matrix is ​​estimated using the RANSAC (Random Sampling Consensus) algorithm;

[0134] Using the transformation matrix, rotating and translating the second image sample;

[0135] The blank areas generated by rotation and translation are filled with an interpolation algorithm to generate a third sample image, thereby ensuring the integrity of the image.

[0136] Configure YOLOv9 data file;

[0137] YOLOv9 has achieved significant performance improvements, surpassing previous models based on convolutional neural networks (CNN) and Transformer. The following are several key features and innovations of the YOLOv9 model:

[0138] Programmable Gradient Information (PGI): This is a novel training strategy that allows programmable control of gradient information during training to guide model learning more effectively. This mechanism can dynamically adjust the gradient of the loss function and optimize the model's learning of samples of different difficulty levels, thereby improving the model's generalization ability and detection accuracy.

[0139] Generalized Efficient Layer Aggregation Network (GELAN): GELAN is a new network architecture design that achieves a balance between the depth and width of the model while maintaining computational efficiency by integrating multiple efficient network layers and structures. This network structure helps improve feature expression capabilities, allowing the model to better capture target features at different scales.

[0140] Simplified model structure: YOLOv9 is based on models / detect / yolov9.yaml in structure, and compared with previous versions, it reduces auxiliary branches, making the model more concise. In inference mode, although the model structure is similar to the previous generation of YOLO, it emphasizes the application of re-parameter technology and GELAN blocks. These improvements help improve inference speed and detection efficiency.

[0141] Improved detection head: YOLOv9 achieves the final output of target detection through its Detect Head, which includes not only traditional operations such as non-maximum suppression (NMS), but also other optimization measures to ensure that the model can accurately and efficiently output detection boxes and category predictions.

[0142] Wide applicability: YOLOv9 is not limited to image recognition, but performs well in the field of computer vision. The deep learning principles behind it are also applicable to fields such as language processing and speech recognition, demonstrating the potential for cross-domain applications.

[0143] Build a model training environment, use the third image sample that has been labeled and automatically oriented as a data set, modify the YOLOv9 configuration file, specifically including: modify the batch size information, learning rate information, and number of iterations of the YOLOv9 configuration, start the training process using the provided training script, and during the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0144] Adjust and optimize the model, export and deploy the optimized model;

[0145] Specific optimizations include:

[0146] According to the training curve, the learning rate is adjusted using the cosine annealing strategy or in stages;

[0147] Enhance the model data by rotating, flipping, scaling, cropping and color jittering; can enhance the generalization ability of the model;

[0148] After training, the model is pruned to remove or quantize unimportant weights and convert floating-point operations to integer operations. This can improve inference speed and reduce model size.

[0149] Based on the basic model, fine-tune on a custom dataset using a smaller learning rate for specific application scenarios or goals; this can optimize the model's ability to recognize specific categories;

[0150] Weighted fusion of model prediction results at different training stages or with different hyperparameter settings;

[0151] Use validation sets or test sets that were not used in training to evaluate model performance; you can adjust the model based on mAP, recall, precision, and other indicators;

[0152] Get the test set and use it to evaluate the model.

[0153] See also Figure 5 The fifth embodiment of the intelligent detection method of the road surface of the logistics transport vehicle at night in the embodiment of the present invention includes:

[0154] Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample;

[0155] Specifically, they include:

[0156] Collect night image samples monitored by the vehicle's front camera;

[0157] Import the night image samples into LabelImg software;

[0158] Select the label and color of the marker box, and adjust the size and position of the marker box;

[0159] After annotation is completed, save and export the first image sample.

[0160] Night image samples include: video or image data recorded by a high-definition camera installed on the front of the vehicle during driving in different night scenes, such as urban streets, suburban roads, highways, etc. These data should cover various weather conditions (such as clear, rainy, foggy, snowy), different road lighting conditions (street lighting, no lighting, reflective signs) and roadside conditions (water accumulation, snow accumulation, cracks, potholes).

[0161] Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample;

[0162] Specifically, they include:

[0163] Determine whether the labeled first image sample is a grayscale image. If not, convert it into a grayscale image to generate a second image sample. If it is already a grayscale image, directly proceed to the next stage.

[0164] calculating horizontal and vertical gradients of the second image samples using an edge detection operator;

[0165] Apply the Harris corner detection algorithm, calculate the Harris response value of each pixel, set a threshold, filter out pixels with response values ​​higher than the threshold as potential corners, apply the non-maximum suppression algorithm to remove non-maximum response values ​​in adjacent pixels, and determine the corner position. This step can accurately determine the corner position;

[0166] Extract feature descriptors around the detected corners;

[0167] Matching the corner features of the second image sample with the corner features of the reference image using a nearest neighbor matching algorithm;

[0168] Using the matched corner point pairs, a transformation matrix is ​​estimated using the RANSAC (Random Sampling Consensus) algorithm;

[0169] Using the transformation matrix, rotating and translating the second image sample;

[0170] The blank areas generated by rotation and translation are filled with an interpolation algorithm to generate a third sample image, thereby ensuring the integrity of the image.

[0171] Configure YOLOv9 data file;

[0172] YOLOv9 has achieved significant performance improvements, surpassing previous models based on convolutional neural networks (CNN) and Transformer. The following are several key features and innovations of the YOLOv9 model:

[0173] Programmable Gradient Information (PGI): This is a novel training strategy that allows programmable control of gradient information during training to guide model learning more effectively. This mechanism can dynamically adjust the gradient of the loss function and optimize the model's learning of samples of different difficulty levels, thereby improving the model's generalization ability and detection accuracy.

[0174] Generalized Efficient Layer Aggregation Network (GELAN): GELAN is a new network architecture design that achieves a balance between the depth and width of the model while maintaining computational efficiency by integrating multiple efficient network layers and structures. This network structure helps improve feature expression capabilities, allowing the model to better capture target features at different scales.

[0175] Simplified model structure: YOLOv9 is based on models / detect / yolov9.yaml in structure, and compared with previous versions, it reduces auxiliary branches, making the model more concise. In inference mode, although the model structure is similar to the previous generation of YOLO, it emphasizes the application of re-parameter technology and GELAN blocks. These improvements help improve inference speed and detection efficiency.

[0176] Improved detection head: YOLOv9 achieves the final output of target detection through its Detect Head, which includes not only traditional operations such as non-maximum suppression (NMS), but also other optimization measures to ensure that the model can accurately and efficiently output detection boxes and category predictions.

[0177] Wide applicability: YOLOv9 is not limited to image recognition, but performs well in the field of computer vision. The deep learning principles behind it are also applicable to fields such as language processing and speech recognition, demonstrating the potential for cross-domain applications.

[0178] Build a model training environment, use the third image sample that has been labeled and automatically oriented as a data set, modify the YOLOv9 configuration file, specifically including: modify the batch size information, learning rate information, and number of iterations of the YOLOv9 configuration, start the training process using the provided training script, and during the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0179] Adjust and optimize the model, export and deploy the optimized model;

[0180] Specific optimizations include:

[0181] According to the training curve, the learning rate is adjusted using the cosine annealing strategy or in stages;

[0182] Enhance the model data by rotating, flipping, scaling, cropping and color jittering; can enhance the generalization ability of the model;

[0183] After training, the model is pruned to remove or quantize unimportant weights and convert floating-point operations to integer operations. This can improve inference speed and reduce model size.

[0184] Based on the basic model, fine-tune on a custom dataset using a smaller learning rate for specific application scenarios or goals; this can optimize the model's ability to recognize specific categories;

[0185] Weighted fusion of model prediction results at different training stages or with different hyperparameter settings;

[0186] Use validation sets or test sets that were not used in training to evaluate model performance; you can adjust the model based on mAP, recall, precision, and other indicators;

[0187] Get the test set and use it to evaluate the model.

[0188] It also includes using visualization tools to compare the model's prediction results on the test set with the actual annotations, which can improve the intuitiveness of the model.

[0189] The above describes the intelligent detection method of the road surface of the logistics transport vehicle at night in the embodiment of the present invention. The following describes the intelligent detection device of the road surface of the logistics transport vehicle at night in the embodiment of the present invention. Figure 6 , the nighttime road surface intelligent detection device for logistics transport vehicles in the embodiment of the present invention includes:

[0190] A night image sample labeling module 601 is used to collect night image samples monitored by a front camera of a vehicle, and label the night image samples to generate first image samples;

[0191] An automatic direction correction module 602 is used to perform automatic direction correction on the first image sample by using Harris corner point detection and nearest neighbor matching algorithm to generate a third image sample;

[0192] A data file configuration module 603 is used to configure the data file of YOLOv9;

[0193] The model training establishment module 604 is used to build a model training environment, use the third image sample that has been labeled and automatically oriented as a data set, modify the configuration file of YOLOv9, use the provided training script to start the training process, and during the training process, regularly monitor the loss function and mAP (mean Average Precision) of the training and validation sets to complete the model establishment;

[0194] The model adjustment and optimization module 605 is used to adjust and optimize the model, and export and deploy the optimized model;

[0195] The test set model evaluation module 606 is used to obtain a test set and use the test set to evaluate the model.

[0196] above Figure 6 The intelligent detection device for nighttime road pavement of a logistics transport vehicle in an embodiment of the present invention is described in detail from the perspective of modular functional entities. The electronic device in an embodiment of the present invention is described in detail from the perspective of hardware processing.

[0197] Figure 7 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.

[0198] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0199] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of a method for intelligent detection of road pavement at night for logistics transport vehicles.

[0200] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0201] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0202] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent detection of road surfaces of logistics transport vehicles at night, characterized in that: The method for intelligent detection of road surfaces of logistics transport vehicles at night comprises: Collect night image samples monitored by the front camera of the vehicle, and annotate the night image samples to generate a first image sample; Using Harris corner detection and nearest neighbor matching algorithm to automatically correct the direction of the first image sample to generate a third image sample; Configure YOLOv9 data file; Build a model training environment, use the annotated and automatically oriented third image samples as the data set, modify the YOLOv9 configuration file, and use the provided training script to start the training process. During the training process, regularly monitor the loss function and mAP of the training and validation sets to complete the model establishment; Adjust and optimize the model, export and deploy the optimized model; Get the test set and use it to evaluate the model.

2. The intelligent detection method for nighttime road surfaces of logistics transport vehicles according to claim 1 is characterized in that: The collecting of nighttime image samples monitored by the front camera of the vehicle and the labeling of the nighttime image samples to generate the first image samples comprises: Collect night image samples monitored by the vehicle's front camera; Import the night image samples into LabelImg software; Select the label and color of the marker box, and adjust the size and position of the marker box; After annotation is completed, save and export the first image sample.

3. The intelligent detection method for nighttime road surfaces of logistics transport vehicles according to claim 2 is characterized in that: The automatic direction correction of the first image sample by using Harris corner detection and nearest neighbor matching algorithm comprises: Determine whether the labeled first image sample is a grayscale image, and if not, convert it into a grayscale image to generate a second image sample; calculating horizontal and vertical gradients of the second image samples using an edge detection operator; Apply the Harris corner detection algorithm, calculate the Harris response value of each pixel, set a threshold, filter out pixels with response values ​​higher than the threshold as potential corners, apply the non-maximum suppression algorithm to remove non-maximum response values ​​in adjacent pixels, and determine the corner position; Extract feature descriptors around the detected corners; Matching the corner features of the second image sample with the corner features of the reference image using a nearest neighbor matching algorithm; Using the matched corner point pairs, a transformation matrix is ​​estimated through the RANSAC algorithm; Using the transformation matrix, rotating and translating the second image sample; The blank areas generated by the rotation and translation are filled by an interpolation algorithm to generate a third sample image.

4. The intelligent detection method for nighttime road surfaces of logistics transport vehicles according to claim 1 is characterized in that: Modifying the configuration file of YOLOv9 includes: modifying the batch size information, learning rate information and number of iterations information configured by YOLOv9.

5. The intelligent detection method for nighttime road surfaces of logistics transport vehicles according to claim 1 is characterized in that: The adjustment and optimization of the model includes: According to the training curve, the learning rate is adjusted using the cosine annealing strategy or in stages; Enhance model data by rotating, flipping, scaling, cropping, and color jittering; After training, the model is pruned, unimportant weights are removed or quantized, and floating-point operations are converted to integer operations. Based on the basic model, fine-tune it on a custom dataset using a smaller learning rate for specific application scenarios or goals; Weighted fusion of model prediction results at different training stages or with different hyperparameter settings; Evaluate model performance using a validation set or test set that was not used in training.

6. The intelligent detection method for nighttime road surfaces of logistics transport vehicles according to claim 1 is characterized in that: Also includes: Use visualization tools to compare the model's predictions on the test set with the actual annotations.

7. An intelligent detection device for nighttime road surfaces of logistics transport vehicles, characterized in that: include: A night image sample labeling module is used to collect night image samples monitored by the front camera of the vehicle, and label the night image samples to generate a first image sample; An automatic direction correction module, used to perform automatic direction correction on the first image sample by using Harris corner point detection and nearest neighbor matching algorithm to generate a third image sample; Data file configuration module, used to configure YOLOv9 data files; The model training module is used to build the model training environment, use the third image samples that have been annotated and automatically oriented as the data set, modify the YOLOv9 configuration file, and use the provided training script to start the training process. During the training process, the loss function and mAP of the training and validation sets are regularly monitored to complete the model establishment; Model adjustment and optimization module, used to adjust and optimize the model, export and deploy the optimized model; The test set model evaluation module is used to obtain the test set and use the test set to evaluate the model.

8. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the electronic device executes each step of the intelligent detection method for nighttime road pavement of a logistics transport vehicle as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the intelligent detection method of the road surface at night for a logistics transport vehicle as claimed in any one of claims 1 to 6 are implemented.

Images