A target object detection method, device and equipment under extreme weather and medium

Abstract

This application discloses a method, apparatus, device, and medium for target object detection under extreme weather conditions, relating to the field of autonomous driving. The method includes: setting extreme weather conditions in a target simulation scene; configuring a virtual camera sensor to collect initial simulation images to generate a virtual dataset; acquiring a real extreme weather dataset; aligning the labels of the virtual dataset and the real extreme weather dataset; extracting simulation images from the virtual dataset based on a preset ratio; adding the simulation images to the real extreme weather dataset to construct a sample dataset; extracting statistical features from the sample dataset to obtain normalized image statistical features to determine the degradation type; performing image enhancement operations on the images to obtain an enhanced dataset; training an initial detection network using the enhanced dataset to obtain a target detection network; enhancing the extreme weather image to be detected to generate an enhanced image; inputting the enhanced image into the target detection network for processing and outputting the target object. This improves the accuracy of target detection and positioning under adverse conditions.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and in particular to a method, apparatus, device, and medium for detecting target objects under extreme weather conditions. Background Technology

[0002] Severe or extreme weather conditions (fog, rain, snow, low light) can cause degradation issues in images captured by vehicle-mounted cameras, such as decreased contrast, blurred target edges, and missing texture features, directly reducing the performance of target detection models. YOLOv8, as a mainstream single-stage detection algorithm, performs well in clear weather, but its performance drops significantly in severe weather. Existing approaches to target detection in severe weather mainly fall into two categories: first, preprocessing the image by dehazing / enhancing it before inputting it into the detection network; second, using generative adversarial networks to synthesize severe weather images and expand the training set.

[0003] In conclusion, improving the recall rate and positioning accuracy of target detection under harsh conditions is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for target object detection under extreme weather conditions, which can improve the recall rate and positioning accuracy of target detection under harsh conditions. The specific solution is as follows: Firstly, this application provides a target object detection method under extreme weather conditions, including: A target simulation scenario is built in a driving simulation platform, and extreme weather conditions including various visibility parameters are set. A virtual camera sensor is configured in the target simulation scenario, the simulation is run, and the initial simulation images under each extreme weather condition are collected through the virtual camera sensor. The category labels and bounding box annotations of the target objects in each initial simulation image are generated to obtain an annotated virtual dataset. A real extreme weather dataset is obtained. A label mapping script is used to align the category semantic labels of the virtual dataset and the real extreme weather dataset. After the labels of the virtual dataset and the real extreme weather dataset are aligned, corresponding simulated images are extracted from the virtual dataset based on a preset ratio. The extracted simulated images are then added to the real extreme weather dataset to construct a sample dataset. The preset ratio is the ratio of the number of simulated images from the virtual dataset to the number of real images from the real extreme weather dataset in the sample dataset. The image statistical features in the sample dataset are extracted and normalized by a preprocessing module to obtain normalized image statistical features. Based on the normalized image statistical features, the degradation type of the image is determined according to a pre-defined rule. Based on the degradation type, the image in the sample dataset is enhanced to obtain an enhanced dataset. Using the enhanced dataset, an initial detection network is trained based on the target detection loss function to obtain a target detection network. The extreme weather image to be detected is then acquired. The image statistical features of the extreme weather image to be detected are extracted and enhanced by the preset preprocessing module to generate an enhanced image. The enhanced image is then input into the target detection network for processing, and the bounding box coordinates and category labels of the target objects in the extreme weather image to be detected are output.

[0005] Optionally, the step of building a target simulation scene in the driving simulation platform and setting extreme weather conditions including various visibility parameters, and configuring a virtual camera sensor in the target simulation scene, includes: In the driving simulation platform, use the line drawing tool with grid and grid alignment functions to draw static scenes; the static scenes include any one or more of the following: straight and curved road paths, lanes, sidewalks, and static scene objects. The target simulation scene is obtained by configuring dynamic participants in the static scene; the dynamic participants include any one or more of the target object, background object, and pedestrians. The weather configuration command sent by the user terminal is obtained through the simulation control protocol, and the extreme weather conditions including various visibility parameters are set based on the weather configuration command; the extreme weather conditions include any one or more of foggy days, rainy days and snowy days; A virtual camera is placed in the target simulation scene, and the horizontal field of view, vertical field of view, and output image resolution of the virtual camera's sensor are configured.

[0006] Optionally, the simulation is run and initial simulation images under each of the extreme weather conditions are acquired using the virtual camera sensor. Category labels and bounding box annotations for target objects in each of the initial simulation images are generated to obtain an annotated virtual dataset, including: Start the simulation and record simulation videos using the virtual camera sensor under each of the aforementioned extreme weather conditions; Key frames are extracted from the simulation video based on a preset sampling interval, and the key frames are converted into an initial simulation image. The target objects in each of the initial simulation images are labeled to generate a label file containing the category labels and the bounding box labels; the target objects include any one or more of the following: vehicles, motorcycles, pedestrians, bicycles, trucks, and buses; The virtual dataset is obtained by integrating the initial simulation images and their corresponding annotation files.

[0007] Optionally, the step of aligning the category semantic labels of the virtual dataset and the real extreme weather dataset using a label mapping script, and after the labels of the virtual dataset and the real extreme weather dataset are aligned, extracting corresponding simulation images from the virtual dataset based on a preset ratio, and adding the extracted simulation images to the real extreme weather dataset to construct a sample dataset, includes: The label mapping script iterates through the annotation files of each image in the virtual dataset and reads the category labels of each annotation file. According to the preset label mapping relationship, the category labels in the virtual dataset are replaced with target category labels that are consistent with the real extreme weather dataset to obtain the aligned simulation dataset; The target number of simulated images to be extracted is determined based on a preset ratio and the number of real images in the real extreme weather dataset. Randomly select the target number of simulation images from the aligned simulation dataset to generate a candidate dataset; The candidate dataset is fused with the real extreme weather dataset to obtain a sample dataset; Accordingly, after constructing the sample dataset, the process also includes: The sample dataset is divided into training, validation and test sets by stratified sampling based on a preset sample division ratio.

[0008] Optionally, the augmented dataset includes the augmented training set, augmented validation set, and augmented test set of the sample dataset; Accordingly, the step of training an initial detection network based on an object detection loss function using the augmented dataset to obtain an object detection network includes: An initial detection network is constructed, comprising a backbone network, a neck network, and a detection head. The backbone network is used to perceive local details and global information, the neck network is used to transmit semantic information, and the detection head is used to predict the target center and bounding box offset. The initial detection network is trained using the enhanced training set with the target detection loss function as the objective until a preset loss condition or a preset number of iterations is met to obtain the trained detection network. The trained detection network is then validated using the enhanced validation set. The validated detection network is determined based on the validation results. The validated detection network is then tested using the enhanced test set. The target detection network is determined based on the performance conclusions obtained from the tests.

[0009] Optionally, the preset image statistical features include sharpness features, brightness features, contrast features, and saturation features; Accordingly, the step of extracting and normalizing preset image statistical features from the images in the sample dataset through a preset preprocessing module to obtain normalized image statistical features includes: Convert the images in the sample dataset to grayscale; The grayscale image is convolved using the Laplacian operator to determine the corresponding variance. A first ratio of the variance to a preset maximum variance value is determined. Based on the first ratio and a preset limit interval, the normalized sharpness feature is generated. A first average value of grayscale values ​​of all pixels in the grayscale image is determined, and a second ratio of the first average value to a preset grayscale level limit value is determined as the normalized brightness feature. Determine the standard deviation of the pixel grayscale values ​​in the grayscale image, and determine the third ratio of the standard deviation to a preset contrast threshold as the normalized contrast feature; The images in the sample dataset are converted to the HSV color space, and the corresponding saturation channels are extracted. The second average value of all pixel values ​​in the saturation channels is determined as the normalized saturation feature.

[0010] Optionally, the step of determining the degradation type of the image based on the normalized image statistical features according to a preset rule, and performing image enhancement operations on the images in the sample dataset based on the degradation type to obtain an enhanced dataset includes: If the contrast feature, the sharpness feature, and the saturation feature are all below the first threshold, the image degradation type is determined to be a foggy image. Dehazing enhancement based on an atmospheric scattering physics model is performed on the foggy image to generate a dehazed image, and the dehazed image is output to the enhanced dataset. If the brightness feature is lower than the second threshold and the contrast feature is lower than the third threshold, the image degradation type is determined to be a low-light image. A preset low-light enhancement operation is performed on the low-light image to generate an enhanced image, and the enhanced image is output to the enhanced dataset. If the image is not determined to be a foggy image or a low-light image, then the image degradation type is determined to be a normal image, the enhancement process is skipped, and the image is directly output to the enhanced dataset.

[0011] Secondly, this application provides a target object detection device for extreme weather conditions, comprising: The virtual data acquisition module is used to build a target simulation scene in the driving simulation platform and set extreme weather conditions including various visibility parameters. A virtual camera sensor is configured in the target simulation scene, the simulation is run and the initial simulation images under each extreme weather condition are collected through the virtual camera sensor, and the category labels and bounding box annotations of the target objects in each initial simulation image are generated to obtain an annotated virtual dataset. A sample construction module is used to acquire a real extreme weather dataset, align the category semantic labels of the virtual dataset and the real extreme weather dataset using a label mapping script, and after the labels of the virtual dataset and the real extreme weather dataset are aligned, extract corresponding simulation images from the virtual dataset based on a preset ratio, and add the extracted simulation images to the real extreme weather dataset to construct a sample dataset; the preset ratio is the ratio of the number of simulation images from the virtual dataset to the number of real images from the real extreme weather dataset in the sample dataset. The data augmentation module is used to extract and normalize preset image statistical features from the images in the sample dataset through a preset preprocessing module to obtain normalized image statistical features. Based on the normalized image statistical features, the degradation type of the image is determined according to preset rules. Based on the degradation type, image augmentation operation is performed on the images in the sample dataset to obtain an augmented dataset. The target detection module is used to train an initial detection network based on the target detection loss function using the enhanced dataset to obtain the target detection network, acquire the extreme weather image to be detected, extract and enhance the image statistical features of the extreme weather image to be detected through the preset preprocessing module to generate an enhanced image, and then input the enhanced image into the target detection network for processing, and output the bounding box position coordinates and category label of the target object in the extreme weather image to be detected.

[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the target object detection method under extreme weather conditions as described above.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the target object detection method under extreme weather conditions as described above.

[0014] In summary, this application constructs a target simulation scenario in a driving simulation platform and sets extreme weather conditions including various visibility parameters. A virtual camera sensor is configured in the target simulation scenario, and the simulation is run. Initial simulation images under each extreme weather condition are acquired using the virtual camera sensor, and category labels and bounding box annotations for target objects in each initial simulation image are generated to obtain an annotated virtual dataset. A real extreme weather dataset is obtained, and the category semantic labels of the virtual dataset and the real extreme weather dataset are aligned using a label mapping script. After the labels of the virtual dataset and the real extreme weather dataset are aligned, corresponding simulation images are extracted from the virtual dataset based on a preset ratio, and the extracted simulation images are added to the real extreme weather dataset to construct a sample dataset. The preset ratio is the proportion of the sample dataset derived from the actual extreme weather conditions. The virtual dataset contains a simulated image set, which is the ratio of the number of simulated images to the number of real images from the real extreme weather dataset. A pre-processing module extracts and normalizes pre-defined image statistical features from the images in the sample dataset to obtain normalized image statistical features. Based on these normalized features, a pre-defined rule is used to determine the degradation type of the images. Based on the degradation type, image enhancement is performed on the images in the sample dataset to obtain an enhanced dataset. Using the enhanced dataset, an initial detection network is trained based on a target detection loss function to obtain a target detection network. Extreme weather images to be detected are then acquired. The pre-processing module extracts and enhances the image statistical features of the extreme weather images to generate enhanced images. The enhanced images are then input into the target detection network for processing, and the bounding box coordinates and category labels of the target objects in the extreme weather images to be detected are output.

[0015] As described above, this application relies on a driving simulation platform to construct simulation scenarios adapted to extreme weather conditions. It uses a virtual camera to collect simulated images under extreme weather conditions with varying visibility, and completes the labeling of image targets' categories and bounding boxes, constructing a high-quality virtual simulation dataset. To achieve virtual-real data adaptation, this scheme uses a label mapping tool to unify the semantic labels of the virtual dataset and the real extreme weather dataset. Then, it fuses the two types of data according to a fixed ratio of virtual to real images to form the final sample dataset. Subsequently, a preprocessing module extracts statistical features from the sample images and performs normalization processing to determine the image degradation type. Targeted image enhancement optimization is then carried out to effectively improve the quality of the sample images and obtain an enhanced dataset. Finally, based on the enhanced dataset, iterative training of the initial detection network is completed using an object detection loss function to obtain an object detection network adapted to extreme weather scenarios. In practical applications, the collected extreme weather images to be detected undergo the same preprocessing and image enhancement operations as during the training phase. Then, the images are input into the trained object detection network, which can accurately output the category information and bounding box coordinates of the target objects within the image, achieving effective object detection under extreme weather conditions. In this way, by expanding the real extreme weather samples with simulated data and unifying the labels, the data can be fused to narrow the domain gap between simulated and real data. After feature determination and targeted enhancement by the preprocessing module, target detection is performed using a detection network. It can make rule-based judgments entirely based on the traditional statistical features of the image itself. In the degradation detection and enhancement parameter generation stages, no training data or neural network inference is required. It relies only on deterministic mathematical calculations, so it can still work stably under extremely low resolution and extremely small sample conditions. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a target object detection method under extreme weather conditions disclosed in this application; Figure 2 This application discloses a simulation scene rendering of four weather conditions: sunny, foggy, rainy, and snowy. Figure 3 This is a schematic diagram of virtual-real data fusion and dataset partitioning disclosed in this application; Figure 4 This is a detailed flowchart of an adaptive preprocessing module disclosed in this application; Figure 5This is a schematic diagram of the core code of a degradation detection module disclosed in this application; Figure 6 This is a schematic diagram of a target object detection device under extreme weather conditions disclosed in this application; Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Currently, severe or extreme weather conditions can lead to degradation issues in images captured by vehicle-mounted cameras, such as decreased contrast, blurred target edges, and missing texture features, directly reducing the performance of target detection models. YOLOv8, as a mainstream single-stage detection algorithm, performs well in clear weather, but its performance drops significantly in severe weather. Existing target detection methods in severe weather mainly fall into two categories: first, preprocessing the image by dehazing / enhancing it before inputting it into the detection network; second, using a generative adversarial network to synthesize severe weather images and expand the training set. To address the aforementioned technical problems, this application discloses a target object detection method, apparatus, device, and medium under extreme weather conditions, which can improve the recall rate and localization accuracy of target detection under adverse conditions.

[0020] See Figure 1 As shown, this embodiment of the invention discloses a target object detection method under extreme weather conditions, including: Step S11: Build a target simulation scene in the driving simulation platform and set extreme weather conditions including various visibility parameters. Configure a virtual camera sensor in the target simulation scene, run the simulation and collect initial simulation images under each extreme weather condition through the virtual camera sensor, and generate category labels and bounding box annotations for target objects in each initial simulation image to obtain an annotated virtual dataset.

[0021] In this embodiment, a static scene is drawn using a line-drawing tool with grid and grid alignment functions in the driving simulation platform. The static scene includes any one or more of the following: straight and curved road paths, lanes, sidewalks, and static scene objects. Dynamic participants are configured in the static scene to obtain the target simulation scene. The dynamic participants include any one or more of the following: target objects, background objects, and pedestrians. Weather configuration commands sent by the user terminal are obtained through a simulation control protocol, and extreme weather conditions including various visibility parameters are set based on the weather configuration commands. The extreme weather conditions include any one or more of the following: foggy, rainy, and snowy weather. A virtual camera is placed in the target simulation scene, and the horizontal field of view, vertical field of view, and output image resolution of the virtual camera's sensor are configured.

[0022] Specifically, in the static road editing module of the driving simulation platform, a road reference line containing straight sections and curves can be drawn using the line drawing tool. Curves can be created by connecting two perpendicular straight lines using the arc addition function. The total length of the reference line is set to 1000 meters, and a path conversion operation is performed to convert it into a road path. Based on this reference line, a center lane without width attributes is created as the lane center. A dashed center dividing line is assigned to this center lane, and a solid dividing line is added on each side of the reference line. Shoulders and sidewalks are then added respectively. The width of the motor vehicle lanes can be... The width of the sidewalk can be set to 3.5 meters, the road type can be set to urban two-way lane; the road surface material can be set to asphalt, the shoulder surface material can be set to curb type, the sidewalk surface material can be set to cobblestone sidewalk type, and the road surface friction coefficient can be set to the preset default value; further, environmental macros can be added, the environmental macro category can be set to town type, the macro type can be set to preset town size, the range of action can be set to 0 meters to 1000 meters, and the environmental macro can be enabled on both sides of the road; in addition, static scene objects such as houses and trees can be added to complete the construction of static scenes.

[0023] Then, in the dynamic scene editing module of the driving simulation platform, the main vehicle can be configured by setting its name to a preset identifier, its type to a sedan, its initial speed, and its driving mode to autonomous driving mode. Background vehicles can also be configured; multiple background vehicles can be generated using a batch traffic flow generation method. Background vehicle types can include sedans, motorcycles, buses, trucks, and bicycles, with speed ranges set to preset intervals and driving modes set to autonomous driving mode. Pedestrians can also be configured; the number of pedestrians can be set to a preset range, their activity area can be set to crosswalks and roadside areas, and their walking method can be set to walking along a preset path. Through the configuration of these dynamic elements, a target simulation scene containing elements such as roads, vehicles, and pedestrians is obtained.

[0024] Next, weather parameters are dynamically configured through a simulation control protocol. This protocol uses an Extensible Markup Language (EXPLAIN) format to organize commands, which are sent to the driving simulation platform server via a preset communication port. The server's task control process receives these commands and distributes them to the image rendering engine module for real-time rendering updates. Specific weather parameter configurations include setting visibility to a preset range, such as 30 to 80 meters, using a multi-level value approach; setting cloud cover to a preset full cloud state, such as 8 / 8, to simulate a cloudy, foggy environment; setting the road surface friction coefficient and road surface condition parameters to preset dry default values; and controlling the angle and intensity of sunlight using simulation date and time parameters. During actual simulation execution, a sunny scene can be presented by default, and simulation control protocol messages can be sent from the terminal to sequentially switch to extreme weather conditions such as dense fog, heavy rain, and heavy snow, thereby obtaining simulation data for multiple extreme weather conditions within the same target simulation scenario.

[0025] It is important to know that after constructing the target simulation scene, a virtual camera needs to be placed in the target simulation scene. This camera can be set to a preset type, such as 60x40_OpenGL; the horizontal field of view can be set to a preset angle value, such as 60.000 degrees; the vertical field of view can be set to a preset angle value, such as 40.000 degrees; the horizontal and vertical view frustum offsets can be set to preset values, such as 0.000 degrees; the near clipping plane can be set to a preset distance, such as 1.0 meter; the far clipping plane can be set to a preset distance, such as 1500 meters; and the output image resolution can be set to a preset low resolution, such as 60×40 pixels.

[0026] Understandably, after constructing the target simulation scene and configuring the camera, the simulation is started, and simulation videos are recorded using the virtual camera sensor under various extreme weather conditions; keyframes are extracted from the simulation videos based on a preset sampling interval, and the keyframes are converted into initial simulation images; target objects in each of the initial simulation images are labeled to generate a labeling file containing the category labels and the bounding box labels; the target objects include any one or more of vehicles, motorcycles, pedestrians, bicycles, trucks, and buses; and the initial simulation images and corresponding labeling files are integrated to obtain the virtual dataset.

[0027] Specifically, the camera sensor is activated, and the target simulation scene is run. Video recording is initiated sequentially under various extreme weather conditions, such as fog, rain, and snow, to collect simulated video data for each weather condition. Keyframes are extracted from the recorded simulation video at preset sampling intervals and converted into single-frame initial simulation images. Annotation tools are used to manually annotate the target objects in the initial simulation images frame by frame. Annotation categories can include vehicles, motorcycles, pedestrians, bicycles, trucks, buses, etc., and a YOLO format annotation file is generated.

[0028] Further, see Figure 2 As shown, for ease of understanding, this application also provides schematic diagrams of four extreme weather simulation scenarios: sunny, foggy, rainy, and snowy. Figure 3 As shown, in sunny scenes, the road surface and vehicle outlines are clear, and the color saturation is normal. In foggy scenes, distant buildings, trees, and vehicle outlines are obscured by varying degrees of fog, significantly reducing the overall contrast and saturation of the image. In rainy scenes, the road surface exhibits a slippery and reflective effect, accompanied by rain streaks, resulting in a decrease in image brightness and clarity. In snowy scenes, numerous snow particles obscure the image, and the road surface shows snow accumulation, significantly reducing the discernibility of target edges. By switching between these different extreme weather conditions within the same target simulation scene, multiple extreme weather simulation images can be generated in batches based on the same set of static scenes and dynamic participant configurations. These images are physically reliable, consistent in scene, and differ only in weather conditions. This significantly reduces data acquisition costs. Furthermore, relying on the precise 3D position information of the driving simulation platform itself, high-precision target bounding box annotations are automatically generated, avoiding the problem of large errors in manual annotation under low visibility conditions.

[0029] Step S12: Obtain a real extreme weather dataset. Align the category semantic labels of the virtual dataset and the real extreme weather dataset using a label mapping script. After the labels of the virtual dataset and the real extreme weather dataset are aligned, extract corresponding simulation images from the virtual dataset based on a preset ratio and add the extracted simulation images to the real extreme weather dataset to construct a sample dataset. The preset ratio is the ratio of the number of simulation images from the virtual dataset to the number of real images from the real extreme weather dataset in the sample dataset.

[0030] In this embodiment, an open-source dataset is first used to obtain a real extreme weather dataset containing various extreme weather conditions. For example, the real extreme weather dataset may contain 300 real traffic scene foggy images. Each image is manually labeled, and the labeling categories can cover six types of targets: pedestrian, bicycle, car, motorcycle, bus, and truck, which correspond to the labeling categories of the virtual dataset.

[0031] Furthermore, the label mapping script iterates through the annotation files of each image in the virtual dataset and reads the category labels of each annotation file; according to the preset label mapping relationship, the category labels in the virtual dataset are replaced with target category labels consistent with the real extreme weather dataset to obtain an aligned simulation dataset; based on a preset ratio and the number of real images in the real extreme weather dataset, a target number of simulation images for extraction is determined; the target number of simulation images are randomly extracted from the aligned simulation dataset to generate a candidate dataset; the candidate dataset is fused with the real extreme weather dataset to obtain a sample dataset.

[0032] Specifically, the label mapping script iterates through the annotation files of each image in the virtual dataset, reading the category labels of each annotation file. Since the order of category labels used in constructing the annotation files in the virtual dataset and the real extreme weather dataset may differ, the label mapping script needs to read the original category labels recorded in the first column of each annotation file in the virtual dataset. Based on a preset label mapping relationship, the category labels in the virtual dataset are replaced with target category labels consistent with those in the real extreme weather dataset. The semantic label mapping relationship between the real extreme weather dataset (DAWN) and the virtual dataset (vtdfog) is shown in Table 1. Table 1

[0033] Traverse each annotation file in the virtual dataset, read the original category label at the start position of each annotation record, replace it with the target category label consistent with the real extreme weather dataset according to the above mapping table, and write the replaced annotation result back to the annotation file to obtain the aligned simulation dataset after category semantic label alignment.

[0034] Then, as Figure 3As shown, assuming complete backups of both the real extreme weather dataset and the virtual dataset, the fusion ratio of the virtual and real images is set to approximately 3:1. This means the number of simulated images extracted from the virtual dataset is about one-third the number of images in the real dataset. For example, if the real extreme weather dataset contains 300 images, the target number of simulated images to be extracted can be determined to be 100. It is understood that this application does not impose a specific fusion ratio; this ratio can be flexibly adjusted according to the actual number of real samples. For example, when real samples are plentiful, the fusion ratio of the virtual data can be appropriately reduced, while when real samples are extremely scarce, the fusion ratio of the virtual data can be appropriately increased.

[0035] Accordingly, after constructing the sample dataset, the method further includes: dividing the sample dataset into training, validation, and test sets using stratified sampling based on a preset sample division ratio. Specifically, the stratified sampling strategy ensures that the number of target boxes for each category is evenly distributed across the training, validation, and test sets, avoiding model training bias due to uneven category distribution. The real extreme weather dataset can be divided into a training part (210 images), a validation part (60 images), and a test part (30 images) in a 7:2:1 ratio. It is important to note that the validation and test sets are consistent with the real data benchmark; only the training set introduces virtual simulation data. This ensures that subsequent improvements in object detection network performance can be directly attributed to the introduction of virtual simulation data, facilitating an objective evaluation of the effectiveness of the method. Specifically, all 100 extracted virtual simulation images are incorporated into the training part, resulting in a final training set containing 310 images (210 real images and 100 virtual images), a validation set containing 60 real images, and a test set containing 30 real images.

[0036] Step S13: Extract preset image statistical features from the images in the sample dataset using a preset preprocessing module and normalize them to obtain normalized image statistical features. Determine the degradation type of the image based on the normalized image statistical features and a preset rule. Perform image enhancement operation on the images in the sample dataset based on the degradation type to obtain the enhanced dataset.

[0037] In this embodiment, as Figure 4As shown, in order to extract and normalize the preset image statistical features, the images in the sample dataset need to be converted into grayscale images; the Laplacian operator is used to perform convolution operations on the grayscale images and the corresponding variance is determined; a first ratio of the variance to a preset maximum variance value is determined; based on the first ratio and a preset limit interval, the normalized sharpness feature is generated; a first average value of all pixel grayscale values ​​in the grayscale image is determined, and a second ratio of the first average value to a preset grayscale level limit value is determined as the normalized brightness feature; the standard deviation of pixel grayscale values ​​in the grayscale image is determined, and a third ratio of the standard deviation to a preset contrast threshold is determined as the normalized contrast feature; the images in the sample dataset are converted to the HSV color space, and the corresponding saturation channels are extracted; the second average value of all pixel values ​​in the saturation channels is determined as the normalized saturation feature.

[0038] Specifically, the degradation detection module converts the input image into a grayscale image and an HSV image, respectively, as the basis for subsequent feature calculations. The grayscale image is convolved using the Laplacian operator to determine the corresponding variance. As a second-order differential operator, the Laplacian operator is sensitive to edges and high-frequency components in the image: when the image edges are sharp, the Laplacian response value is large, and the corresponding variance value is high; when the image is blurry, the edge sharpness decreases, the Laplacian response value approaches zero, and the corresponding variance value is low. A first ratio of the variance to a preset maximum variance value is determined. Based on the first ratio and the normalized [0,1] interval, a normalized sharpness feature is generated through truncation. The core code of the degradation detection module is as follows: Figure 5 As shown.

[0039] Determine the average value of all pixels in the grayscale image, that is, calculate the arithmetic mean of all pixels in the image, and determine the ratio of the average value to the preset grayscale level limit value. For example, the grayscale level limit can be set to 255, thereby obtaining the brightness feature normalized to the [0,1] interval. The lower the value, the darker the image.

[0040] Determine the standard deviation of the pixel grayscale values ​​in the grayscale image; and determine the third ratio of the standard deviation to the preset contrast threshold. The preset contrast threshold can be set to 128, which is half of the grayscale range of 255. In another possible implementation, the preset contrast threshold can also be adjusted to other empirical values, such as 80, according to the statistical distribution of the actual image data, so as to obtain the contrast features normalized to the [0,1] interval. The lower the value, the more gray the image and the worse the sense of layering between the target and the background.

[0041] The images in the sample dataset are converted to the HSV color space, and the corresponding saturation channel, i.e., the S channel, is extracted. The average value of all pixel values ​​in the saturation channel is determined as the normalized saturation feature. The lower this value, the darker the image color, which is also one of the typical characteristics of foggy images. It can be understood that since the value range of the saturation channel in the HSV color space is usually 0 to 255, the average value can be further divided by 255 to complete the normalization process, resulting in a saturation feature with a value range of [0,1].

[0042] In this embodiment, after determining the preset image statistical features, namely sharpness, brightness, contrast, and saturation, in a specific embodiment, if the contrast, sharpness, and saturation features are all below a first threshold, the image degradation type is determined to be a foggy image. Dehazing enhancement based on an atmospheric scattering physics model is then performed on the foggy image to generate a dehazed image, which is then output to the enhanced dataset. Specifically, if the contrast is below 0.4, the sharpness is below 0.4, and the saturation is below 0.4, it is determined to be a foggy image, and dehazing enhancement based on an atmospheric scattering physics model is performed. For foggy images, this invention uses a well-known atmospheric scattering physics model as the theoretical basis for dehazing. This model represents a foggy image as follows: ; Where J(x) represents the clear image, A is the global atmospheric light value, and t(x) is the transmittance map.

[0043] Transmittance maps are estimated using dark channel priors. According to dark channel prior theory, transmittance is estimated using the following formula: ; Where ω is the dehazing control parameter, which is set to 0.95; Ω(x) is the local window; and C is the RGB three-channel.

[0044] After obtaining the transmittance map, a clear image is recovered through inverse operations using a physical model: ; Here, t0=0.1 is the lower limit value to prevent excessively low transmittance. Finally, the dehazed image is output to the enhanced dataset.

[0045] In another specific embodiment, if the brightness feature is below a second threshold and the contrast feature is below a third threshold, the image is determined to be a low-light image. A preset low-light enhancement operation is performed on the low-light image to generate an enhanced image, and the enhanced image is output to the enhanced dataset. Specifically, if the brightness is below 0.35 and the contrast is below 0.5, it is determined to be a low-light image, and LAB space CLAHE enhancement is performed. For low-light images, the LAB color space CLAHE (Contrast Limited Adaptive Histogram Equalization) enhancement algorithm is used. After converting the image to LAB space, only the brightness channel (L channel) is subjected to contrast limited adaptive histogram equalization, while the A and B channels (color channels) remain unchanged. Finally, it is converted back to BGR space to obtain the enhanced image, which is then output to the enhanced dataset.

[0046] In a third specific embodiment, if the image is not determined to be a foggy image or a low-light image, then the image degradation type is determined to be a normal image, the enhancement process is skipped, and it is directly output to the enhanced dataset. Specifically, if none of the above conditions are met, it is determined to be a normal image, the original image is directly output, and all enhancement processes are skipped.

[0047] Step S14: Using the enhanced dataset, train the initial detection network based on the target detection loss function to obtain the target detection network, acquire the extreme weather image to be detected, extract and enhance the image statistical features of the extreme weather image to be detected through the preset preprocessing module to generate the enhanced image, and then input the enhanced image into the target detection network for processing, and output the bounding box position coordinates and category label of the target object in the extreme weather image to be detected.

[0048] In this embodiment, an initial detection network is first constructed. This initial detection network includes a backbone network, a neck network, and a detection head. The backbone network is used to perceive local details and global information, the neck network is used to transmit semantic information, and the detection head is used to predict the target center and bounding box offset. Specifically, the initial detection network can use the YOLOv8 detection network. Its network structure can include: the backbone network can adopt an improved CSPDarknet structure, where the C3 module in YOLOv5 is replaced with a C2f module, dividing the input feature map into several branches. Each branch undergoes independent convolution transformation before merging the output, thereby strengthening the gradient flow path and mitigating the gradient vanishing phenomenon in deep networks; the end of the backbone network can use a Spatial Pyramid Pooling (SPPF) module, which generates multi-scale feature maps through three consecutive max pooling operations and then concatenates them for output, thereby simultaneously perceiving local details and global semantic information of the image. The neck network can employ a Path Aggregation Network (PANet) structure, achieving full interaction between multi-scale feature maps through bidirectional feature fusion paths: the top-down path transmits high-level semantic information to low-level feature maps to enhance the semantic expressiveness of low-level features; the bottom-up path transmits low-level detail information such as edges and textures to high-level feature maps to compensate for spatial information loss during downsampling. The detection head can adopt an anchor-free design and a decoupled structure: the anchor-free design directly predicts the target center position and bounding box offset, simplifying the tuning of anchor parameters and eliminating performance loss caused by mismatch between preset anchors and the actual target size distribution; the decoupled structure separates the target classification task from the bounding box regression task, using independent convolutional branches for prediction, thereby improving detection accuracy. It is important to note that a small-sized (Small) pre-trained detection model with approximately 11.2M parameters can achieve a good balance between detection accuracy and inference speed, and is less prone to overfitting under small-sample training conditions.

[0049] Furthermore, the enhanced dataset includes an enhanced training set, an enhanced validation set, and an enhanced test set of the sample dataset. The initial detection network is trained using the enhanced training set with the target detection loss function as the objective until a preset loss condition or a preset number of iterations is met to obtain a trained detection network. The trained detection network is validated using the enhanced validation set, and a validated detection network is determined based on the validation results. The validated detection network is tested using the enhanced test set, and the target detection network is determined based on the performance conclusions obtained from the tests.

[0050] Specifically, the initial detection network is first trained using the augmented training set, with the following training configuration: (1) Pre-trained model: The weights were pre-trained using the official yolov8s.pt file; (2) Configuration file: Create a configuration file based on the yaml format of the COCO dataset, set the number of categories (nc) to 6, and the class names are car, motorcycle, person, bicycle, truck, bus; (3) Optimizer: The Adam optimizer is used, and the initial learning rate is set to 1×10. -4 , ; (4) Batch size: set to 16, number of worker threads: set to 2; (5) Number of training rounds: Try 50 rounds first, then 100 rounds, and if the verification indicator shows an upward trend, increase to 150 rounds; (6) Early stopping mechanism: Enable the early stopping mechanism (patience=10), that is, automatically stop training when the validation set index does not improve after 10 consecutive rounds to prevent overfitting of small samples and save only the best model file; (7) Loss function: The total loss function consists of three parts: bounding box regression loss, classification loss, and distribution focusing loss. The total loss function is a weighted sum of the three factors: ; The bounding box regression loss uses the CIoU loss function, which comprehensively considers the overlap area between the predicted box and the ground truth box, the distance between the center points, and the consistency of the aspect ratio, based on the traditional IoU loss. ; Where ρ represents the center point b of the predicted bounding box and the center point b of the ground truth bounding box. gt The Euclidean distance between them is given by c, where c is the diagonal length of the minimum closure region, v is the aspect ratio similarity, and α is the balancing weight coefficient.

[0051] The classification loss uses binary cross-entropy loss (BCE Loss), while the distribution focusing loss is used to optimize the bounding box prediction probability distribution and improve positioning accuracy.

[0052] Then, the trained detection network is obtained. The trained detection network is then validated using a post-amplified validation set to see if it meets the training accuracy requirements. If it does, the validated detection network is confirmed. The validated detection network is then evaluated using a post-amplified test set. The model performance is evaluated using the following metrics: (1) Precision: The proportion of correctly predicted positive samples out of all predicted positive samples. ; Where TP represents the number of correctly detected positive samples and FP represents the number of incorrectly detected negative samples.

[0053] (2) Recall: The proportion of correctly predicted positive samples out of all true samples: ; Where FN is the number of positive samples that were missed.

[0054] (3) Average Precision (AP): Plot a PR curve with recall on the x-axis and precision on the y-axis. The area under the curve is the average precision. ; (4) Mean Precision (mAP): The arithmetic mean of AP for all categories: ; Where N is the total number of categories.

[0055] It is important to know that the mean precision (mAP50-95) and recall rate within the intersection-union ratio threshold range of 0.5 to 0.95 can be the key indicators for evaluating the effectiveness of the method in this application. Finally, the results should be determined based on the performance evaluation results.

[0056] Therefore, when acquiring extreme weather images to be detected, which may contain complex meteorological interference such as rain, snow, fog, haze, strong light, or low illumination, a pre-processing module extracts statistical features from the image. By analyzing low-level visual information such as brightness distribution, contrast, gradient magnitude, and noise statistics, the degree of degradation is quantified, and enhancement operations are adaptively performed accordingly. For example, histogram equalization, dehazing algorithms, or Retinex-based illumination correction are used to restore texture and edge details obscured by weather, thereby generating an enhanced image. The enhanced image is then fed into a pre-trained object detection network. The network performs dense predictions or region proposals on the feature map through multi-layer convolution and attention mechanisms, ultimately outputting the bounding box coordinates of each detected object and its corresponding category label, such as pedestrian, vehicle, or traffic sign.

[0057] As described above, this application relies on a driving simulation platform to construct simulation scenarios adapted to extreme weather conditions. It uses a virtual camera to collect simulated images under extreme weather conditions with varying visibility, and completes the labeling of image targets' categories and bounding boxes, constructing a high-quality virtual simulation dataset. To achieve virtual-real data adaptation, this scheme uses a label mapping tool to unify the semantic labels of the virtual dataset and the real extreme weather dataset. Then, it fuses the two types of data according to a fixed ratio of virtual to real images to form the final sample dataset. Subsequently, a preprocessing module extracts statistical features from the sample images and performs normalization processing to determine the image degradation type. Targeted image enhancement optimization is then carried out to effectively improve the quality of the sample images and obtain an enhanced dataset. Finally, based on the enhanced dataset, iterative training of the initial detection network is completed using an object detection loss function to obtain an object detection network adapted to extreme weather scenarios. In practical applications, the collected extreme weather images to be detected undergo the same preprocessing and image enhancement operations as during the training phase. Then, the images are input into the trained object detection network, which can accurately output the category information and bounding box coordinates of the target objects within the image, achieving effective object detection under extreme weather conditions. In this way, by expanding the real extreme weather samples with simulated data and unifying the labels, the data can be fused to narrow the domain gap between simulated and real data. After feature determination and targeted enhancement by the preprocessing module, target detection is performed using a detection network. It can make rule-based judgments entirely based on the traditional statistical features of the image itself. In the degradation detection and enhancement parameter generation stages, no training data or neural network inference is required. It relies only on deterministic mathematical calculations, so it can still work stably under extremely low resolution and extremely small sample conditions.

[0058] See Figure 6 As shown, this embodiment of the invention discloses a target object detection device under extreme weather conditions, comprising: The virtual data acquisition module 11 is used to build a target simulation scene in the driving simulation platform and set extreme weather conditions including various visibility parameters. A virtual camera sensor is configured in the target simulation scene, the simulation is run and the initial simulation images under each extreme weather condition are collected through the virtual camera sensor, and the category labels and bounding box annotations of the target objects in each initial simulation image are generated to obtain an annotated virtual dataset. The sample construction module 12 is used to acquire a real extreme weather dataset, perform category semantic label alignment between the virtual dataset and the real extreme weather dataset using a label mapping script, and after the labels of the virtual dataset and the real extreme weather dataset are aligned, extract corresponding simulation images from the virtual dataset based on a preset ratio, and add the extracted simulation images to the real extreme weather dataset to construct a sample dataset; the preset ratio is the ratio of the number of simulation images from the virtual dataset to the number of real images from the real extreme weather dataset in the sample dataset. Data augmentation module 13 is used to extract and normalize preset image statistical features from the images in the sample dataset through a preset preprocessing module to obtain normalized image statistical features, determine the degradation type of the image based on the normalized image statistical features and a preset rule, and perform image augmentation operation on the images in the sample dataset based on the degradation type to obtain an augmented dataset. The target detection module 14 is used to train an initial detection network based on the target detection loss function using the enhanced dataset to obtain a target detection network, acquire the extreme weather image to be detected, extract and enhance the image statistical features of the extreme weather image to be detected through the preset preprocessing module to generate an enhanced image, and then input the enhanced image into the target detection network for processing, and output the bounding box position coordinates and category label of the target object in the extreme weather image to be detected.

[0059] As described above, this application relies on a driving simulation platform to construct simulation scenarios adapted to extreme weather conditions. It uses a virtual camera to collect simulated images under extreme weather conditions with varying visibility, and completes the labeling of image targets' categories and bounding boxes, constructing a high-quality virtual simulation dataset. To achieve virtual-real data adaptation, this scheme uses a label mapping tool to unify the semantic labels of the virtual dataset and the real extreme weather dataset. Then, it fuses the two types of data according to a fixed ratio of virtual to real images to form the final sample dataset. Subsequently, a preprocessing module extracts statistical features from the sample images and performs normalization processing to determine the image degradation type. Targeted image enhancement optimization is then carried out to effectively improve the quality of the sample images and obtain an enhanced dataset. Finally, based on the enhanced dataset, iterative training of the initial detection network is completed using an object detection loss function to obtain an object detection network adapted to extreme weather scenarios. In practical applications, the collected extreme weather images to be detected undergo the same preprocessing and image enhancement operations as during the training phase. Then, the images are input into the trained object detection network, which can accurately output the category information and bounding box coordinates of the target objects within the image, achieving effective object detection under extreme weather conditions. In this way, by expanding the real extreme weather samples with simulated data and unifying the labels, the data can be fused to narrow the domain gap between simulated and real data. After feature determination and targeted enhancement by the preprocessing module, target detection is performed using a detection network. It can make rule-based judgments entirely based on the traditional statistical features of the image itself. In the degradation detection and enhancement parameter generation stages, no training data or neural network inference is required. It relies only on deterministic mathematical calculations, so it can still work stably under extremely low resolution and extremely small sample conditions.

[0060] In some specific implementations, the virtual data acquisition module 11 may specifically include: The scene drawing unit is used to draw static scenes using the line drawing tool with grid and grid alignment functions enabled in the driving simulation platform; the static scene includes any one or more of the following: straight and curved road paths, lanes, sidewalks, and static scene objects. A scene acquisition unit is used to configure dynamic participants in the static scene to obtain the target simulation scene; the dynamic participants include any one or more of the target object, background object, and pedestrians; The condition setting unit is used to obtain the weather configuration command sent by the user terminal through the simulation control protocol, and set the extreme weather conditions including various visibility parameters based on the weather configuration command; the extreme weather conditions include any one or more of foggy days, rainy days and snowy days; A camera configuration unit is used to place a virtual camera in the target simulation scene and configure the horizontal field of view, vertical field of view, and output image resolution of the virtual camera's sensor.

[0061] In some specific implementations, the virtual data acquisition module 11 may specifically include: A video recording unit is used to start the simulation and record simulation videos using the virtual camera sensor under each of the extreme weather conditions. An image unit is used to extract key frames from the simulation video based on a preset sampling interval and convert the key frames into an initial simulation image. The file generation unit is used to annotate the target objects in each of the initial simulation images and generate an annotation file containing the category labels and the bounding box annotations; the target objects include any one or more of the following: vehicles, motorcycles, pedestrians, bicycles, trucks, and buses; The first dataset acquisition unit is used to integrate the initial simulation images and their corresponding annotation files to obtain the virtual dataset.

[0062] In some specific implementations, the sample construction module 12 may specifically include: The label reading unit is used to traverse the annotation files of each image in the virtual dataset through the label mapping script and read the category labels of each annotation file; The label alignment unit is used to replace the category labels in the virtual dataset with target category labels that are consistent with the real extreme weather dataset according to the preset label mapping relationship, so as to obtain the aligned simulation dataset. The quantity determination unit is used to determine the target number of simulated images to be extracted based on a preset ratio and the number of real images in the real extreme weather dataset. A dataset generation unit is used to randomly extract the target number of simulation images from the aligned simulation dataset to generate a candidate dataset; The second dataset acquisition unit is used to fuse the candidate dataset with the real extreme weather dataset to obtain a sample dataset; Accordingly, after constructing the sample dataset, the process may further include: The dataset partitioning unit is used to partition the sample dataset using stratified sampling based on a preset sample partitioning ratio to obtain corresponding training set, validation set and test set.

[0063] In some specific implementations, the augmented dataset includes an augmented training set, an augmented validation set, and an augmented test set of the sample dataset; correspondingly, the object detection module 14 may specifically include: A network construction unit is used to construct an initial detection network. The initial detection network includes a backbone network, a neck network, and a detection head. The backbone network is used to sense local details and global information, the neck network is used to transmit semantic information, and the detection head is used to predict the target center and bounding box offset. The network determination unit is used to train the initial detection network with the target detection loss function as the target using the enhanced training set until a preset loss condition or a preset number of iterations is met to obtain a trained detection network, to verify the trained detection network using the enhanced validation set, to determine the validated detection network based on the validation results, to test the validated detection network using the enhanced test set, and to determine the target detection network based on the performance conclusions obtained from the test.

[0064] In some specific implementations, the preset image statistical features include sharpness features, brightness features, contrast features, and saturation features; correspondingly, the data enhancement module 13 may specifically include: An image conversion unit is used to convert images in the sample dataset into grayscale images; The first feature generation unit is used to perform convolution operation on the grayscale image using the Laplacian operator and determine the corresponding variance, determine a first ratio of the variance to a preset maximum variance value, and generate the normalized sharpness feature based on the first ratio and a preset limit interval. The second feature generation unit is used to determine the first average value of the grayscale values ​​of all pixels in the grayscale image, and to determine the second ratio of the first average value to the preset grayscale level limit value, as the normalized brightness feature. The third feature generation unit is used to determine the standard deviation of the pixel grayscale values ​​in the grayscale image, and to determine the third ratio of the standard deviation to the preset contrast threshold as the normalized contrast feature. The fourth feature generation unit is used to convert the images in the sample dataset to the HSV color space, extract the corresponding saturation channels, and determine the second average value of all pixel values ​​in the saturation channels as the normalized saturation feature.

[0065] In some specific implementations, the data enhancement module 13 may specifically include: The image dehazing unit is used to determine that the image degradation type is a foggy image if the contrast feature, the sharpness feature and the saturation feature are all lower than a first threshold, perform dehazing enhancement on the foggy image based on an atmospheric scattering physical model to generate a dehazed image, and output the dehazed image to the enhanced dataset. An image enhancement unit is configured to determine that the degradation type of the image is a low-light image if the brightness feature is lower than a second threshold and the contrast feature is lower than a third threshold, perform a preset low-light enhancement operation on the low-light image to generate an enhanced image, and output the enhanced image to the enhanced dataset. The dataset output unit is used to determine that if the image is not determined to be a foggy image or a low-light image, the degradation type of the image is a normal image, skip the enhancement process, and directly output it to the enhanced dataset.

[0066] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the target object detection method under extreme weather conditions disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0067] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0068] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0069] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the target object detection method under extreme weather conditions disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0070] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned target object detection method under extreme weather conditions. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0071] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0072] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0073] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0074] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A target object detection method under extreme weather conditions, characterized in that, include: A target simulation scenario is built in a driving simulation platform, and extreme weather conditions including various visibility parameters are set. A virtual camera sensor is configured in the target simulation scenario, the simulation is run, and the initial simulation images under each extreme weather condition are collected through the virtual camera sensor. The category labels and bounding box annotations of the target objects in each initial simulation image are generated to obtain an annotated virtual dataset. A real extreme weather dataset is obtained. The virtual dataset and the real extreme weather dataset are aligned with category semantic labels using a label mapping script. After the labels of the virtual dataset and the real extreme weather dataset are aligned, corresponding simulation images are extracted from the virtual dataset based on a preset ratio. The extracted simulation images are then added to the real extreme weather dataset to construct a sample dataset. The preset ratio is the ratio of the number of simulated images from the virtual dataset to the number of real images from the real extreme weather dataset in the sample dataset; The image statistical features in the sample dataset are extracted and normalized by a preprocessing module to obtain normalized image statistical features. Based on the normalized image statistical features, the degradation type of the image is determined according to a pre-defined rule. Based on the degradation type, the image in the sample dataset is enhanced to obtain an enhanced dataset. Using the enhanced dataset, an initial detection network is trained based on the target detection loss function to obtain a target detection network. The extreme weather image to be detected is then acquired. The image statistical features of the extreme weather image to be detected are extracted and enhanced by the preset preprocessing module to generate an enhanced image. The enhanced image is then input into the target detection network for processing, and the bounding box coordinates and category labels of the target objects in the extreme weather image to be detected are output.

2. The target object detection method under extreme weather conditions according to claim 1, characterized in that, The process involves building a target simulation scenario within a driving simulation platform and setting extreme weather conditions including various visibility parameters. Virtual camera sensors are then configured within the target simulation scenario, including: In the driving simulation platform, use the line drawing tool with grid and grid alignment functions to draw static scenes; the static scenes include any one or more of the following: straight and curved road paths, lanes, sidewalks, and static scene objects. The target simulation scene is obtained by configuring dynamic participants in the static scene; the dynamic participants include any one or more of the target object, background object, and pedestrians. The weather configuration command sent by the user terminal is obtained through the simulation control protocol, and the extreme weather conditions including various visibility parameters are set based on the weather configuration command; the extreme weather conditions include any one or more of foggy days, rainy days and snowy days; A virtual camera is placed in the target simulation scene, and the horizontal field of view, vertical field of view, and output image resolution of the virtual camera's sensor are configured.

3. The target object detection method under extreme weather conditions according to claim 1, characterized in that, The simulation is run and initial simulation images under various extreme weather conditions are acquired using the virtual camera sensor. Category labels and bounding box annotations for target objects in each initial simulation image are generated to obtain an annotated virtual dataset, including: Start the simulation and record simulation videos using the virtual camera sensor under each of the aforementioned extreme weather conditions; Key frames are extracted from the simulation video based on a preset sampling interval, and the key frames are converted into an initial simulation image. The target objects in each of the initial simulation images are labeled to generate a label file containing the category labels and the bounding box labels; the target objects include any one or more of the following: vehicles, motorcycles, pedestrians, bicycles, trucks, and buses; The virtual dataset is obtained by integrating the initial simulation images and their corresponding annotation files.

4. The target object detection method under extreme weather conditions according to claim 3, characterized in that, The process involves aligning the category semantic labels of the virtual dataset and the real extreme weather dataset using a label mapping script. After the labels of the virtual dataset and the real extreme weather dataset are aligned, corresponding simulation images are extracted from the virtual dataset based on a preset ratio, and the extracted simulation images are added to the real extreme weather dataset to construct a sample dataset, including: The label mapping script iterates through the annotation files of each image in the virtual dataset and reads the category labels of each annotation file. According to the preset label mapping relationship, the category labels in the virtual dataset are replaced with target category labels that are consistent with the real extreme weather dataset to obtain the aligned simulation dataset; The target number of simulated images to be extracted is determined based on a preset ratio and the number of real images in the real extreme weather dataset. Randomly select the target number of simulation images from the aligned simulation dataset to generate a candidate dataset; The candidate dataset is fused with the real extreme weather dataset to obtain a sample dataset; Accordingly, after constructing the sample dataset, the process also includes: The sample dataset is divided into training, validation and test sets by stratified sampling based on a preset sample division ratio.

5. The target object detection method under extreme weather conditions according to claim 4, characterized in that, The augmented dataset includes the augmented training set, augmented validation set, and augmented test set of the sample dataset; Accordingly, the step of training an initial detection network based on an object detection loss function using the augmented dataset to obtain an object detection network includes: An initial detection network is constructed, comprising a backbone network, a neck network, and a detection head. The backbone network is used to perceive local details and global information, the neck network is used to transmit semantic information, and the detection head is used to predict the target center and bounding box offset. The initial detection network is trained using the enhanced training set with the target detection loss function as the objective until a preset loss condition or a preset number of iterations is met to obtain the trained detection network. The trained detection network is then validated using the enhanced validation set. The validated detection network is determined based on the validation results. The validated detection network is then tested using the enhanced test set. The target detection network is determined based on the performance conclusions obtained from the tests.

6. The target object detection method under extreme weather conditions according to any one of claims 1 to 5, characterized in that, The preset image statistical features include sharpness features, brightness features, contrast features, and saturation features; Accordingly, the step of extracting and normalizing preset image statistical features from the images in the sample dataset through a preset preprocessing module to obtain normalized image statistical features includes: Convert the images in the sample dataset to grayscale; The grayscale image is convolved using the Laplacian operator to determine the corresponding variance. A first ratio of the variance to a preset maximum variance value is determined. Based on the first ratio and a preset limit interval, the normalized sharpness feature is generated. A first average value of grayscale values ​​of all pixels in the grayscale image is determined, and a second ratio of the first average value to a preset grayscale level limit value is determined as the normalized brightness feature. Determine the standard deviation of the pixel grayscale values ​​in the grayscale image, and determine the third ratio of the standard deviation to a preset contrast threshold as the normalized contrast feature; The images in the sample dataset are converted to the HSV color space, and the corresponding saturation channels are extracted. The second average value of all pixel values ​​in the saturation channels is determined as the normalized saturation feature.

7. The target object detection method under extreme weather conditions according to claim 6, characterized in that, The process involves determining the degradation type of an image based on the normalized image statistical features using preset rules, and then performing image enhancement operations on the images in the sample dataset based on the degradation type to obtain an enhanced dataset, including: If the contrast feature, the sharpness feature, and the saturation feature are all below the first threshold, the image degradation type is determined to be a foggy image. Dehazing enhancement based on an atmospheric scattering physics model is performed on the foggy image to generate a dehazed image, and the dehazed image is output to the enhanced dataset. If the brightness feature is lower than the second threshold and the contrast feature is lower than the third threshold, the image degradation type is determined to be a low-light image. A preset low-light enhancement operation is performed on the low-light image to generate an enhanced image, and the enhanced image is output to the enhanced dataset. If the image is not determined to be a foggy image or a low-light image, then the image degradation type is determined to be a normal image, the enhancement process is skipped, and the image is directly output to the enhanced dataset.

8. A target object detection device for extreme weather conditions, characterized in that, include: The virtual data acquisition module is used to build a target simulation scene in the driving simulation platform and set extreme weather conditions including various visibility parameters. A virtual camera sensor is configured in the target simulation scene, the simulation is run and the initial simulation images under each extreme weather condition are collected through the virtual camera sensor, and the category labels and bounding box annotations of the target objects in each initial simulation image are generated to obtain an annotated virtual dataset. The sample construction module is used to acquire a real extreme weather dataset, perform category semantic label alignment between the virtual dataset and the real extreme weather dataset through a label mapping script, and after the labels of the virtual dataset and the real extreme weather dataset are aligned, extract corresponding simulation images from the virtual dataset based on a preset ratio and add the extracted simulation images to the real extreme weather dataset to construct a sample dataset. The preset ratio is the ratio of the number of simulated images from the virtual dataset to the number of real images from the real extreme weather dataset in the sample dataset; The data augmentation module is used to extract and normalize preset image statistical features from the images in the sample dataset through a preset preprocessing module to obtain normalized image statistical features. Based on the normalized image statistical features, the degradation type of the image is determined according to preset rules. Based on the degradation type, image augmentation operation is performed on the images in the sample dataset to obtain an augmented dataset. The target detection module is used to train an initial detection network based on the target detection loss function using the enhanced dataset to obtain the target detection network, acquire the extreme weather image to be detected, extract and enhance the image statistical features of the extreme weather image to be detected through the preset preprocessing module to generate an enhanced image, and then input the enhanced image into the target detection network for processing, and output the bounding box position coordinates and category label of the target object in the extreme weather image to be detected.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the target object detection method under extreme weather conditions as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the target object detection method under extreme weather conditions as described in any one of claims 1 to 7.

Images