Target identification method and device based on black light camera, electronic equipment and medium

Abstract

The invention relates to a target identification method and device based on a black light camera, equipment and a medium. The method comprises the steps of obtaining a to-be-recognized image; the to-be-recognized image is an image which is collected by a black light camera and contains a driving scene; inputting the to-be-recognized image into a trained target recognition model, so that the target recognition model judges whether the to-be-recognized image needs to be defogged or not through a defogging judgment network; if the to-be-recognized image needs to be defogged, outputting the to-be-recognized image to a defogging parameter prediction network of the target recognition model for defogging, and outputting a defogged image of the to-be-recognized image; and inputting a defogged image of the to-be-recognized image into a target detection network of the target recognition model, so that the target recognition model performs target detection on the defogged image through the target detection network, and outputting a target category and a target position of the to-be-recognized image.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a target recognition method, device, electronic device and medium based on a black light camera. Background Art

[0002] Target recognition is a technology in the field of computer vision. This technology enables high-performance processing devices such as on-board computers to obtain surrounding environment information through environmental acquisition devices such as cameras, process this information fully autonomously, and ultimately provide feedback on specific information about specific types of targets in the surrounding environment.

[0003] In the related technology, the existing target recognition technology in driving scenes is often mainly used in urban driving scenes, which often rely on sunlight, city lights and driving lights to construct a relatively well-lit driving scene. However, in the case of poor visual environment, such as haze and other weather conditions, this method makes it difficult for drivers to make judgments about pedestrians, vehicles, wild animals, traffic signs and other information in advance at a long distance. Therefore, how to achieve rapid target recognition at night or in low-light environments is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a target recognition method, device, electronic device and medium based on a black light camera.

[0005] In a first aspect, the present disclosure provides a target recognition method based on a black light camera, comprising:

[0006] Acquire an image to be identified; the image to be identified is an image containing a driving scene captured by a black light camera;

[0007] Inputting the image to be identified into a trained target recognition model, so that the target recognition model determines whether the image to be identified needs to be defogged through a defogging discrimination network;

[0008] If the image to be identified needs to be defogged, the defogging parameter prediction network of the target recognition model is output to perform defogging, and a defogged image of the image to be identified is output;

[0009] The defogging image of the image to be identified is input into the target detection network of the target recognition model, so that the target recognition model performs target detection on the defogging image through the target detection network, and outputs the target category and target position of the image to be identified.

[0010] As an optional implementation of the embodiment of the present disclosure, the step of determining whether the image to be identified needs to be defogged by a defogging discrimination network further includes:

[0011] If the image to be identified does not need to be dehazed, it is output to the target detection network, so that the target recognition model performs target detection on the image to be identified through the target detection network, and outputs the target category and target position of the image to be identified.

[0012] As an optional implementation of the embodiment of the present disclosure, the step of acquiring the image to be recognized includes:

[0013] Collecting original images containing driving scenes through a black light camera;

[0014] The original image is resized to obtain an image to be recognized.

[0015] As an optional implementation of the embodiment of the present disclosure, the training method of the target recognition model includes:

[0016] Obtain an image training set; the image training set includes: a plurality of images containing categories of objects of interest in driving scenes; wherein all images carry label information, and the label information includes whether there is fog, category information, and location information;

[0017] Establishing a preset recognition model; the preset recognition model includes a defogging discrimination network, a defogging parameter prediction network and a target detection network, the defogging discrimination network is used to judge whether the image to be recognized needs to be defogged, the defogging parameter prediction network is used to defog the image to be recognized, and the target detection network is used to perform target recognition on the defogged image;

[0018] The image training set is input into a preset recognition model, and the preset recognition model is end-to-end trained using back propagation and gradient descent algorithms to obtain a target recognition model.

[0019] As an optional implementation of the embodiment of the present disclosure, the step of obtaining an image training set includes:

[0020] Collecting a plurality of original images containing the categories of objects of interest in the driving scene by a black light camera, annotating the plurality of original images to obtain a plurality of original images carrying the annotated information, and constructing a first image set;

[0021] Performing fogging processing to different degrees on each image containing the category of the object of interest in the driving scene in the first image set and adding a binary label for the defogging discriminant network to obtain a plurality of fogged images carrying the labeled information, and constructing a second image set;

[0022] An image training set is constructed based on the first image set and the second image set.

[0023] As an optional implementation of the embodiment of the present disclosure, the defogging parameter prediction network includes: an atmospheric scattering model defogging module, a median filtering module and a smoothing filtering module; the atmospheric scattering model defogging module is used to perform a defogging operation on the image by constructing an atmospheric scattering model, the median filtering module is used to enhance the contrast of the image, and the smoothing filtering module is used to adjust the smoothing intensity of different areas of the image.

[0024] As an optional implementation of the embodiment of the present disclosure, the atmospheric scattering model constructed by the atmospheric scattering model defogging module is as follows:

[0025]

[0026] Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

[0027] In a second aspect, an embodiment of the present disclosure provides a target recognition device based on a black light camera, comprising:

[0028] An acquisition module, used to acquire an image to be identified; the image to be identified is an image containing a driving scene acquired by a black light camera;

[0029] A judgment module, used for inputting the image to be recognized into a trained target recognition model, so that the target recognition model judges whether the image to be recognized needs to be defogged through a defogging discrimination network;

[0030] A processing module, configured to output the defogging parameter prediction network of the target recognition model to perform defogging if the image to be recognized needs to be defogged, and output a defogged image of the image to be recognized;

[0031] The recognition module is used to input the defogging image of the image to be recognized into the target detection network of the target recognition model, so that the target recognition model performs target detection on the defogging image through the target detection network, and outputs the target category and target position of the image to be recognized.

[0032] As an optional implementation of the embodiment of the present disclosure, the judgment module is further used to:

[0033] If the image to be identified does not need to be dehazed, it is output to the target detection network, so that the target recognition model performs target detection on the image to be identified through the target detection network, and outputs the target category and target position of the image to be identified.

[0034] As an optional implementation of the embodiment of the present disclosure, the acquisition module is specifically used to:

[0035] Collecting original images containing driving scenes through a black light camera;

[0036] The original image is resized to obtain an image to be recognized.

[0037] As an optional implementation of the embodiment of the present disclosure, the training method of the target recognition model includes:

[0038] Obtain an image training set; the image training set includes: a plurality of images containing categories of objects of interest in driving scenes; wherein all images carry label information, and the label information includes whether there is fog, category information, and location information;

[0039] Establishing a preset recognition model; the preset recognition model includes a defogging discrimination network, a defogging parameter prediction network and a target detection network, the defogging discrimination network is used to judge whether the image to be recognized needs to be defogged, the defogging parameter prediction network is used to defog the image to be recognized, and the target detection network is used to perform target recognition on the defogged image;

[0040] The image training set is input into a preset recognition model, and the preset recognition model is end-to-end trained using back propagation and gradient descent algorithms to obtain a target recognition model.

[0041] As an optional implementation of the embodiment of the present disclosure, the step of obtaining an image training set includes:

[0042] Collecting a plurality of original images containing the categories of objects of interest in the driving scene by a black light camera, annotating the plurality of original images to obtain a plurality of original images carrying the annotated information, and constructing a first image set;

[0043] Performing fogging processing to different degrees on each image containing the category of the object of interest in the driving scene in the first image set and adding a binary label for the defogging discriminant network to obtain a plurality of fogged images carrying the labeled information, and constructing a second image set;

[0044] An image training set is constructed based on the first image set and the second image set.

[0045] As an optional implementation of the embodiment of the present disclosure, the defogging parameter prediction network includes: an atmospheric scattering model defogging module, a median filtering module and a smoothing filtering module; the atmospheric scattering model defogging module is used to perform a defogging operation on the image by constructing an atmospheric scattering model, the median filtering module is used to enhance the contrast of the image, and the smoothing filtering module is used to adjust the smoothing intensity of different areas of the image.

[0046] As an optional implementation of the embodiment of the present disclosure, the atmospheric scattering model constructed by the atmospheric scattering model defogging module is as follows:

[0047]

[0048] Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

[0049] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, it implements the target recognition method based on a black light camera described in the first aspect or any embodiment of the first aspect.

[0050] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target recognition method based on a black light camera as described in any embodiment of the first aspect.

[0051] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art: obtaining an image to be identified, wherein the image to be identified is an image containing a driving scene collected by a black light camera, inputting the image to be identified into a trained target recognition model, judging whether the image to be identified needs to be defogged through a defogging discrimination network of the target recognition model, and if the image to be identified needs to be defogged, outputting the image to be identified to a defogging parameter prediction network of the target recognition model for defogging, outputting a defogged image of the image to be identified, inputting the defogged image into a target detection network of the target recognition model, performing target detection on the defogged image through the target detection network, and outputting a target category and target position of the image to be identified. First, the image is classified through the defogging discrimination network, and the necessity of the defogging preprocessing operation is directly judged from the detection effect, which increases the adaptability of image recognition in weather environments and effectively increases the program running speed in specific environments. Then, by adding image defogging preprocessing to the end-to-end target recognition model, the target recognition effect under harsh visual environment conditions (such as foggy days) can be significantly improved. Further, by combining image defogging preprocessing and target detection end-to-end into the same network model, the human judgment of the defogging effect is avoided, and the defogging effect is directly evaluated by the target detection effect, which can enhance the target detection effect. In addition, since the image to be identified collected by the black light camera includes the brightness information, color information and object contour of the image, the image to be identified is input into the target recognition model, which can realize rapid target recognition at night or in low light environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0054] Figure 1 is a flowchart of a target recognition method based on a black light camera provided in an embodiment of the present disclosure;

[0055] Figure 2 It is a schematic diagram of an improved yolov8 network structure provided by an embodiment of the present disclosure;

[0056] Figure 3 is a schematic structural diagram of a target recognition device based on a black light camera provided in an embodiment of the present disclosure;

[0057] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0060] Relational terms such as “first” and “second” in the description and claims of the present disclosure are merely used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0061] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0062] In some embodiments, Figure 1 As shown, a target recognition method based on a black light camera is provided, comprising the following steps S11-S14:

[0063] S11, obtaining an image to be recognized.

[0064] The image to be identified is an image containing a driving scene captured by a black light camera.

[0065] A blacklight camera is a device that can capture clear color images in extremely low light conditions. Blacklight cameras usually use two starlight-level image sensors, one to collect image brightness information and object outlines through infrared fill light, and the other to collect color information. The image information collected by the two sensors is fused through an image fusion algorithm to output a bright and colorful image.

[0066] In some embodiments, the above step S11 (obtaining the image to be identified) can be implemented in the following manner:

[0067] Collecting original images containing driving scenes through a black light camera;

[0068] The original image is resized to obtain an image to be recognized.

[0069] Resizing an image involves changing the image's width and height pixel values ​​to achieve the desired new size.

[0070] Specifically, the original image containing the driving scene is collected by the vehicle-mounted black light camera, and the original image is resized to obtain an image to be recognized that is adapted to the target detection network (such as the yolov8 network). It should be noted that the image resizing here does not change the original image ratio.

[0071] Exemplarily, since the image size collected by the black light camera is generally 2560x1440 pixels, the target detection network in the disclosed embodiment uses an improved yolov8 network. The input image size of the yolov8 network is generally 640x640 pixels, so it is necessary to resize the collected original image to obtain the image to be identified. Since direct size conversion will cause the aspect ratio of the image to change, the Letterbox method is used to process the image: that is, when the aspect ratio of the input image is inconsistent with that required by the model, Letterbox will add black pixels to fill on both sides of the image to maintain the consistency of the aspect ratio. Taking 2560x1440 pixels to 640x640 pixels as an example, the aspect ratio of 2560x1440 pixels is 16:9, so first convert 2560x1440 pixels to 640x360, and then fill black pixels along the second dimension to make it 640x640 pixels.

[0072] S12, inputting the image to be recognized into a trained target recognition model, so that the target recognition model determines whether the image to be recognized needs to be defogged through a defogging discrimination network.

[0073] Specifically, the defogging discriminant network is used to determine whether the image to be identified needs to be defogged. The defogging discriminant network includes 5 convolutional layers, two fully connected layers, a ReLu activation function layer, and a sigmoid normalization function layer. The input of the defogging discriminant network is the image to be identified, and the output is 0 or 1. Among them, the output of 0 indicates that defogging preprocessing is not required, and the output of 1 indicates that defogging preprocessing is required.

[0074] S13: If the image to be identified needs to be defogged, the image is output to the defogging parameter prediction network of the target recognition model for defogging, and a defogged image of the image to be identified is output.

[0075] Specifically, if the image to be identified needs to be defogged, the image to be identified is input into the defogging parameter prediction network for defogging to obtain a defogged image with good visual effect. That is, the input of the defogging parameter prediction network is the image to be identified, and the output is the defogged image of the image to be identified.

[0076] In some embodiments, if the image to be identified does not need to be dehazed, it is output to the target detection network so that the target recognition model performs target detection on the image to be identified through the target detection network and outputs the target category and target position of the image to be identified.

[0077] Specifically, if the image to be identified does not need to be dehazed, the image to be identified is output to the target detection network, and the target detection network performs target detection on the image to be identified, and outputs the target category and target position of the image to be identified.

[0078] For example, for images that do not require dehazing, pedestrians, vehicles, wild animals, traffic signs and other targets on the images are uniformly detected through the improved yolov8 network, and the target information obtained after detection and the image carrying the target information are output and passed to subsequent display and analysis devices.

[0079] S14, inputting the defogging image of the image to be identified into the target detection network of the target recognition model, so that the target recognition model performs target detection on the defogging image through the target detection network, and outputs the target category and target position of the image to be identified.

[0080] Specifically, for the preprocessed dehazed image, a target detection network (such as a yolov8 network) is used to perform a target recognition operation to obtain the target category and target position of the image to be recognized.

[0081] The embodiments provided by the present disclosure can deploy the target recognition model on edge devices such as vehicle-mounted computing devices. After deployment, the model is not only applicable to well-lit urban scenes, but also can achieve good recognition effects in dark, foggy, and wild animal-infested outdoor scenes. This method meets the target recognition effect in all-weather driving scenarios, while still having the target recognition effect required by users in poor visual environments, meeting the driver's assisted driving needs in poor visual environments.

[0082] All-weather driving scenarios refer to all possible driving situations in the real world, including daytime, nighttime, dusk, and other time periods, and in sunny, rainy, snowy, foggy, and other weather conditions. Different from general urban driving scenarios, this scenario generally has insufficient visibility, which makes the driver's understanding of the external environment significantly lower than that of urban driving scenarios, and poses a greater driving safety hazard.

[0083] In some embodiments, the training method of the target recognition model in step S14 above includes the following steps:

[0084] a. Obtain image training set.

[0085] The image training set includes: a plurality of images containing categories of objects of interest in driving scenes; wherein all images carry label information, and the label information includes whether there is fog, category information and location information.

[0086] Optionally, the above step a (obtaining an image training set) can be implemented in the following manner:

[0087] (1) Using a black light camera, collect multiple original images containing categories of objects of interest in the driving scene, annotate the multiple original images, obtain multiple original images carrying annotated information, and construct a first image set.

[0088] (2) Each image in the first image set containing the category of the object of interest in the driving scene is subjected to different degrees of fogging processing and a binary label for the defogging discriminant network is added to obtain multiple fogged images carrying labeled information to construct a second image set.

[0089] Specifically, fogging the image may be performed using a fogging algorithm, or by adjusting the RGB channels of the image to simulate the fog effect.

[0090] Exemplarily, based on standard optics, the intensity of the haze is calculated through an exponential decay function, and the brightness of each pixel is adjusted so that its brightness is affected.

[0091] (3) Construct an image training set based on the first image set and the second image set.

[0092] b. Establish a preset recognition model.

[0093] Among them, the preset recognition model includes a defogging discrimination network, a defogging parameter prediction network and a target detection network. The defogging discrimination network is used to determine whether the image to be identified needs to be defogged, the defogging parameter prediction network is used to defog the image to be identified, and the target detection network is used to perform target recognition on the defogged image.

[0094] Optionally, the defogging parameter prediction network includes: an atmospheric scattering model defogging module, a median filtering module and a smoothing filtering module; the atmospheric scattering model defogging module is used to perform a defogging operation on the image by constructing an atmospheric scattering model, the median filtering module is used to enhance the contrast of the image, and the smoothing filtering module is used to adjust the smoothing intensity of different areas of the image.

[0095] Among them, the atmospheric scattering model constructed by the atmospheric scattering model defogging module is as follows:

[0096]

[0097] Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

[0098] Specifically, for RGB images, the Koschmieder law (a scientific term in the field of atmosphere) that introduces the strength of the atmospheric curtain is used to model the image to describe the impact of fog, smoke or other atmospheric conditions on the image. Assuming that the camera has a linear response, the model believes that smoke effects such as haze have two effects on grayscale images. The first effect is to reduce the contrast of objects, thereby reducing the visibility of objects in the scene; the second effect is to add a white atmospheric curtain. The model is as follows:

[0099]

[0100] Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

[0101] It can be understood that in an image, each pixel has a coordinate position, usually represented by (x, y), where x represents the horizontal position and y represents the vertical position. By converting this model, we can get the atmospheric scattering model:

[0102]

[0103] Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

[0104] Use median filtering and dark channel to model V(x,y) and estimate the curtain strength of the atmosphere:

[0105] V(x,y)=max(min(p(A(x,y)-median sv (WA)(x,y)),W(w,y)),0)

[0106] A(x,y)=median si (W(x,y))

[0107] Among them, V(x,y) represents the intensity of the atmospheric curtain, and W(x,y) is the RGB image I fog (x, y), A(x, y) represents the average brightness of white objects in the image, A(x, y) is the local average value obtained by filtering W(x, y) with a median filter, p represents the defogging parameter, sv represents the contrast amplification parameter, and si represents the adaptive smoothing parameter.

[0108] Median filtering is a nonlinear digital filtering technique, which is usually used to remove image noise, especially salt and pepper noise, while maintaining image edge information. The basic principle of median filtering is, first, select a neighborhood of each pixel in the image, which can be a 3x3, 5x5 or larger square area, then sort all the pixel values ​​in the neighborhood to get a pixel value sequence, and finally replace the original pixel value with the sorted median value.

[0109] Specifically, the calculation of V(x,y) requires two median filters. The first median filter is to perform median filtering on W(x,y) to obtain A(x,y). Therefore, there are two hyperparameters sv and si, which represent the maximum size of the assumed white object in the image and the maximum size of the noise amplified by adaptive smoothing to reduce the recovery process.

[0110] In addition, different images may require different degrees of dehazing, so a balance can be found between maintaining the natural appearance of the image and enhancing visibility by adjusting the value p of the percentage of the removed atmospheric curtain. A higher p value will result in a stronger dehazing effect, but may make the image color too saturated or lose details; a lower p value provides a gentler restoration effect.

[0111] c. Input the image training set into a preset recognition model, and use back propagation and gradient descent algorithms to perform end-to-end training on the preset recognition model to obtain a target recognition model.

[0112] Back propagation mainly uses the chain rule to propagate the gradient of the loss function with respect to the network output to each layer of the network, thereby calculating the gradient of the loss function with respect to each network parameter. Then, based on these gradients, an optimization algorithm is used to update the network parameters so that the value of the loss function gradually decreases.

[0113] Specifically, the dehazing discriminative network uses back-propagation to update the network parameters through the loss function calculated between the output of the entire model and the manually annotated credible dataset.

[0114] The defogging parameter prediction network consists of 5 convolutional layers and 2 fully connected layers. The defogging parameter prediction network is used to estimate the hyperparameters p, sv, and si. For images that need to be defogged, the defogging parameters are estimated by the defogging parameter prediction network, and then the parameters are applied to the defogging parameter prediction network to defog the image.

[0115] The target information (i.e., whether there is fog, target category, and target location) output by the target detection network is used to calculate the loss function with the labeled data set information. The network weight parameters of the improved yolov8 are updated using the loss function, back propagation, and gradient descent methods. Figure 2 As shown in the figure, for the improved yolov8 architecture used, a detection head is added in the large-scale image range to retain the small target features of the image. The improved yolov8 network includes: a backbone network (for extracting image features), a neck network (for further fusion and processing of the features extracted by the backbone network), and a detection head (for classifying and locating targets). Among them, NMS (Non-Maximum Suppression) is a commonly used technique in target detection tasks. Its main function is to remove redundant bounding boxes and only retain the boxes that are most likely to contain the target object. In target detection, due to sliding windows of different sizes and positions, the same object may be detected multiple times, resulting in multiple overlapping bounding boxes. NMS selects the best bounding box by confidence and overlap (usually measured by intersection over union (IoU)).

[0116] On this basis, through training with a large number of data sets and detection effect testing and indicator evaluation of some test sets, we obtained a usable network model that can identify distant tiny targets such as pedestrians, cars, and wild animals in high-speed and dark environments.

[0117] The present disclosure provides a target recognition method based on a blacklight camera, which obtains an image to be recognized, wherein the image to be recognized is an image containing a driving scene collected by a blacklight camera, the image to be recognized is input into a trained target recognition model, and a defogging discrimination network of the target recognition model is used to determine whether the image to be recognized needs to be defogged. If the image to be recognized needs to be defogged, the image to be recognized is output to a defogging parameter prediction network of the target recognition model for defogging, and a defogged image of the image to be recognized is output. The defogged image is input into a target detection network of the target recognition model, and target detection is performed on the defogged image through the target detection network, and a target category and target position of the image to be recognized are output. First, the image is classified through the defogging discrimination network, and the necessity of the defogging preprocessing operation is directly judged from the detection effect, which increases the adaptability of image recognition in weather environments and effectively increases the program running speed in specific environments. Then, by adding image defogging preprocessing to the end-to-end target recognition model, the target recognition effect under harsh visual environment conditions (such as foggy days) can be significantly improved. Further, by combining image defogging preprocessing and target detection end-to-end into the same network model, the human judgment of the defogging effect is avoided, and the defogging effect is directly evaluated by the target detection effect, which can enhance the target detection effect. In addition, since the image to be identified collected by the black light camera includes the brightness information, color information and object contour of the image, the image to be identified is input into the target recognition model, which can realize rapid target recognition at night or in low light environments.

[0118] In some embodiments, reference Figure 3 As shown, a target recognition device 300 based on a black light camera is provided, comprising:

[0119] An acquisition module, used to acquire an image to be identified; the image to be identified is an image containing a driving scene acquired by a black light camera;

[0120] A judgment module, used for inputting the image to be recognized into a trained target recognition model, so that the target recognition model judges whether the image to be recognized needs to be defogged through a defogging discrimination network;

[0121] A processing module, configured to output the defogging parameter prediction network of the target recognition model to perform defogging if the image to be recognized needs to be defogged, and output a defogged image of the image to be recognized;

[0122] The recognition module is used to input the defogging image of the image to be recognized into the target detection network of the target recognition model, so that the target recognition model performs target detection on the defogging image through the target detection network, and outputs the target category and target position of the image to be recognized.

[0123] As an optional implementation of the embodiment of the present disclosure, the judgment module is further used to:

[0124] If the image to be identified does not need to be dehazed, it is output to the target detection network, so that the target recognition model performs target detection on the image to be identified through the target detection network, and outputs the target category and target position of the image to be identified.

[0125] As an optional implementation of the embodiment of the present disclosure, the acquisition module is specifically used to:

[0126] Collecting original images containing driving scenes through a black light camera;

[0127] The original image is resized to obtain an image to be recognized.

[0128] As an optional implementation of the embodiment of the present disclosure, the training method of the target recognition model includes:

[0129] Obtain an image training set; the image training set includes: a plurality of images containing categories of objects of interest in driving scenes; wherein all images carry label information, and the label information includes whether there is fog, category information, and location information;

[0130] Establishing a preset recognition model; the preset recognition model includes a defogging discrimination network, a defogging parameter prediction network and a target detection network, the defogging discrimination network is used to judge whether the image to be recognized needs to be defogged, the defogging parameter prediction network is used to defog the image to be recognized, and the target detection network is used to perform target recognition on the defogged image;

[0131] The image training set is input into a preset recognition model, and the preset recognition model is end-to-end trained using back propagation and gradient descent algorithms to obtain a target recognition model.

[0132] As an optional implementation of the embodiment of the present disclosure, the step of obtaining an image training set includes:

[0133] Collecting a plurality of original images containing the categories of objects of interest in the driving scene by a black light camera, annotating the plurality of original images to obtain a plurality of original images carrying the annotated information, and constructing a first image set;

[0134] Performing fogging processing to different degrees on each image containing the category of the object of interest in the driving scene in the first image set and adding a binary label for the defogging discriminant network to obtain a plurality of fogged images carrying the labeled information, and constructing a second image set;

[0135] An image training set is constructed based on the first image set and the second image set.

[0136] As an optional implementation of the embodiment of the present disclosure, the defogging parameter prediction network includes: an atmospheric scattering model defogging module, a median filtering module and a smoothing filtering module; the atmospheric scattering model defogging module is used to perform a defogging operation on the image by constructing an atmospheric scattering model, the median filtering module is used to enhance the contrast of the image, and the smoothing filtering module is used to adjust the smoothing intensity of different areas of the image.

[0137] As an optional implementation of the embodiment of the present disclosure, the atmospheric scattering model constructed by the atmospheric scattering model defogging module is as follows:

[0138]

[0139] Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

[0140] The present disclosure provides a target recognition device based on a black light camera, which obtains an image to be recognized, wherein the image to be recognized is an image containing a driving scene collected by a black light camera, and the image to be recognized is input into a trained target recognition model, and a defogging discrimination network of the target recognition model is used to determine whether the image to be recognized needs to be defogged. If the image to be recognized needs to be defogged, the image to be recognized is output to a defogging parameter prediction network of the target recognition model for defogging, and a defogged image of the image to be recognized is output, and the defogged image is input into a target detection network of the target recognition model, and target detection is performed on the defogged image through the target detection network, and a target category and target position of the image to be recognized are output. First, the image is classified through the defogging discrimination network, and the necessity of the defogging preprocessing operation is directly judged from the detection effect, which increases the adaptability of image recognition in weather environments and effectively increases the program running speed in specific environments. Then, by adding image defogging preprocessing to the end-to-end target recognition model, the target recognition effect under harsh visual environment conditions (such as foggy days) can be significantly improved. Further, by combining image defogging preprocessing and target detection end-to-end into the same network model, the human judgment of the defogging effect is avoided, and the defogging effect is directly evaluated by the target detection effect, which can enhance the target detection effect. In addition, since the image to be identified collected by the black light camera includes the brightness information, color information and object contour of the image, the image to be identified is input into the target recognition model, which can realize rapid target recognition at night or in low light environments.

[0141] For the specific definition of the target recognition device based on the blacklight camera, please refer to the definition of the target recognition method based on the blacklight camera above, which will not be repeated here. Each module in the above-mentioned target recognition device based on the blacklight camera can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor of the electronic device in the form of hardware, or can be stored in the processor of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0142] The present disclosure also provides an electronic device, Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 4 As shown, the electronic device provided in this embodiment includes: a memory 41 and a processor 42, the memory 41 is used to store a computer program; the processor 42 is used to execute the steps executed in any embodiment of the target recognition method based on a black light camera of the electronic device provided by the above method embodiment when calling the computer program. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a target recognition method based on a black light camera of an electronic device is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0143] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In some embodiments, the target recognition device based on the black light camera provided by the present disclosure can be implemented in the form of a computer, and the computer program can be used in Figure 4 The computer program composed of various program modules enables the processor to execute the steps of the target recognition method based on a black light camera in various embodiments of the present disclosure described in this specification.

[0145] The embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the target recognition method based on a black light camera of an electronic device provided in the above method embodiment is implemented.

[0146] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0147] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0148] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0149] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0150] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0151] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target recognition method based on a black light camera, characterized in that: The method comprises: Acquire an image to be identified; the image to be identified is an image containing a driving scene captured by a black light camera; Inputting the image to be identified into a trained target recognition model, so that the target recognition model determines whether the image to be identified needs to be defogged through a defogging discrimination network; If the image to be identified needs to be defogged, the defogging parameter prediction network of the target recognition model is output to perform defogging, and a defogged image of the image to be identified is output; The defogging image of the image to be identified is input into the target detection network of the target recognition model, so that the target recognition model performs target detection on the defogging image through the target detection network, and outputs the target category and target position of the image to be identified.

2. The method according to claim 1, characterized in that The step of judging whether the image to be identified needs to be defogged by a defogging discrimination network further includes: If the image to be identified does not need to be dehazed, it is output to the target detection network, so that the target recognition model performs target detection on the image to be identified through the target detection network, and outputs the target category and target position of the image to be identified.

3. The method according to claim 1, characterized in that: The step of obtaining the image to be identified comprises: Collecting original images containing driving scenes through a black light camera; The original image is resized to obtain an image to be recognized.

4. The method according to claim 1, characterized in that The training method of the target recognition model includes: Obtain an image training set; the image training set includes: a plurality of images containing categories of objects of interest in driving scenes; wherein all images carry label information, and the label information includes whether there is fog, category information, and location information; Establishing a preset recognition model; the preset recognition model includes a defogging discrimination network, a defogging parameter prediction network and a target detection network, the defogging discrimination network is used to judge whether the image to be recognized needs to be defogged, the defogging parameter prediction network is used to defog the image to be recognized, and the target detection network is used to perform target recognition on the defogged image; The image training set is input into a preset recognition model, and the preset recognition model is end-to-end trained using back propagation and gradient descent algorithms to obtain a target recognition model.

5. The method according to claim 4, characterized in that The step of obtaining an image training set comprises: Collecting a plurality of original images containing the categories of objects of interest in the driving scene by a black light camera, annotating the plurality of original images to obtain a plurality of original images carrying the annotated information, and constructing a first image set; Performing fogging processing to different degrees on each image containing the category of the object of interest in the driving scene in the first image set and adding a binary label for the defogging discriminant network to obtain a plurality of fogged images carrying the labeled information, and constructing a second image set; An image training set is constructed based on the first image set and the second image set.

6. The method according to claim 4, characterized in that The defogging parameter prediction network includes: an atmospheric scattering model defogging module, a median filtering module and a smoothing filtering module; the atmospheric scattering model defogging module is used to perform a defogging operation on an image by constructing an atmospheric scattering model, the median filtering module is used to enhance the contrast of the image, and the smoothing filtering module is used to adjust the smoothing intensity of different areas of the image.

7. The method according to claim 6, characterized in that The atmospheric scattering model constructed by the atmospheric scattering model defogging module is as follows: Among them, (x, y) represents the coordinates of each pixel in the image, I defog (x, y) represents the pixel brightness of each pixel of the output image, I fog (x, y) represents the pixel brightness of each pixel of the input image, V(x, y) represents the atmospheric curtain intensity, I sky Indicates the brightness of the sky.

8. A target recognition device based on a black light camera, characterized in that: include: An acquisition module, used for acquiring an image to be recognized; The image to be identified is an image containing a driving scene collected by a black light camera; A judgment module, used for inputting the image to be recognized into a trained target recognition model, so that the target recognition model judges whether the image to be recognized needs to be defogged through a defogging discrimination network; A processing module, configured to output the defogging parameter prediction network of the target recognition model to perform defogging if the image to be recognized needs to be defogged, and output a defogged image of the image to be recognized; The recognition module is used to input the defogging image of the image to be recognized into the target detection network of the target recognition model, so that the target recognition model performs target detection on the defogging image through the target detection network, and outputs the target category and target position of the image to be recognized.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the target recognition method based on a black light camera according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the target recognition method based on a black light camera described in any one of claims 1 to 7 is implemented.

Images