Nighttime vehicle light recognition method, apparatus, device, and storage medium

Abstract

The present application relates to the field of computer vision, and discloses a night vehicle light recognition method, device, equipment and storage medium, comprising: acquiring a to-be-processed vehicle light image; determining a binary image of the to-be-processed vehicle light image based on an adaptive segmentation threshold; expanding a vehicle light frame based on the minimum circumscribed box of each connected domain in the binary image to form a vehicle light recognition candidate set; placing the vehicle light frame with a classification confidence greater than a classification threshold in the vehicle light recognition candidate set into a target vehicle light set; identifying the type of vehicle light corresponding to the vehicle light frame based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set; wherein the type of vehicle light includes oncoming vehicle light and opposite direction vehicle light; the present application can improve the accuracy of vehicle light classification, thereby further improving the accuracy and reliability of night vehicle light recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a night vehicle light recognition method, device, equipment and storage medium. BACKGROUND

[0002] As an important function of night-time assisted driving, the intelligent high beam control system plays a crucial role in night-time assisted driving. In the night-time assisted driving scenario, when the intelligent high beam control system detects the headlight of an oncoming vehicle, the taillight of a same-direction vehicle, or other visible light sources, it can automatically switch the light source of the vehicle, switching the high beam to the low beam, to prevent the high beam of the vehicle from dazzling other drivers. Since the vehicle light is the most prominent representative feature of the vehicle in the night-time scenario, it is extremely critical to improve the accuracy of vehicle light detection. However, at the present stage, due to the strong reflection, large halo, and low distinguishability of the vehicle light, the accuracy of vehicle light detection is low. SUMMARY

[0003] Therefore, the present application provides a night vehicle light recognition method, device, equipment and storage medium to solve the problem that the accuracy of vehicle light detection is low due to the strong reflection, large halo, and low distinguishability of the vehicle light at the present stage.

[0004] In a first aspect, the present application provides a night vehicle light recognition method, which comprises: acquiring a vehicle light image to be processed; determining a binary image of the vehicle light image to be processed based on an adaptive segmentation threshold; expanding a light frame based on the minimum circumscribed box of each connected domain in the binary image to form a light recognition candidate set; placing the light frame in the target light set if the classification confidence of the light frame in the light recognition candidate set is greater than a classification threshold; and determining the type of the light frame based on the center coordinates and color space information of each light frame in the target light set, wherein the type of the light frame includes the light of an oncoming vehicle and the light of a same-direction vehicle. Through the above process, the accuracy of light classification can be improved, thereby further improving the accuracy and reliability of night-time light recognition.

[0005] In an optional implementation, the acquisition of the vehicle light image to be processed comprises:

[0006] acquiring a source image and vanishing point coordinates of the source image;

[0007] cutting a light region to be processed from the source image with the vanishing point coordinates as the center;

[0008] filtering noise points in the light region to be processed to obtain the vehicle light image to be processed.

[0009] In an optional implementation, the determination of the binary image of the vehicle light image to be processed based on the adaptive segmentation threshold comprises:

[0010] Obtaining pixel values of each pixel in the to-be-processed vehicle lamp image and an adaptive segmentation threshold corresponding to each pixel;

[0011] Determining a binary image of the to-be-processed vehicle lamp image based on a difference between the pixel value of each pixel and the adaptive segmentation threshold.

[0012] In an optional implementation, the adaptive segmentation threshold is obtained by:

[0013] Obtaining target neighborhood pixels corresponding to the current pixel and neighborhood pixel values of the target neighborhood pixels;

[0014] Determining a target neighborhood pixel mean value and a target neighborhood pixel amplitude change rate based on the neighborhood pixel values;

[0015] Determining the adaptive segmentation threshold based on the target neighborhood pixel mean value and the target neighborhood pixel amplitude change rate.

[0016] In an optional implementation, a vehicle lamp frame is expanded based on a minimum bounding box of each connected domain in the binary image to form a vehicle lamp recognition candidate set, including:

[0017] Creating a minimum bounding box set based on the minimum bounding box of each connected domain in the binary image;

[0018] Placing elements with a luminance value greater than a luminance threshold in the minimum bounding box set in a target rectangle set;

[0019] Obtaining frame centers of each minimum bounding box in the target rectangle set;

[0020] Expanding the minimum bounding box with the frame center as the origin according to a preset length-width ratio to obtain the vehicle lamp recognition candidate set.

[0021] In an optional implementation, a vehicle lamp frame with a classification confidence greater than a classification threshold in the vehicle lamp recognition candidate set is placed in a target vehicle lamp set, including:

[0022] Inputting the vehicle lamp frame in the vehicle lamp recognition candidate set into a target vehicle lamp classification model to obtain a classification confidence of each vehicle lamp frame;

[0023] Placing the vehicle lamp frame with the classification confidence greater than the classification threshold in the target vehicle lamp set.

[0024] In an optional implementation, the target vehicle lamp classification model is obtained by:

[0025] Obtaining sample data and labels of the sample data, the labels including vehicle lamp frames and reflective regions;

[0026] Inputting the sample data into the vehicle lamp classification model to obtain a predicted classification confidence;

[0027] Based on the prediction classification confidence and the label, parameters of the vehicle light classification model are updated to determine a target vehicle light classification model.

[0028] In an optional implementation, the method further includes:

[0029] A vehicle light matching distance is determined based on the vanishing point coordinates and the center coordinates of the vehicle light frame.

[0030] It is determined whether the two vehicle light frames belong to the same vehicle based on the vehicle light matching distance.

[0031] The two vehicle light frames corresponding to the same vehicle are tracked.

[0032] In a second aspect, the present application provides a night vehicle light recognition device, which mainly includes a figure acquisition module, a binary processing module, a light frame expansion module, a light frame classification module, and a vehicle light recognition module. The figure acquisition module is configured to acquire a vehicle light image to be processed. The binary processing module is configured to determine a binary image of the vehicle light image to be processed based on a self-adaptive segmentation threshold. The light frame expansion module is configured to expand a vehicle light frame based on a minimum circumscribed frame of each connected domain in the binary image to form a vehicle light recognition candidate set. The light frame classification module is configured to place a vehicle light frame with a classification confidence greater than a classification threshold in the vehicle light recognition candidate set into a target vehicle light set. The vehicle light recognition module is configured to recognize a vehicle light type corresponding to the vehicle light frame based on center coordinates and color space information of each vehicle light frame in the target vehicle light set. The vehicle light type includes a same-direction oncoming vehicle light and an opposite-direction oncoming vehicle light. Through the above process, the accuracy of vehicle light classification can be improved, thereby further improving the accuracy and reliability of night vehicle light recognition.

[0033] In a third aspect, the present application provides a computer device, which includes a memory and a processor, the memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the night vehicle light recognition method of the first aspect or any of the corresponding embodiments thereof.

[0034] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the night vehicle light recognition method of the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0036] Figure 1 is a schematic diagram of an application environment of an embodiment of the present application;

[0037] Figure 2 is a flowchart of a night vehicle light recognition method of an embodiment of the present application;

[0038] Figure 3 is a flowchart of another night vehicle light recognition method of an embodiment of the present application;

[0039] Figure 4 is a flowchart of still another night vehicle light recognition method of an embodiment of the present application;

[0040] Figure 5 is a data flow diagram of yet another night vehicle light recognition method of an embodiment of the present application;

[0041] Figure 6 is a structural block diagram of a night vehicle light recognition device of an embodiment of the present application;

[0042] Figure 7 is a hardware structure schematic diagram of a computer device of an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0044] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device. "Multiple" in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0045] Please refer to Figure 1 , Figure 1is a schematic diagram of an application environment provided by an embodiment of the present application, which includes a display 101, a processor 102 and a memory 103 that can be included in a client 100. The client 100 can be communicatively connected to a server 200 through a network 300, the server 200 can be used to provide services (such as application services, etc.) for a server or a client installed on the server, a database 201 can be set on the server 200 or independently of the server 200, and used to provide data storage services for the server 200. In addition, the server 200 can run a processing engine 202, which can be used to execute the steps performed by the server 200.

[0046] Optionally, the client 100 can be, but is not limited to, a terminal that can calculate data, such as a mobile terminal (for example, a mobile phone, a tablet computer), a notebook computer, a PC (Personal Computer, personal computer) terminal, etc. The network can include, but is not limited to, a wireless network or a wired network. The wireless network includes Bluetooth, WIFI (Wireless Fidelity, Wireless Fidelity) and other wireless communication networks. The wired network can include, but is not limited to, a wide area network, a metropolitan area network, and a server cluster. The server 200 can include, but is not limited to, any hardware device that can perform calculations.

[0047] In addition, in the embodiment, the night vehicle lamp recognition method can also be applied to a separate processing device with strong processing capability without data interaction, for example, the processing device can be, but is not limited to, a terminal device with strong processing capability, that is, each operation in the night vehicle lamp recognition method can be integrated in a separate processing device. The above is only an example, and the present embodiment is not limited in this regard.

[0048] Optionally, in the embodiment, the night vehicle lamp recognition method can be executed by the server 200, or by the client 100, or by the server 200 and the client 100 together. The client 100 executing the night vehicle lamp recognition method of the embodiment of the present application can also be executed by the client installed thereon.

[0049] According to the embodiment of the present application, a night vehicle lamp recognition method embodiment is provided. It should be noted that the steps shown in the flowchart of the drawing can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] In the embodiment, a night vehicle lamp recognition method is provided, which can be used in the above-mentioned client, the client communicates with the server through a local area network,Figure 2 is a flowchart of a night vehicle light recognition method according to an embodiment of the present application, as shown in the figure, the flow includes the following steps: Figure 2

[0051] In step S201, a vehicle light image to be processed is acquired.

[0052] In this embodiment, the vehicle light image to be processed is acquired so as to generate a corresponding binary image based on the image to be processed.

[0053] In an alternative implementation, a source image of a target size can be acquired first, and vanishing point coordinates of the source image are acquired, and then a vehicle light region to be processed is cropped from the source image with the vanishing point coordinates of the source image as the center, and then noise points in the vehicle light region to be processed are filtered to highlight image bright spot information, so as to obtain the vehicle light image to be processed.

[0054] In a specific implementation, the source image can be scaled to 640*360 in size through bilinear interpolation, and the vehicle light region to be processed is cropped with the vanishing point coordinates as the center, and a height of 128 and a width of 256. Then, the noise points in the vehicle light region to be processed are filtered through Gaussian filtering to highlight the image bright spot information, so as to obtain the vehicle light image to be processed.

[0055] The source image can be a front view image captured by a night vehicle camera; the 100W camera 1280*720 used by the current vehicle camera is halved in size, and the loss of pixel information of the vehicle light is less. The size here is the pixel value. The vanishing point is obtained according to the camera calibration in the early stage, and the final result is a plan view. The role of the vanishing point coordinates is that within the camera view angle range, the vehicle is concentrated in the cropped range of interest, and the interference information in other regions of the image is eliminated.

[0056] It can be understood that the cropping is performed with reference to the vanishing point, and the size can cover most of the vehicles and light sources in the front view range. In actual engineering projects, vehicle light regions to be processed of different sizes can be set to cover the light source regions to be detected. At the same time, in order to improve the efficiency of the cropping of the vehicle light region to be processed, the cropping region is preferably a regular region.

[0057] In step S202, a binary image of the vehicle light image to be processed is determined based on an adaptive segmentation threshold.

[0058] In this embodiment, the binary image of the vehicle light image to be processed is determined based on an adaptive segmentation threshold, which provides data support for the acquisition of the vehicle light frame.

[0059] ​In an alternative implementation, pixel values of pixels in the to-be-processed vehicle lamp image and adaptive segmentation thresholds corresponding to the pixels can be obtained first, and then a binary image of the to-be-processed vehicle lamp image can be determined based on differences between the pixel values and the adaptive segmentation thresholds.

[0060] In a specific implementation, pixel values of pixels in the to-be-processed vehicle lamp image and adaptive segmentation thresholds corresponding to the pixels can be obtained first, and then a binary image of the to-be-processed vehicle lamp image can be obtained by subtracting the adaptive segmentation thresholds from the pixel values and setting pixels with subtraction values greater than 0 to 1 and otherwise to 0.

[0061] In step S203, a vehicle lamp frame is extended based on a minimum bounding box of each connected domain in the binary image to form a vehicle lamp recognition candidate set.

[0062] In this embodiment, the vehicle lamp recognition candidate set is formed by extending a vehicle lamp frame based on a minimum bounding box of each connected domain in the binary image, thereby improving the accuracy of vehicle lamp classification.

[0063] In an alternative implementation, a minimum bounding box set can be created based on minimum bounding boxes of each connected domain in the binary image, elements with luminance values greater than a luminance threshold in the minimum bounding box set can be placed in a target rectangle set, the frame centers of the minimum bounding boxes in the target rectangle set can be determined, and the minimum bounding boxes can be extended with the frame centers as the origins and according to a preset length-width ratio to obtain the vehicle lamp recognition candidate set.

[0064] In a specific implementation, a minimum bounding box set can be obtained according to the connected domain information in the binary image, the pixel sum of each pixel in each connected domain can be obtained, and the luminance value of each minimum bounding box in the minimum bounding box set and the luminance threshold corresponding to each minimum bounding box in the minimum bounding box set can be determined based on the ratio of the pixel sum of each connected domain to the area of the corresponding minimum bounding box. Optionally, the luminance threshold is the average luminance value. The minimum bounding boxes with luminance values greater than the average luminance value can be placed in the target rectangle set. Then, the frame centers of the minimum bounding boxes in the target rectangle set can be obtained, and the minimum bounding boxes can be extended with the frame centers as the origins and according to a preset length-width ratio, such as a quarter length-width ratio, to obtain the vehicle lamp recognition candidate set.

[0065] Optionally, the peak and trough values of the above ratio can be obtained, and the minimum bounding boxes greater than the maximum trough value can be placed in the target rectangle set.

[0066] In step S204, the vehicle lamp frames with classification confidence greater than a classification threshold in the vehicle lamp recognition candidate set can be placed in a target vehicle lamp set.

[0067] In this embodiment, the vehicle light frame with a classification confidence greater than a classification threshold in the vehicle light recognition candidate set is placed in the target vehicle light set to further improve the accuracy of classifying the vehicle light frame.

[0068] In an optional embodiment, the vehicle light frame in the vehicle light recognition candidate set can be input into a target vehicle light classification model to obtain the classification confidence of each vehicle light frame; the vehicle light frame with a classification confidence greater than a classification threshold is placed in the target vehicle light set. The target vehicle light classification model can be a model trained by a deep learning network.

[0069] In step S205, the type of the vehicle light corresponding to the vehicle light frame is recognized based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set.

[0070] In this embodiment, the type of the vehicle light corresponding to the vehicle light frame is recognized based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set to determine whether the vehicle light corresponding to the vehicle light frame is a same-direction oncoming vehicle light or an opposite-direction oncoming vehicle light. The color space information can assist in determining whether the vehicle light is a headlight or a tail light.

[0071] In an optional embodiment, the center coordinates of each vehicle light frame in the target vehicle light set are first obtained, and then the matching distance of the vehicle light is determined based on the center coordinates and the center coordinates of the vehicle light frame; whether the two vehicle light frames belong to the same vehicle is determined based on the matching distance of the vehicle light. The type of the vehicle light corresponding to the vehicle light frame of the same vehicle is recognized according to the color space information of each vehicle light frame in the target vehicle light set to determine whether the vehicle light corresponding to the vehicle light frame is a same-direction oncoming vehicle light or an opposite-direction oncoming vehicle light. Finally, the matched vehicle light frame is tracked according to the Kalman filter, and the tracking of the target vehicle is realized.

[0072] The night vehicle light recognition method provided in this embodiment first acquires a to-be-processed vehicle light image to facilitate generating a corresponding binary image based on the to-be-processed image; determines the binary image of the to-be-processed vehicle light image based on an adaptive segmentation threshold to provide data support for acquiring the vehicle light frame; expands the vehicle light frame based on the minimum circumscribed frame of each connected domain in the binary image to form a vehicle light recognition candidate set, thereby improving the accuracy of vehicle light classification; places the vehicle light frame with a classification confidence greater than a classification threshold in the vehicle light recognition candidate set in the target vehicle light set to further improve the accuracy of classifying the vehicle light frame; recognizes the type of the vehicle light corresponding to the vehicle light frame based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set to determine whether the vehicle light corresponding to the vehicle light frame is a same-direction oncoming vehicle light or an opposite-direction oncoming vehicle light; therefore, the accuracy of vehicle light classification can be improved, and the accuracy and reliability of night vehicle light recognition can be further improved.

[0073] This embodiment provides a method for nighttime vehicle light recognition, which can be used in the aforementioned client. The client communicates with the server via a local area network. Figure 3 This is a flowchart of a nighttime vehicle light recognition method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0074] Step S301: Obtain the image of the vehicle headlights to be processed.

[0075] In this embodiment, an image of the vehicle headlights to be processed is acquired so that a corresponding binarized image can be generated based on the image to be processed.

[0076] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0077] Step S302: Determine the binarized image of the headlight image to be processed based on the adaptive segmentation threshold.

[0078] In this embodiment, the binarized image of the headlight image to be processed is determined based on an adaptive segmentation threshold, providing data support for the acquisition of the headlight frame.

[0079] Specifically, step S302 above includes:

[0080] Step S3021: Obtain the pixel value of each pixel in the image of the vehicle headlights to be processed, and the adaptive segmentation threshold corresponding to each pixel.

[0081] In this embodiment, the pixel values ​​of each pixel in the headlight image to be processed, as well as the adaptive segmentation threshold corresponding to each pixel, are obtained so as to calculate the binarized image of the headlight image to be processed based on the adaptive segmentation threshold, thereby improving the accuracy of filtering headlights from the headlight image to be processed.

[0082] In one optional implementation, during the adaptive segmentation threshold calculation process, the target neighboring pixels corresponding to the current pixel and the neighboring pixel values ​​of the target neighboring pixels can be obtained; then, the mean value of the target neighboring pixels and the rate of change of the target neighboring pixel amplitude are determined based on the neighboring pixel values; finally, the adaptive segmentation threshold is determined based on the mean value of the target neighboring pixels and the rate of change of the target neighboring pixel amplitude.

[0083] In implementation, the pixel coordinates of each pixel in the to-be-processed vehicle lamp image can be acquired, eight neighborhood pixels centered on the current pixel coordinates can be determined, and the neighborhood pixel values of each neighborhood in the eight neighborhood pixels can be acquired. The average value of the eight neighborhood pixels can be calculated based on the neighborhood pixel values. The amplitude values of the neighborhood pixels can be calculated based on the neighborhood pixel values and the average value of the eight neighborhood pixels. The amplitude variation rate of the eight neighborhood pixels can be calculated based on the amplitude values of the neighborhood pixels, and then the adaptive segmentation threshold can be determined based on the average value of the eight neighborhood pixels and the amplitude variation rate of the eight neighborhood pixels. It can be understood that in other embodiments, the number of target neighborhood pixels can also be adjusted according to actual needs.

[0084] In an alternative embodiment, the calculation model of the adaptive segmentation threshold is:

[0085]

[0086] wherein, is the amplitude value of the target neighborhood pixel, ; is the pixel value of the current pixel coordinates in the to-be-processed vehicle lamp image, is the average value of the target neighborhood pixel, is a coefficient factor, and the default value is 0.5.

[0087] In step S3022, the binary image of the to-be-processed vehicle lamp image is determined based on the difference between the pixel value of each pixel and the adaptive segmentation threshold.

[0088] In this embodiment, the binary image of the to-be-processed vehicle lamp image is determined based on the difference between the pixel value of each pixel and the adaptive segmentation threshold, so as to further improve the accuracy of the light spot acquisition.

[0089] In an alternative embodiment, the pixel value of each pixel in the to-be-processed vehicle lamp image and the adaptive segmentation threshold corresponding to each pixel are first acquired, and then the pixel value of each pixel is subtracted by the corresponding adaptive segmentation threshold. The pixels greater than 0 after subtraction are set to 1, and otherwise set to 0, so as to obtain the binary image of the to-be-processed vehicle lamp image.

[0090] In step S303, the vehicle lamp frame is expanded based on the minimum circumscribed frame of each connected domain in the binary image, so as to form a vehicle lamp recognition candidate set.

[0091] In this embodiment, the vehicle lamp frame is expanded based on the minimum circumscribed frame of each connected domain in the binary image, so as to form a vehicle lamp recognition candidate set, thereby improving the accuracy of vehicle lamp classification.

[0092] For details, please refer to step S203 of the embodiment shown in Figure 2 , which will not be described here again.

[0093] Step S304, placing the vehicle light frame with the classification confidence greater than the classification threshold in the candidate set into the target vehicle light set.

[0094] In this embodiment, the vehicle light frame with the classification confidence greater than the classification threshold in the candidate set is placed into the target vehicle light set, so as to further improve the accuracy of classifying the vehicle light frame.

[0095] For details, please refer to Figure 2 Step S204 of the embodiment shown will not be repeated here.

[0096] Step S305, identifying the type of the vehicle light corresponding to the vehicle light frame based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set.

[0097] In this embodiment, the type of the vehicle light corresponding to the vehicle light frame is identified based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set, so as to determine whether the vehicle light corresponding to the vehicle light frame is a same-direction oncoming vehicle light or an opposite-direction oncoming vehicle light.

[0098] For details, please refer to Figure 2 Step S205 of the embodiment shown will not be repeated here.

[0099] The night vehicle light identification method provided in this embodiment, by acquiring a to-be-processed vehicle light image, so as to generate a corresponding binary image based on the to-be-processed image; by determining the binary image of the to-be-processed vehicle light image based on the adaptive segmentation threshold, providing data support for the acquisition of the vehicle light frame; by expanding the vehicle light frame based on the minimum circumscribed frame of each connected domain in the binary image, to form a vehicle light identification candidate set, so as to improve the accuracy of vehicle light classification; by placing the vehicle light frame with the classification confidence greater than the classification threshold in the candidate set into the target vehicle light set, so as to further improve the accuracy of classifying the vehicle light frame; by identifying the type of the vehicle light corresponding to the vehicle light frame based on the center coordinates and color space information of each vehicle light frame in the target vehicle light set, so as to determine whether the vehicle light corresponding to the vehicle light frame is a same-direction oncoming vehicle light or an opposite-direction oncoming vehicle light; therefore, the accuracy of vehicle light classification can be improved, so as to further improve the accuracy and reliability of night vehicle light identification.

[0100] In this embodiment, a night vehicle light identification method is provided, which can be used in the client described above, and the client communicates with the server through a local area network, Figure 4 is a flowchart of the night vehicle light identification method according to the embodiment of the present application, as Figure 4 shown, the flowchart includes the following steps:

[0101] Step S401, acquiring a to-be-processed vehicle light image.

[0102] For details, please refer to Figure 2Step S201 of the illustrated embodiment will not be described here again.

[0103] Step S402, determining a binary image of the to-be-processed vehicle lamp image based on the adaptive segmentation threshold.

[0104] In the embodiment, the binary image of the to-be-processed vehicle lamp image is determined based on the adaptive segmentation threshold, thereby providing data support for obtaining the vehicle lamp frame.

[0105] For details, please refer to Figure 3 Step S302 of the illustrated embodiment will not be described here again.

[0106] Step S403, expanding the vehicle lamp frame based on the minimum bounding box of each connected domain in the binary image to form a vehicle lamp recognition candidate set.

[0107] In the embodiment, the vehicle lamp frame is expanded based on the minimum bounding box of each connected domain in the binary image to form a vehicle lamp recognition candidate set, thereby improving the accuracy of vehicle lamp classification.

[0108] For details, please refer to Figure 2 Step S203 of the illustrated embodiment will not be described here again.

[0109] Step S404, placing the vehicle lamp frame with a classification confidence greater than a classification threshold in the vehicle lamp recognition candidate set into a target vehicle lamp set

[0110] In the embodiment, the vehicle lamp frame with a classification confidence greater than a classification threshold in the vehicle lamp recognition candidate set is placed into a target vehicle lamp set, thereby further improving the accuracy of classifying the vehicle lamp frame.

[0111] Specifically, the above step S404 includes:

[0112] Step S4041, inputting the vehicle lamp frame in the vehicle lamp recognition candidate set into a target vehicle lamp classification model to obtain the classification confidence of each vehicle lamp frame.

[0113] In the embodiment, the vehicle lamp frame in the vehicle lamp recognition candidate set is input into a target vehicle lamp classification model to obtain the classification confidence of each vehicle lamp frame, so as to select a vehicle lamp frame with higher reliability from the vehicle lamp recognition candidate set according to the classification confidence of each vehicle lamp frame.

[0114] In an optional implementation, sample data and labels of the sample data can be first obtained, the labels including vehicle lamp frames and reflective regions; the sample data is input into a vehicle lamp classification model to obtain a predicted classification confidence; parameters of the vehicle lamp classification model are updated based on the predicted classification confidence and the labels to obtain a target vehicle lamp classification model, and then the vehicle lamp frame in the vehicle lamp recognition candidate set is input into the target vehicle lamp classification model to obtain the classification confidence of each vehicle lamp frame.

[0115] In an alternative embodiment, the car light frame and the reflection area can also be manually labeled, a positive and negative sample training data set is constructed, a convolutional neural network is used as a basic framework to construct a car light classification training model, the number of samples selected for one training is set to 128, and the learning rate is set to 0.01; and the car light classification model is trained. The car light classification model is used to classify the car light frame in the car light recognition candidate set, and the classification is classified into car light and non-car light. Then, the classification confidence of each car light frame in the car light classification is calculated by using a normalized exponential function.

[0116] In step S4042, the car light frame with a classification confidence greater than a classification threshold is placed in the target car light set.

[0117] In this embodiment, the car light frame with a classification confidence greater than a classification threshold is placed in the target car light set, so as to further filter out the car light frame with a low classification confidence.

[0118] In an alternative embodiment, the car light frame with a classification confidence greater than a classification threshold of 0.8 is placed in the target car light set, so as to further filter out the car light frame with a low classification confidence.

[0119] In step S405, the car light type corresponding to the car light frame is identified based on the center coordinates and color space information of each car light frame in the target car light set.

[0120] In this embodiment, the car light type corresponding to the car light frame is identified based on the center coordinates and color space information of each car light frame in the target car light set, so as to determine whether the car light corresponding to the car light frame is a car light of a same direction car or a car light of an opposite direction car.

[0121] For details, please refer to Figure 2 In step S205 of the embodiment shown, no further description is given here.

[0122] The night car light recognition method provided in this embodiment, by acquiring a to-be-processed car light image, so as to generate a corresponding binary image based on the to-be-processed image; by determining the binary image of the to-be-processed car light image based on the adaptive segmentation threshold, data support is provided for the acquisition of the car light frame; by expanding the car light frame based on the minimum circumscribed frame of each connected domain in the binary image, a car light recognition candidate set is formed, so as to improve the accuracy of car light classification; by placing the car light frame with a classification confidence greater than a classification threshold in the target car light set, the accuracy of classifying the car light frame is further improved; by identifying the car light type corresponding to the car light frame based on the center coordinates and color space information of each car light frame in the target car light set, it is determined whether the car light corresponding to the car light frame is a car light of a same direction car or a car light of an opposite direction car; therefore, the accuracy of car light classification can be improved, so as to further improve the accuracy and reliability of night car light recognition.

[0123] The embodiment provides a night vehicle lamp identification method, which can be used for the client, Figure 5 The embodiment provides a night vehicle lamp identification method, which can be used for the client, Figure 5 The embodiment provides a night vehicle lamp identification method, which can be used for the client,

[0124] Step S501: acquiring a to-be-processed vehicle lamp image.

[0125] In the embodiment, the to-be-processed vehicle lamp image is acquired, so that a corresponding binary image is generated based on the to-be-processed image.

[0126] For details, refer to step S201 in the embodiment shown in Figure 2 which will not be repeated here.

[0127] Step S502: determining a binary image of the to-be-processed vehicle lamp image based on an adaptive segmentation threshold.

[0128] In the embodiment, the binary image of the to-be-processed vehicle lamp image is determined based on the adaptive segmentation threshold, thereby providing data support for acquisition of the vehicle lamp frame.

[0129] For details, refer to step S302 in the embodiment shown in Figure 3 which will not be repeated here.

[0130] Step S503: expanding a vehicle lamp frame based on a minimum circumscribed frame of each connected domain in the binary image, so as to form a vehicle lamp identification candidate set.

[0131] In the embodiment, the vehicle lamp frame is expanded based on the minimum circumscribed frame of each connected domain in the binary image, so as to form the vehicle lamp identification candidate set, thereby improving the accuracy of vehicle lamp classification.

[0132] For details, refer to step S203 in the embodiment shown in Figure 2 which will not be repeated here.

[0133] Step S504: placing a vehicle lamp frame with a classification confidence greater than a classification threshold in the vehicle lamp identification candidate set into a target vehicle lamp set.

[0134] In the embodiment, the vehicle lamp frame with the classification confidence greater than the classification threshold in the vehicle lamp identification candidate set is placed into the target vehicle lamp set, thereby further improving the accuracy of classification of the vehicle lamp frame.

[0135] For details, refer to step S404 in the embodiment shown in Figure 4 which will not be repeated here.

[0136] Step S505: identifying a vehicle lamp type corresponding to the vehicle lamp frame based on center coordinates and color space information of each vehicle lamp frame in the target vehicle lamp set.

[0137] In the embodiment, the type of the vehicle lamp corresponding to the vehicle lamp frame is recognized based on the center coordinates of each vehicle lamp frame in the target vehicle lamp set and the color space information, so as to determine whether the vehicle lamp corresponding to the vehicle lamp frame is a same-direction oncoming vehicle lamp or an opposite-direction oncoming vehicle lamp.

[0138] For details, please refer to Figure 2 The step S205 of the embodiment is not described here again.

[0139] In step S506, the vehicle lamp matching distance is determined based on the vanishing point coordinates and the center coordinates of the vehicle lamp frame.

[0140] In the embodiment, the vehicle lamp matching distance is determined based on the vanishing point coordinates and the center coordinates of the vehicle lamp frame, so as to determine whether the two vehicle lamp frames belong to the same vehicle based on the vehicle lamp matching distance, and to determine whether the bright spot region in the distance is a vehicle lamp.

[0141] It can be understood that, according to the camera pinhole imaging principle, the near is large and the far is small, the vanishing point position in the source image is infinitely far away in the world coordinate system, and the width of the vehicle lamp corresponding to the specific ordinate position (the center coordinates of the vehicle lamp frame) can be determined according to the vanishing point position in the source image (the distance between the abscissas), so as to determine whether the bright spot region in the distance is a vehicle lamp. For example, the conventional vehicle width in reality is two meters and five, and the distance between the two vehicle lamps can be estimated according to the Y coordinate values of the vanishing point and the vehicle lamp.

[0142] In step S507, it is determined whether the two vehicle lamp frames belong to the same vehicle based on the vehicle lamp matching distance.

[0143] In the embodiment, it is determined whether the two vehicle lamp frames belong to the same vehicle based on the vehicle lamp matching distance, so as to further determine whether the bright spot region is a vehicle lamp. For example, if the distance between the two bright spots meets the minimum matching distance, the other bright spot can be inferred as a vehicle lamp through the already determined vehicle lamp.

[0144] In step S508, the two vehicle lamp frames corresponding to the same vehicle are tracked.

[0145] In the embodiment, the two vehicle lamp frames corresponding to the same vehicle are tracked, so as to track the vehicle within the target distance, provide necessary conditions for subsequent automatic switching of the vehicle light source, such as switching the high beam to the low beam, and prevent the high beam of the vehicle from dazzling other drivers.

[0146] The night vehicle lamp recognition method provided in the embodiment is characterized in that: a to-be-processed vehicle lamp image is acquired, so as to generate a corresponding binary image based on the to-be-processed image; the binary image of the to-be-processed vehicle lamp image is determined based on an adaptive segmentation threshold, thereby providing data support for acquisition of a vehicle lamp frame; a vehicle lamp frame is expanded based on a minimum circumscribed frame of each connected domain in the binary image, so as to form a vehicle lamp recognition candidate set, thereby improving the accuracy of vehicle lamp classification; a vehicle lamp frame with a classification confidence greater than a classification threshold in the vehicle lamp recognition candidate set is placed in a target vehicle lamp set, so as to further improve the accuracy of classification of the vehicle lamp frame; a vehicle lamp type corresponding to the vehicle lamp frame is recognized based on the center coordinates and color space information of each vehicle lamp frame in the target vehicle lamp set, so as to determine whether the vehicle lamp frame corresponds to a same-direction oncoming vehicle lamp or an opposite-direction oncoming vehicle lamp; a vehicle lamp matching distance is determined based on the extinction point coordinates and the center coordinates of the vehicle lamp frame, so as to determine whether two vehicle lamp frames belong to the same vehicle based on the vehicle lamp matching distance, and whether a bright spot region far away is a vehicle lamp; whether two vehicle lamp frames belong to the same vehicle is determined based on the vehicle lamp matching distance, so as to further determine whether the bright spot region is a vehicle lamp; two vehicle lamp frames corresponding to the same vehicle are tracked, so as to track vehicle lamps close to each other; therefore, the accuracy of vehicle lamp classification can be improved, so as to further improve the accuracy and reliability of night vehicle lamp recognition, thereby providing necessary conditions for subsequent automatic switching of vehicle light sources, such as switching a high beam to a low beam, and preventing the high beam of the vehicle from dazzling other drivers.

[0147] In the embodiment, a night vehicle lamp recognition device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0148] The embodiment provides a night vehicle lamp recognition device, as shown in the following Figure 6 The device comprises:

[0149] The device comprises:

[0150] In some optional embodiments, the image acquisition module 601 comprises:

[0151] The information acquisition unit is configured to acquire a source image and extinction point coordinates of the source image.

[0152] The region intercepting unit is configured to intercept a vehicle lamp to-be-processed region from the source image with the extinction point coordinates as the center.

[0153] The noise filtering unit is configured to filter noise points in the vehicle lamp to-be-processed region to obtain a vehicle lamp to-be-processed image.

[0154] The binarization processing module 602 is configured to determine a binarization image of the vehicle lamp to-be-processed image based on the adaptive segmentation threshold.

[0155] In some optional embodiments, the binarization processing module 602 includes:

[0156] The pixel acquisition unit is configured to acquire pixel values of each pixel in the vehicle lamp to-be-processed image and an adaptive segmentation threshold corresponding to each pixel.

[0157] Optionally, in the process of acquiring the adaptive segmentation threshold, first, target neighborhood pixels corresponding to the current pixel and neighborhood pixel values of the target neighborhood pixels are acquired; the mean value of the target neighborhood pixels and the amplitude variation rate of the target neighborhood pixels are determined based on the neighborhood pixel values; and the adaptive segmentation threshold is determined based on the mean value of the target neighborhood pixels and the amplitude variation rate of the target neighborhood pixels.

[0158] The binarization processing unit is configured to determine a binarization image of the vehicle lamp to-be-processed image based on a difference between the pixel value of each pixel and the adaptive segmentation threshold.

[0159] The lamp frame expansion module 603 is configured to expand a vehicle lamp frame based on a minimum bounding box of each connected domain in the binarization image to form a vehicle lamp recognition candidate set.

[0160] In some optional embodiments, the lamp frame expansion module 603 includes:

[0161] The minimum bounding box set creation unit is configured to create a minimum bounding box set based on the minimum bounding box of each connected domain in the binarization image.

[0162] The element adjustment unit is configured to place elements with a luminance value greater than a luminance threshold in the minimum bounding box set in a target rectangle set.

[0163] The frame center acquisition unit is configured to acquire a frame center of each minimum bounding box in the target rectangle set.

[0164] The bounding box expansion unit is configured to expand the minimum bounding box with the frame center as the origin according to a preset length-width ratio to obtain the vehicle lamp recognition candidate set.

[0165] The lamp frame classification module 604 is configured to place a vehicle lamp frame with a classification confidence greater than a classification threshold in the vehicle lamp recognition candidate set in a target vehicle lamp set.

[0166] In some optional embodiments, the lamp frame classification module 604 includes:

[0167] The confidence acquisition unit inputs the vehicle lamp frame in the vehicle lamp recognition candidate set into a target vehicle lamp classification model to obtain classification confidence of each vehicle lamp frame.

[0168] Optionally, when the target vehicle lamp classification model is acquired, first, sample data and labels of the sample data are acquired, the labels including vehicle lamp frames and reflection regions; the sample data are input into the vehicle lamp classification model to obtain predicted classification confidence; and based on the predicted classification confidence and the labels, parameters of the vehicle lamp classification model are updated to determine the target vehicle lamp classification model.

[0169] The lamp frame adjustment unit is configured to place the vehicle lamp frame with the classification confidence greater than the classification threshold in the target vehicle lamp set.

[0170] The vehicle lamp recognition module 605 is configured to recognize a vehicle lamp type corresponding to the vehicle lamp frame based on the center coordinates and the color space information of each vehicle lamp frame in the target vehicle lamp set; and the vehicle lamp type includes a same-direction oncoming vehicle lamp and an opposite-direction oncoming vehicle lamp.

[0171] In some optional embodiments, the apparatus further includes:

[0172] The lamp frame tracking module is configured to determine a vehicle lamp matching distance based on the extinguishing point coordinates and the center coordinates of the vehicle lamp frame; determine whether two vehicle lamp frames belong to the same vehicle based on the vehicle lamp matching distance; and track the two vehicle lamp frames corresponding to the same vehicle.

[0173] The night vehicle lamp recognition apparatus in the embodiment is presented in the form of functional units, and the units refer to ASIC circuits, processors and memories for executing one or more software or fixed programs, and / or other devices capable of providing the above functions.

[0174] Further function descriptions of the above modules and units are the same as those of the corresponding embodiments, and will not be described here.

[0175] The embodiment of the application further provides a computer device having the above Figure 6 night vehicle lamp recognition apparatus.

[0176] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of a computer device provided by an optional embodiment of the application, as Figure 7As shown, the computer device includes one or more processors 10, memory 20, and interfaces 30 for the various components to communicate with one another. The various components communicate through the use of the various buses, and can be mounted on a common motherboard or in other manners as appropriate. The processor 10 can process instructions for execution within the computer device, including instructions stored in the memory 20 or elsewhere to implement routines for displaying graphical information, such as a GUI on an external input / output device, such as a display device coupled to the interface 30. In some embodiments, multiple processors and / or multiple buses can be employed as appropriate, as will be appreciated by those skilled in the art. Additionally, various components of the computer device can be used for processing instructions according to the embodiments, as will be appreciated. The computer device can be one of a plurality of computer devices in communication with one another through a network, as is common in an enterprise setup, for example. Figure 7 The processor 10 is taken as an example.

[0177] The processor 10 can be a central processing unit, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0178] The memory 20 stores instructions that are executable by the at least one processor 10, to cause the at least one processor 10 to perform the methods illustrated in the above embodiments.

[0179] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created by the use of the computer device according to the presentation of a small program landing page, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some alternative embodiments, the memory 20 can optionally include a memory that is remotely located with respect to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a server cluster, a mobile communication network, and a combination thereof.

[0180] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk. The memory 20 can further include a combination of the above-mentioned kinds of memories.

[0181] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0182] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or be implemented as computer codes stored in a remote storage medium or a non-transitory machine readable storage medium and stored in a local storage medium to be downloaded through a network, so that the method described herein can be processed by such software on a storage medium using a general computer, a special processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer codes, and when the software or computer codes are accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0183] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A night vehicle light recognition method, characterized by, The method comprises: acquiring a to-be-processed vehicle lamp image; determining a binary image of the to-be-processed vehicle lamp image based on an adaptive segmentation threshold value; extending a vehicle lamp frame based on a minimum bounding box of each connected domain in the binary image to form a vehicle lamp recognition candidate set; placing a vehicle lamp frame with a classification confidence greater than a classification threshold value in the vehicle lamp recognition candidate set into a target vehicle lamp set; recognizing a vehicle lamp type corresponding to the vehicle lamp frame based on center coordinates and color space information of each vehicle lamp frame in the target vehicle lamp set; wherein the vehicle lamp type comprises a same-direction oncoming vehicle lamp and an opposite-direction oncoming vehicle lamp; acquiring the adaptive segmentation threshold value, comprising: acquiring target neighborhood pixels corresponding to a current pixel and neighborhood pixel values of the target neighborhood pixels; determining a target neighborhood pixel mean value and a target neighborhood pixel amplitude change rate based on the neighborhood pixel values; determining the adaptive segmentation threshold value based on the target neighborhood pixel mean value and the target neighborhood pixel amplitude change rate.

2. The method of claim 1, wherein, acquiring the to-be-processed vehicle lamp image, comprising: acquiring a source image and vanishing point coordinates of the source image; cutting a vehicle lamp to-be-processed region from the source image with the vanishing point coordinates as the center; filtering noise points in the vehicle lamp to-be-processed region to obtain the to-be-processed vehicle lamp image.

3. The method according to claim 1 or 2, characterized in that, determining the binary image of the to-be-processed vehicle lamp image based on the adaptive segmentation threshold value, comprising: acquiring pixel values of each pixel in the to-be-processed vehicle lamp image and adaptive segmentation threshold values corresponding to each pixel; determining the binary image of the to-be-processed vehicle lamp image based on a difference between the pixel values of each pixel and the adaptive segmentation threshold values.

4. The method of claim 3, wherein, extending the vehicle lamp frame based on the minimum bounding box of each connected domain in the binary image to form the vehicle lamp recognition candidate set, comprising: creating a minimum bounding box set based on the minimum bounding box of each connected domain in the binary image; placing elements with a luminance value greater than a luminance threshold value in the minimum bounding box set into a target rectangle set; acquiring frame centers of each minimum bounding box in the target rectangle set; extending the minimum bounding box according to a preset length-width ratio with the frame center as the origin to obtain the vehicle lamp recognition candidate set.

5. The method of claim 4, wherein, placing the vehicle lamp frame with the classification confidence greater than the classification threshold value in the vehicle lamp recognition candidate set into the target vehicle lamp set, comprising: inputting the vehicle lamp frame in the vehicle lamp recognition candidate set into a target vehicle lamp classification model to obtain a classification confidence of each vehicle lamp frame; placing the vehicle lamp frame with the classification confidence greater than the classification threshold value into the target vehicle lamp set.

6. The method of claim 5, wherein, acquiring the target vehicle lamp classification model, comprising: acquiring sample data and labels of the sample data, the labels comprising vehicle lamp frames and reflective regions; inputting the sample data into a vehicle lamp classification model to obtain a predicted classification confidence; updating parameters of the vehicle lamp classification model based on the predicted classification confidence and the labels to determine the target vehicle lamp classification model.

7. The method of claim 2, wherein, The method further comprises: determining a vehicle lamp matching distance based on the vanishing point coordinates and the center coordinates of the vehicle lamp frame; determining whether two vehicle lamp frames belong to the same vehicle based on the vehicle lamp matching distance; tracking two vehicle lamp frames corresponding to the same vehicle.

8. A night vehicle light recognition device, characterized by comprising: The device comprises: a graph acquisition module configured to acquire a to-be-processed vehicle lamp image; The binarization processing module is configured to determine a binarization image of the to-be-processed vehicle lamp image based on an adaptive segmentation threshold value; the adaptive segmentation threshold value is obtained by: obtaining target neighborhood pixels corresponding to a current pixel and neighborhood pixel values of the target neighborhood pixels; determining a target neighborhood pixel mean value and a target neighborhood pixel amplitude variation rate based on the neighborhood pixel values; and determining the adaptive segmentation threshold value based on the target neighborhood pixel mean value and the target neighborhood pixel amplitude variation rate; The lamp frame expansion module is configured to expand a vehicle lamp frame based on a minimum circumscribed frame of each connected domain in the binarization image to form a vehicle lamp recognition candidate set; The lamp frame classification module is configured to place a vehicle lamp frame with a classification confidence greater than a classification threshold value in the vehicle lamp recognition candidate set into a target vehicle lamp set; The vehicle lamp recognition module is configured to recognize a vehicle lamp type corresponding to the vehicle lamp frame based on center coordinates and color space information of each vehicle lamp frame in the target vehicle lamp set; and the vehicle lamp type includes a same-direction oncoming vehicle lamp and an opposite-direction oncoming vehicle lamp.

9. A computer device, comprising: The computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the method in any one of claims 1 to 7. The computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, ​

Images