Light detection method, electronic equipment and storage medium

Abstract

The invention discloses a light detection method, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target image; performing light detection on the target image to obtain an initial detection result of light in the target image; performing interferent detection on the target image to obtain a detection result of an interferent in the target image; correcting the initial detection result of the lamplight by using the detection result of the interferent to obtain a target detection result of the lamplight; and identifying light in the target image based on the target detection result. In this way, the light detection accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a light detection method, electronic device and storage medium. Background Art

[0002] When a vehicle is driving at night, it needs to turn on the lights. Different lights need to be turned on in different environments. For example, when there are no street lights and no vehicles around, the high beam lights need to be turned on to ensure a better field of vision; when there are vehicles on the road or when there is sufficient light, the low beam lights need to be turned on. Therefore, the vehicle needs to accurately detect the lights in the surrounding environment to switch between the high and low beam lights. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a light detection method, an electronic device and a storage medium, which can improve the accuracy of light detection.

[0004] To solve the above technical problems, the first aspect of the present application provides a light detection method, which includes: acquiring a target image; performing light detection on the target image to obtain an initial detection result of the light in the target image; and performing interference detection on the target image to obtain a detection result of the interference in the target image; using the detection result of the interference to correct the initial detection result of the light to obtain a target detection result of the light; and identifying the light in the target image based on the target detection result.

[0005] The process of performing light detection on a target image to obtain an initial detection result of the light in the target image includes: performing binarization processing on the target image to obtain an initial binarized image representing the light in the target image, and using the initial binarized image as an initial detection result.

[0006] The interference object detection is performed on the target image to obtain the detection result of the interference object in the target image, including: performing pre-segmentation processing on the target image to obtain the segmentation result representing the interference object in the target image, and using the segmentation result as the detection result of the interference object.

[0007] Among them, the initial detection result of the light is corrected by using the detection result of the interference object to obtain the target detection result of the light, including: obtaining the overlapping area in the initial detection result that overlaps with the detection result of the interference object, changing the first pixel value of the pixel point in the overlapping area to the second pixel value, and obtaining the target detection result; wherein the first pixel value represents the pixel point as a luminous area, and the second pixel value represents the pixel point as a non-luminous area.

[0008] Among them, based on the target detection result, identifying the light in the target image includes: performing a clustering operation based on the target detection result to obtain a clustering result of the light in the target image; and performing category detection on the clustering result to determine the category of the light.

[0009] Among them, the target detection result includes a target binary image; a clustering operation is performed based on the target detection result to obtain a clustering result of the light in the target image, including: taking a pixel point whose pixel value of each pixel point in the target binary image is a first pixel value as a target pixel point; respectively forming a pixel block of a plurality of continuous target pixels in each target area in the target direction to obtain at least one pixel block; determining a first distance between two pixel blocks belonging to the same target area, and determining whether there is an intersection area between two pixel blocks belonging to different target areas; based on the first distance and whether there is an intersection area, determining whether each pixel block in the target binary image belongs to the same category.

[0010] Wherein, determining the first distance between two pixel blocks belonging to the same target area includes: obtaining the first coordinate value of the last pixel point constituting the first pixel block in the target direction, and obtaining the second coordinate value of the first pixel point constituting the second pixel block in the target direction; wherein the first pixel block and the second pixel block are arranged in sequence along the target direction, and the first pixel block and the second pixel block are adjacent; and calculating the difference between the first coordinate value and the second coordinate value to obtain the first distance.

[0011] Wherein, determining whether there is an intersection area between two pixel blocks belonging to different target areas includes: obtaining the third coordinate value of each pixel point constituting the third pixel block in the target direction, and forming a first coordinate interval based on each third coordinate value; and obtaining the fourth coordinate value of each pixel point constituting the fourth pixel block in the target direction, and forming a second coordinate interval based on each fourth coordinate value; wherein the third pixel block and the fourth pixel block belong to different target areas, and the target areas to which the third pixel block and the fourth pixel block belong are adjacent; determining whether there is an intersection area between the first coordinate interval and the second coordinate interval.

[0012] Wherein, based on the first distance and whether there is an intersection area, determining whether each pixel block in the target binary image belongs to the same category includes: in response to the first distance being greater than a first distance threshold, determining that each pixel block belonging to the same target area in the target direction belongs to the same category; in response to the existence of an intersection area, determining that each pixel block belonging to different target areas in the target direction belongs to the same category.

[0013] In which, when it is determined that two pixel blocks belonging to the same target area in the target direction belong to the same category, and a pixel point with a pixel value of the second pixel value is included between the two pixel blocks, it is determined that the pixel point with the pixel value of the second pixel value belongs to the same category as the two pixel blocks.

[0014] Wherein, the target direction is a row direction, and the same target area is the same row; or the target direction is a column direction, and the same target area is the same column.

[0015] The clustering results include bounding boxes of lights of various categories; before performing category detection on the clustering results, the method further includes: enlarging the bounding box by a preset multiple to obtain an enlarged box; cropping the target image based on the position of the enlarged box to obtain a target image area where the enlarged box is located; performing category detection on the clustering results to determine the category of the light, including: classifying the target image area using an image classification model to obtain the category of the light.

[0016] Among them, after performing category detection on the clustering results and determining the category of the light, it also includes: determining whether the light is a valid light; in response to the light being a valid light, generating a control signal, and controlling the switching of the high and low beam lights based on the control signal.

[0017] Among them, determining whether the light is a valid light includes: obtaining predicted pixel coordinates; wherein the predicted pixel coordinates are predicted based on the actual pixel coordinates of the center point of the light in the target image at the previous moment using Kalman filtering; based on the difference between the actual pixel coordinates of the center point of the light in the target image at the current moment and the predicted pixel coordinates, determining whether the light is a valid light.

[0018] Wherein, determining whether the light is a valid light includes: determining a second distance between the light and the optical center of the camera based on the height of the bounding box of the light included in the clustering result; and determining whether the light is a valid light based on a relationship between the second distance and a second distance threshold.

[0019] Wherein, in response to the light being a valid light, a control signal is generated, and the switching of the high beam and the low beam is controlled based on the control signal, including: in response to the light being a valid light, a first control signal is generated, and the high beam is controlled to be turned off and the low beam is controlled to be turned on based on the first control signal; or, in response to the light being a valid light and the second distance being less than a second distance threshold, a second control signal is generated, and the high beam is controlled to be turned off and the low beam is controlled to be turned on based on the second control signal; or, in response to the light being a valid light and the second distance being greater than the second distance threshold, a third control signal is generated, and the high beam is controlled to be turned on and the low beam is controlled to be turned off based on the third control signal.

[0020] Among them, obtaining the target image includes: obtaining a wide-angle image captured by the camera in a wide-angle state and a narrow-angle image captured by the camera in a narrow-angle state; fusing the wide-angle image and the narrow-angle image to obtain a fused image; preprocessing the fused image to obtain a target image; wherein the preprocessing includes at least one of cropping and downsampling.

[0021] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor, the memory storing program instructions; the processor is used to execute the program instructions stored in the memory to implement the method provided by the first aspect above.

[0022] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed by a processor to implement the method provided by the first aspect above.

[0023] The beneficial effects of the present application are as follows: Different from the prior art, after acquiring the target image, the present application performs light detection and interference detection on the target image respectively, obtains the initial detection result of the light and the detection result of the interference, uses the detection result of the interference to correct the initial detection result of the light, obtains the target detection result of the light; finally, based on the target detection result, identifies the light in the target image. Since the target detection result is corrected, the accuracy of the target detection result is higher, and the light in the target image can be accurately identified to improve the accuracy of light detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of an implementation method of a light detection method provided by the present application;

[0025] Figure 2 It is a flowchart of an implementation method of step S14 provided by this application;

[0026] Figure 3 It is a flow chart of an implementation method of a clustering operation provided by the present application;

[0027] Figure 4 It is a flowchart of another implementation method of the light detection method provided by the present application;

[0028] Figure 5 It is a schematic diagram of the framework structure of an embodiment of an electronic device provided by the present application;

[0029] Figure 6 It is a schematic diagram of the framework structure of an implementation scheme of a computer-readable storage medium provided in this application. DETAILED DESCRIPTION

[0030] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] It should be noted that in the embodiments of the present application, there are descriptions involving "first", "second", etc., which are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0032] In the description of the application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise clearly and specifically limited. In the embodiment of the present application, all directional indications (such as up, down, left, right, front, back ...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their deformations are intended to cover non-exclusive inclusions. For example, the process, method, system, product or equipment comprising a series of steps or units is not limited to the steps or units listed, but optionally also includes the steps or units not listed, or optionally also includes other steps or units inherent to these processes, methods, products or equipment.

[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] See also Figure 1 , Figure 1 : is a flow chart of an embodiment of a light detection method provided by the present application, the method comprising:

[0035] S11: Acquire a target image.

[0036] In one embodiment, the present application is used to detect lights around a vehicle while the vehicle is driving; therefore, the target image may be acquired by using an image acquisition device on the vehicle while the vehicle is driving, or may be acquired by processing the acquired image.

[0037] In other implementations, the present application may not detect the lights around the vehicle, for example, the lights in a fixed place may be detected. In this case, the target image may be obtained by photographing the scene to be detected.

[0038] In a specific implementation, the target image is obtained by processing an image captured by a camera on the vehicle. Specifically, the camera acquires a wide-angle image in a wide-angle state and acquires a narrow-angle image in a narrow-angle state, and the wide-angle image and the narrow-angle image are fused to obtain a fused image; the fused image is preprocessed to obtain the target image; wherein the preprocessing includes at least one of cropping and downsampling.

[0039] Among them, the process of fusing the wide-angle image and the narrow-angle image may include: dividing the wide-angle image into multiple areas, the multiple areas include the area in front of the vehicle, the area on the left side of the vehicle, the area on the right side of the vehicle, the sky area, etc., replacing the area in front of the vehicle in the wide-angle image with the narrow-angle image to obtain a fused image. In this embodiment, considering that the wide-angle image can cover a relatively comprehensive area around the vehicle, but the image clarity of the area directly in front of the camera in the wide-angle image is poor, which is not conducive to subsequent light detection, while the image clarity of the area directly in front of the camera in the narrow-angle image is better, so the narrow-angle image is used to replace the area directly in front of the camera in the wide-angle image to improve the accuracy of subsequent light recognition.

[0040] Furthermore, in order to improve the efficiency of subsequent processing, the fused image may be subjected to at least one of cropping and downsampling to obtain a target image. It is understandable that in other implementations, the fused image may also be directly used as the target image.

[0041] S12: Performing light detection on the target image to obtain an initial detection result of the light in the target image; and performing interference detection on the target image to obtain a detection result of the interference in the target image.

[0042] In one embodiment, the same method may be used to perform light detection and interference detection on the target image. For example, a target recognition method may be used to respectively use light and interference as targets to obtain corresponding detection results.

[0043] In another embodiment, different detection methods may be used to detect light and interference in the target image. For example, a target recognition method may be used to detect light, and a semantic segmentation method may be used to detect interference. For another example, a binarization method may be used to detect light, determine the light in the target image, and a semantic segmentation method may be used to detect interference.

[0044] In a specific implementation, the target image is binarized to obtain an initial binarized image representing the light in the target image, and the initial binarized image is used as the initial detection result. The target image is pre-segmented to obtain a segmentation result representing the interference in the target image, and the segmentation result is used as the detection result of the interference.

[0045] Specifically, after obtaining the target image, the pixel value of each pixel in the target image is obtained to determine whether the pixel value of the pixel is greater than the pixel threshold. If so, the pixel is considered to belong to the luminous area, that is, the point belongs to the light emitted by the luminous object. Otherwise, the point is considered not to belong to the luminous area. The luminous area determined in this way may cause false detection. For example, reflective lane lines and reflective signs may be mistaken for luminous areas. Therefore, lane lines, drivable areas, poles, etc. are used as interferences, and the target image is segmented using a pre-trained semantic segmentation model to obtain a segmentation result that represents the interferences in the target image.

[0046] S13: using the detection result of the interference object to correct the initial detection result of the light, to obtain the target detection result of the light.

[0047] In one embodiment, the area where the interference object is located can be determined based on the interference object detection result; the area where the light is located can be determined based on the initial light detection result. It is determined whether part of the area where the light is located overlaps with the area where the interference object is located. If so, the overlapping area where the light is located is considered to be the interference object, not the real light-emitting area.

[0048] Specifically, an overlapping area in the initial detection result that overlaps with the detection result of the interference object is obtained, and the first pixel value of the pixel point in the overlapping area is changed to a second pixel value to obtain the target detection result; wherein the first pixel value represents the pixel point as a luminous area, and the second pixel value represents the pixel point as a non-luminous area.

[0049] In other embodiments, vehicles or street lights may be used as interference objects. The vehicle may be used as an interference object, or the headlights in the vehicle may be used as interference objects. The target image is segmented to obtain segmentation results of the headlights and street lights, and the overlapping area in the initial detection result that overlaps with the segmentation result is used as the real luminous area.

[0050] S14: Based on the target detection result, identify the light in the target image.

[0051] In one embodiment, the light in the target image can be identified based on the target detection result to obtain basic information of the light. The basic information of the light can include at least one of the position of the light, the type of the light, and whether the light is a valid light.

[0052] In one implementation, the position of the light in the target image may be determined based on the target detection result, and the position of the light in the three-dimensional world may be further determined based on the position of the light in the target image.

[0053] In another embodiment, a clustering operation may be performed based on the target detection result to obtain a clustering result of the lights in the target image; and a category detection may be performed on the clustering result to determine the category of the lights.

[0054] In other embodiments, the coordinates of the pixel center point corresponding to the center point of the light in the target image are determined based on the target detection result, and whether the light is a valid light is determined based on the coordinates of the pixel center point corresponding to the center point of the light in the target image.

[0055] In the above method, after acquiring the target image, the target image is respectively subjected to light detection and interference detection to obtain the initial light detection result and interference detection result, and the initial light detection result is corrected by using the interference detection result to obtain the target light detection result; finally, the light in the target image is identified based on the target detection result. Since the target detection result is corrected, the accuracy of the target detection result is higher, and the light in the target image can be accurately identified to improve the accuracy of light detection.

[0056] Please refer to Figure 2 and Figure 3 , Figure 2 is a flowchart of an implementation method of step S14 provided by this application, Figure 3 : is a flow chart of an implementation method of a clustering operation provided by the present application; based on the target detection result, identifying the light in the target image includes:

[0057] S241: Perform a clustering operation based on the target detection result to obtain a clustering result of the lights in the target image.

[0058] In one embodiment, the target detection result includes a target binary image, which is obtained by correcting the initial binary image using the interference detection result, and the initial binary image is obtained by binarizing the target image, in which white pixels represent luminous areas, black pixels represent non-luminous areas, the pixel value of the white pixel is the first pixel value, and the pixel value of the black pixel is the second pixel value. Please refer to the above-mentioned related description for the specific acquisition process of the target binary image, which will not be repeated here.

[0059] The clustering process specifically includes:

[0060] S3411: taking the pixel value of each pixel point in the target binary image as the first pixel value as the target pixel point.

[0061] That is to say, each white pixel in the target binary image is taken as the target pixel.

[0062] S3412: Grouping a plurality of continuous target pixel points in each target area in the target direction into a pixel block to obtain at least one pixel block.

[0063] In one embodiment, the target direction is a row direction or a column direction, that is, a row direction or a column direction of a target binary image; the target direction includes a plurality of target regions, and the target region is each row or each column of the target binary image. If the target direction is a row direction, the i-th target region is the i-th row in the target binary image. If the target direction is a column direction, the i-th target region is the i-th column in the target binary image.

[0064] For each target area, a plurality of continuous target pixels are grouped into a pixel block to obtain at least one pixel block. For example, if the first pixel, the second pixel, the fifth pixel, and the sixth pixel in the i-th row of the target binary image are all target pixels, the first pixel and the second pixel are grouped into a first pixel block, and the fifth pixel and the sixth pixel are grouped into a second pixel block. It can be understood that for each target area, at least one pixel block can be numbered and a pixel block set can be obtained.

[0065] S3413: Determine a first distance between two pixel blocks belonging to the same target area, and determine whether there is an intersection area between two pixel blocks belonging to different target areas.

[0066] In one embodiment, when there are multiple pixel blocks in the same target area, the first distances between the multiple pixel blocks can be determined respectively to determine whether the multiple pixel blocks belong to the same category. Alternatively, it can be determined in order whether two adjacent pixel blocks belong to the same category. If they do, the two adjacent pixel blocks are merged to obtain a new pixel block, and then it is determined whether the new pixel block and the next pixel block belong to the same category, and so on, until the pixel blocks in the same target area are traversed and the next pixel block is adjacent to the new pixel block.

[0067] The first distance between two pixel blocks can be obtained by the difference between the first coordinate value and the second coordinate value. Specifically, the first coordinate value of the last pixel point constituting the first pixel block in the target direction is obtained, and the second coordinate value of the first pixel point constituting the second pixel block in the target direction is obtained; wherein the first pixel block and the second pixel block are arranged in sequence along the target direction, and the first pixel block and the second pixel block are adjacent; the difference between the first coordinate value and the second coordinate value is used as the first distance. wherein the target direction is the row direction or the column direction, taking the row direction as the x direction and the column direction as the y direction as an example, the coordinate value of the pixel point in the target direction is the x coordinate value or the y coordinate value.

[0068] In one embodiment, when it is determined that two pixel blocks belonging to the same target area in the target direction belong to the same category, and a pixel point having a pixel value of the second pixel value is included between the two pixel blocks, it is determined that the pixel point having the pixel value of the second pixel value and the two pixel blocks belong to the same category. In other words, when a black pixel point is included between two white pixel blocks, and the two white pixel blocks belong to the same category, it is considered that the black pixel point and the two white pixel blocks belong to the same category.

[0069] In one embodiment, for two pixel blocks in different target areas, the third coordinate values ​​of each pixel point constituting the third pixel block in the target direction can be obtained, and a first coordinate interval is formed based on each third coordinate value; taking the third pixel block including 3 target pixel points as an example, the third coordinate values ​​of the 3 target pixel points are 3, 4, and 5 respectively, and the first coordinate interval formed is [3,5]. And, the fourth coordinate values ​​of each pixel point constituting the fourth pixel block in the target direction are obtained, and a second coordinate interval is formed based on each fourth coordinate value. The second coordinate interval is formed in the same manner as the first coordinate interval, which will not be repeated here; wherein the third pixel block and the fourth pixel block belong to different target areas, and the target areas to which the third pixel block and the fourth pixel block belong are adjacent, that is, the third pixel block and the fourth pixel block belong to two adjacent rows or columns. Determine whether there is an intersection area between the first coordinate interval and the second coordinate interval. Taking the first coordinate interval as [3,5] and the second coordinate interval as [2,4] as an example, there is an intersection area between the first coordinate interval and the second coordinate interval.

[0070] It can be understood that for each pixel block in different target areas, it can be determined whether any two pixel blocks have an intersection area, or it can be determined in sequence whether each pixel block in a target area has an intersection area with each pixel block in another target area. If it is determined that an intersection area exists, it is not necessary to determine whether the pixel block has an intersection area with other pixel blocks. For example, if the first pixel block in the first target area has an intersection area with the third pixel block in the second target area, it is not necessary to determine whether the first pixel block in the first target area has an intersection area with other pixel blocks in the second target area.

[0071] S3414: Based on the first distance and whether there is an intersection area, determine whether each pixel block in the target binary image belongs to the same category.

[0072] In one embodiment, in response to the first distance being greater than the first distance threshold, it is determined that the pixel blocks belonging to the same target area in the target direction belong to the same category; in response to the existence of the intersection area, it is determined that the pixel blocks belonging to different target areas in the target direction belong to the same category. Wherein the target direction is the row direction, the same target area is the same row; or the target direction is the column direction, the same target area is the same column.

[0073] After clustering, the position, area, center point, and bounding box of each category of light in the target image can be obtained.

[0074] S242: Perform category detection on the clustering results to determine the category of the light.

[0075] Since the lights of other vehicles and street lights need to be considered during the switching between high and low beams, but other types of lights do not need to be considered, the category of the light can be further determined after the presence of the light is detected. In one embodiment, a pre-trained image classification model can be used to perform category prediction based on clustering results to determine the category of the light. The categories of lights include, but are not limited to, "headlights", "taillights", "street lights" and "others".

[0076] In another embodiment, based on the bounding box of the light in the clustering result, the bounding box can be magnified by a preset multiple to obtain an enlarged frame; based on the position of the enlarged frame, the target image is cropped to obtain the target image area where the enlarged frame is located, and the target image area is classified using an image classification model to obtain the category of the light. It can be understood that for each category of light, a corresponding target image area can be obtained. The preset multiple can be set as needed, for example, 1.5 times, 2 times, etc.

[0077] In a specific implementation, the target direction is the row direction, and each row of the target binary image can be traversed to obtain an initial pixel block set, denoted as P = {p i|i=1,2,3,…}, where the ID of the corresponding pixel block is -i. Match the pixel blocks in the current row, calculate the difference between the first coordinate value of the last pixel point constituting the first pixel block in the row direction and the second coordinate value of the first pixel point constituting the second pixel block in the row direction, and obtain the first distance. If the first distance is less than the first distance threshold, it is considered that the first pixel block and the second pixel block match, and the two pixel blocks are combined to obtain a new pixel block. The ID of the new pixel block is the same as the ID of the first pixel block or the ID of the second pixel block. Continue to repeat the above operation, and so on, until the traversal of the set P is completed. Then determine whether there is an intersection area between the pixel blocks in the current row and the pixel blocks in the previous row. The judgment process refers to the above description and will not be repeated here. It should be noted that if the number of pixel blocks in the current row is greater than the number of pixel blocks in the previous row, then there must be a new "light spot" in the current row. If the number of pixel blocks in the current row is less than the number of pixel blocks in the previous row, it is considered that there is a "light spot" in the previous row that has been clustered. Furthermore, if the pixel blocks in the current row match the pixel blocks in the previous row, it means that the pixel blocks in the current row and the pixel blocks in the previous row belong to the same category, and the IDs of the two are changed to be consistent. In one embodiment, the IDs of the two can be changed to new preset values. After the current row is judged, the next row is used as the new current row, and the above steps are repeated until the entire target binary image is traversed.

[0078] See also Figure 4 , Figure 4 : is a flow chart of another embodiment of the light detection method provided by the present application, the method comprising:

[0079] S41: Acquire a target image.

[0080] S42: Performing light detection on the target image to obtain an initial detection result of the light in the target image; and performing interference detection on the target image to obtain a detection result of the interference in the target image.

[0081] S43: using the detection result of the interference object to correct the initial detection result of the light, to obtain the target detection result of the light.

[0082] S44: Perform a clustering operation based on the target detection result to obtain a clustering result of the lights in the target image.

[0083] S45: Perform category detection on the clustering results to determine the category of the light.

[0084] Among them, please refer to the aforementioned steps S11-S13 for steps S41-S43, and please refer to the aforementioned steps S241-S242 for steps S44-S45, which will not be repeated here.

[0085] S46: Determine whether the light is a valid light.

[0086] Considering that lights have a life cycle, only lights that meet the minimum life cycle requirements can be considered effective lights, and effective lights can have an impact on the switching of high and low beams. Therefore, after obtaining the light category, it is also possible to determine whether the light is effective.

[0087] It can be understood that in other embodiments, the step of determining that the light is a valid light may also occur after detecting that there is light in the current target image, that is, before classification.

[0088] In one embodiment, a Kalman filter method may be used to determine whether a light is a valid light. Specifically, a Kalman filter is used to predict the actual pixel coordinates of the center point of the light in the target image at the previous moment to obtain predicted pixel coordinates, and the predicted pixel coordinates are obtained. Based on the difference between the actual pixel coordinates of the center point of the light in the target image at the current moment and the predicted pixel coordinates, it is determined whether the light is a valid light. If the difference is within a preset range, the light detected according to the target image is determined to be a valid light, otherwise, the light is considered to be an invalid light.

[0089] For each type of light, let the pixel coordinate of the center point of the light be x=(C u ,C v ), and perform state prediction and position filtering through the following formula:

[0090] predict:

[0091] x k =Ax k-1 +Bu k

[0092]

[0093] renew:

[0094]

[0095] Among them, A is the state transfer matrix, B is the control matrix, H is the measurement matrix, P k is the prior estimate covariance matrix, Q is the process excitation noise covariance matrix, K is the gain matrix, z k Indicates the measured value, represents the prior state estimate at time k, x k represents the posterior state estimate at time k, x k-1 Represents the posterior state estimate at time k-1.

[0096] In another embodiment, it is also possible to determine whether a light is a valid light based on the distance of the light. Lights that are too far away have no reference significance and can be considered invalid lights. The distance of the light is the second distance between the light and the optical center of the camera. The second distance between the light and the optical center of the camera can be determined based on the height of the bounding box of the light included in the clustering result; based on the relationship between the second distance and the second distance threshold, it is determined whether the light is a valid light. If the second distance is less than the second distance threshold, it is determined that the light is a valid light; if the second distance is greater than the second distance threshold, it is determined that the light is not a valid light. The second distance can be specifically calculated using the following formula:

[0097] d=H / h*f

[0098] Where d is the second distance, f is the focal length of the camera, h is the height of the bounding box, and H is the actual scale of the light. The actual scale of the light can be obtained by checking the national standard or actual measurement.

[0099] In other implementations, the Kalman filter and the light distance may be combined to determine the effective light. Only when both conditions are met, the light is determined to be an effective light.

[0100] S47: In response to the light being a valid light, generating a control signal, and controlling the switching between the high and low beam lights based on the control signal.

[0101] In one embodiment, when the existence of effective light is detected according to the target image, in response to the light being effective light, a first control signal is generated, so that the controller controls the high beam to be turned off and the low beam to be turned on based on the first control signal; when the existence of effective light is detected according to the target image, a fourth control signal is generated, so that the controller controls the high beam to be turned on and the low beam to be turned off based on the fourth control signal. In this embodiment, a control signal can be generated only according to whether there is effective light to control the switching of high and low beams.

[0102] In other embodiments, a control signal may be generated according to whether there is a valid light and the second distance. In response to the light being a valid light and the second distance being less than the second distance threshold, a second control signal is generated, and the high beam is turned off and the low beam is turned on based on the second control signal; or, in response to the light being a valid light and the second distance being greater than the second distance threshold, a third control signal is generated, and the high beam is turned on and the low beam is turned off based on the third control signal.

[0103] After detecting the presence of lights in the target image, the present application will further classify the lights, determine the lights that can affect the switching of the vehicle's high and low beams, and then determine whether the lights are effective lights. Based on the effective lights finally determined, the switching of the high and low beams will be controlled, thereby achieving accurate control of the switching of the high and low beams.

[0104] See also Figure 5, Figure 5 It is a schematic diagram of the framework structure of an embodiment of an electronic device provided in the present application.

[0105] The electronic device 50 includes a memory 51 and a processor 52 coupled to each other, the memory 51 stores program instructions, and the processor 52 is used to execute the program instructions to implement the steps in any of the above method embodiments. Specifically, the electronic device 50 may include but is not limited to: a desktop computer, a laptop computer, a server, a mobile phone, a tablet computer, etc., which are not limited here.

[0106] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps in any of the above method embodiments. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.

[0107] See also Figure 6 , Figure 6 It is a schematic diagram of the framework structure of an implementation scheme of a computer-readable storage medium provided in this application.

[0108] The computer-readable storage medium 60 stores program instructions 61 , and when the program instructions 61 are executed by a processor, they are used to implement the steps in any of the above method embodiments.

[0109] The computer-readable storage medium 60 may specifically be a medium that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the computer program. The server may send the stored computer program to other devices for execution, or may also execute the stored computer program itself.

[0110] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0111] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A light detection method, characterized in that: include: Get the target image; Performing light detection on the target image to obtain an initial detection result of the light in the target image; and performing interference detection on the target image to obtain a detection result of the interference in the target image; Using the detection result of the interference object to correct the initial detection result of the light, to obtain the target detection result of the light; Based on the target detection result, the light in the target image is identified.

2. The method according to claim 1, characterized in that: The performing light detection on the target image to obtain an initial detection result of the light in the target image includes: Binarizing the target image to obtain an initial binary image representing the light in the target image, and using the initial binary image as the initial detection result; And / or, performing interference detection on the target image to obtain a detection result of the interference in the target image includes: The target image is pre-segmented to obtain a segmentation result representing the interference object in the target image, and the segmentation result is used as the detection result of the interference object.

3. The method according to claim 1, characterized in that The method of using the detection result of the interference object to correct the initial detection result of the light to obtain the target detection result of the light includes: Obtain an overlapping area in the initial detection result that overlaps with the detection result of the interference object, change the first pixel value of the pixel point in the overlapping area to a second pixel value, and obtain the target detection result; wherein the first pixel value represents that the pixel point is a luminous area, and the second pixel value represents that the pixel point is a non-luminous area.

4. The method according to claim 1, characterized in that: The step of identifying the light in the target image based on the target detection result includes: Performing a clustering operation based on the target detection result to obtain a clustering result of the lights in the target image; Perform category detection on the clustering result to determine the category of the light.

5. The method according to claim 4, characterized in that The target detection result includes a target binary image; The performing a clustering operation based on the target detection result to obtain a clustering result of lights in the target image includes: The pixel value of each pixel point in the target binary image is the pixel point with the first pixel value as the target pixel point; Respectively grouping a plurality of continuous target pixel points in each target area in the target direction into a pixel block to obtain at least one pixel block; Determine a first distance between two pixel blocks belonging to the same target area, and determine whether there is an intersection area between two pixel blocks belonging to different target areas; Based on the first distance and whether the intersection area exists, it is determined whether the pixel blocks in the target binary image belong to the same category.

6. The method according to claim 5, characterized in that The determining a first distance between two pixel blocks belonging to the same target area comprises: Obtaining the first coordinate value of the last pixel point constituting the first pixel block in the target direction, and, Acquire a second coordinate value of a first pixel point constituting a second pixel block in the target direction; wherein the first pixel block and the second pixel block are arranged in sequence along the target direction, and the first pixel block and the second pixel block are adjacent to each other; The difference between the first coordinate value and the second coordinate value is calculated to obtain the first distance.

7. The method according to claim 5, characterized in that The determining whether there is an intersection area between two pixel blocks belonging to different target areas comprises: Acquire the third coordinate value of each pixel point constituting the third pixel block in the target direction, and form a first coordinate interval based on each of the third coordinate values; and Acquire fourth coordinate values ​​of each pixel point constituting the fourth pixel block in the target direction, and form a second coordinate interval based on each of the fourth coordinate values; wherein the third pixel block and the fourth pixel block belong to different target areas, and the target areas to which the third pixel block and the fourth pixel block belong are adjacent; Determine whether the first coordinate interval and the second coordinate interval have the intersection area.

8. The method according to any one of claims 5 to 7, characterized in that: The determining, based on the first distance and whether the intersection area exists, whether the pixel blocks in the target binary image belong to the same category includes: In response to the first distance being greater than a first distance threshold, determining that each pixel block belonging to the same target area in the target direction belongs to the same category; In response to the existence of the intersection area, it is determined that the pixel blocks belonging to different target areas in the target direction belong to the same category.

9. The method according to claim 8, characterized in that When it is determined that two pixel blocks belonging to the same target area in the target direction belong to the same category, and the two pixel blocks include a pixel point with a pixel value of the second pixel value, it is determined that the pixel point with the pixel value of the second pixel value belongs to the same category as the two pixel blocks.

10. The method according to claim 5, characterized in that The target direction is a row direction, and the same target area is a same row; or the target direction is a column direction, and the same target area is a same column.

11. The method according to claim 4, characterized in that The clustering result includes the bounding box of the light of each category; Before performing category detection on the clustering result, the method further includes: Enlarging the boundary box by a preset multiple to obtain an enlarged frame; Based on the position of the magnifying frame, the target image is cropped to obtain a target image region where the magnifying frame is located; The performing category detection on the clustering result to determine the category of the light includes: The target image area is classified using an image classification model to obtain the category of the light.

12. The method according to claim 4, characterized in that After performing category detection on the clustering result to determine the category of the light, the method further includes: Determining whether the light is a valid light; In response to the light being a valid light, a control signal is generated, and switching between the high and low beam lights is controlled based on the control signal.

13. The method according to claim 12, characterized in that The step of determining whether the light is a valid light comprises: Acquire predicted pixel coordinates; wherein the predicted pixel coordinates are predicted by using Kalman filtering based on the actual pixel coordinates of the center point of the light in the target image at the previous moment; Based on the difference between the actual pixel coordinates of the center point of the light in the target image at the current moment and the predicted pixel coordinates, it is determined whether the light is a valid light.

14. The method according to claim 12 or 13, characterized in that The step of determining whether the light is a valid light comprises: Determine a second distance between the light and an optical center of a camera based on a height of a bounding box of the light included in the clustering result; Based on the relationship between the second distance and a second distance threshold, it is determined whether the light is a valid light.

15. The method according to claim 14, characterized in that In response to the light being a valid light, generating a control signal, and controlling the switching of the high and low beam lights based on the control signal, comprises: In response to the light being a valid light, generating a first control signal, and controlling the high beam to be turned off and the low beam to be turned on based on the first control signal; or, in response to the light being a valid light and the second distance being less than the second distance threshold, generating a second control signal, and controlling the high beam to be turned off and the low beam to be turned on based on the second control signal; Or, in response to the light being a valid light and the second distance being greater than the second distance threshold, a third control signal is generated, and based on the third control signal, the high beam is controlled to be turned on and the low beam is controlled to be turned off.

16. The method according to claim 1, characterized in that The acquiring of the target image comprises: Acquire a wide-angle image captured by the camera in a wide-angle state and a narrow-angle image captured by the camera in a narrow-angle state; fusing the wide-angle image and the narrow-angle image to obtain a fused image; The fused image is preprocessed to obtain the target image; wherein the preprocessing includes at least one of cropping and downsampling.

17. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions; The processor is used to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program instructions, and the program instructions can be executed by a processor to implement the method according to any one of claims 1 to 16.

Images