Image processing method and device, vehicle and storage medium

Abstract

The application discloses an image processing method and device, a vehicle and a storage medium, and relates to the technical field of vehicles. The method is applied to a vehicle, the vehicle comprises a camera, and the method comprises the following steps: acquiring an original image of a traveling direction of the vehicle collected by the camera; performing preprocessing on the original image based on a preset pixel value, so as to obtain a target position of a target light group in the original image; performing processing on the original image based on a preset algorithm, so as to obtain a road contour line and a skyline in the original image; acquiring a target relative position of the target position and the road contour line and the skyline; and filtering the target light group based on the target relative position. According to the method, the light group collected in the traveling direction of the vehicle is filtered based on the road contour line and the skyline, the accuracy of detecting the light group in the traveling direction of the vehicle is improved, and the experience of a user is improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and more specifically, to an image processing method, apparatus, vehicle, and storage medium. Background Technology

[0002] With the development of science and technology and the improvement of people's living standards, vehicles are becoming increasingly common in people's daily lives. Currently, intelligent high beams in vehicles use a narrow-angle camera mounted in front of the windshield to detect the presence of strong light clusters within the vehicle's field of vision to assist in judging oncoming traffic conditions, switching to low beams in a timely manner to avoid interfering with the vision of oncoming vehicles, and meeting the international standard requirement of 400 meters. However, in related technologies, because the identification of oncoming light sources based on their intensity, size, and position to determine the vehicle's direction of travel suffers from low accuracy in many low-light scenarios. Summary of the Invention

[0003] In view of the above problems, this application proposes an image processing method, apparatus, vehicle, and storage medium that can filter out light clusters in the direction of vehicle travel based on road contour lines and skylines, thereby improving the accuracy of detecting light clusters in the direction of vehicle travel and enhancing the user experience.

[0004] In a first aspect, embodiments of this application provide an image processing method applied to a vehicle, the vehicle including a camera, the method comprising: acquiring an original image of the vehicle's travel direction captured by the camera; preprocessing the original image based on preset pixel values ​​to obtain a target position of a target light cluster in the original image; processing the original image based on a preset algorithm to obtain a road outline and a skyline in the original image; obtaining the target position relative to the road outline and the skyline; and filtering the target light cluster based on the target relative position.

[0005] Secondly, embodiments of this application provide an image processing apparatus applied to a vehicle, the vehicle including a camera, the apparatus comprising: an image acquisition module, a light spot position acquisition module, a road outline and skyline acquisition module, a target relative position acquisition module, and a light spot filtering module. The image acquisition module is used to acquire an original image of the vehicle's travel direction captured by the camera; the light spot position acquisition module is used to preprocess the original image based on preset pixel values ​​to obtain the target position of a target light spot in the original image; the road outline and skyline acquisition module is used to process the original image based on a preset algorithm to obtain the road outline and skyline in the original image; the target relative position acquisition module is used to obtain the target relative position between the target position and the road outline and skyline; and the light spot filtering module is used to filter the target light spot based on the target relative position.

[0006] Thirdly, embodiments of this application provide a vehicle, including a camera, a memory, and a processor, wherein the camera is disposed in the vehicle and is used to acquire raw images of the vehicle's direction of travel; the memory is coupled to the processor and stores instructions, and when the instructions are executed by the processor, the processor performs the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be invoked by a processor to execute the above-described method.

[0008] The image processing method, apparatus, vehicle, and storage medium provided in this application are applied to a vehicle, which includes a camera. The method involves acquiring an original image of the vehicle's travel direction captured by the camera; preprocessing the original image based on preset pixel values ​​to obtain the target position of a target light cluster in the original image; processing the original image based on a preset algorithm to obtain the road outline and skyline in the original image; obtaining the target position relative to the road outline and skyline; and filtering the target light cluster based on the target relative position. This improves the accuracy of detecting light clusters in the vehicle's travel direction by filtering out light clusters acquired based on the road outline and skyline, and also enhances the user experience. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0011] Figure 2 A schematic diagram of the original image provided in one embodiment of this application is shown;

[0012] Figure 3 A schematic diagram of a binary image provided in an embodiment of this application is shown;

[0013] Figure 4 This illustration shows a schematic diagram of obtaining light clusters by performing connected component processing on a binary image according to an embodiment of this application;

[0014] Figure 5 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0015] Figure 6 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0016] Figure 7 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0017] Figure 8 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0018] Figure 9 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0019] Figure 10 A schematic diagram showing the target location, road outline, and skyline provided in one embodiment of this application is shown;

[0020] Figure 11 A schematic diagram showing the target location, road outline, and skyline provided in one embodiment of this application is shown;

[0021] Figure 12 A schematic flowchart of an image processing method provided in an embodiment of this application is shown;

[0022] Figure 13 A block diagram of an image processing apparatus provided in one embodiment of this application is shown;

[0023] Figure 14 A block diagram of a vehicle for performing an image processing method according to an embodiment of this application is shown;

[0024] Figure 15 A block diagram of a vehicle for performing an image processing method according to an embodiment of this application is shown;

[0025] Figure 16 An embodiment of the present application shows a storage unit for storing or carrying program code that implements the image processing method according to the embodiment of the present application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0027] Currently, vehicles identify light sources by sensing the intensity, size, and location of light clusters. However, due to the complexity and diversity of road conditions, the accuracy of light source identification is low in many vulnerable scenarios, such as misidentifying house lights, streetlights, and reflective signs as light sources for oncoming vehicles on slopes, curves, and overpasses.

[0028] To address the aforementioned problems, the inventors, through research, discovered and proposed the image processing method, apparatus, vehicle, and storage medium provided in the embodiments of this application. Based on road contour lines and skylines, the method filters out light clusters in the vehicle's direction of travel, improving the accuracy of detecting light clusters in the vehicle's direction of travel and enhancing the user experience. The specific image processing method will be described in detail in subsequent embodiments.

[0029] Please see Figure 1 , Figure 1 A schematic flowchart of an image processing method according to an embodiment of this application is shown. This image processing method filters out light clusters in the direction of vehicle travel based on road contours and skylines, improving the accuracy of detecting light clusters in the direction of vehicle travel and enhancing the user experience. In specific embodiments, this image processing method can be applied to, for example... Figure 13 The image processing device 200 and the vehicle 100 equipped with the image processing device 200 are shown. Figure 14 The following will use a vehicle as an example to illustrate the specific process of this embodiment. The vehicle may include a camera; however, it is understood that the vehicle used in this embodiment may include bicycles, motorcycles, robots, and other means of transportation, and is not limited thereto. The following will focus on... Figure 1 The process shown will be described in detail. The image processing method may specifically include the following steps:

[0030] Step S110: Obtain the original image of the vehicle's direction of travel captured by the camera.

[0031] In this embodiment of the application, the vehicle may include a camera that can capture original images of the vehicle's direction of travel; the number of cameras is not fixed and may be one, two, three, etc., and is not limited here.

[0032] Optionally, the camera that captures the original image of the vehicle's direction of travel can be a front-view camera or a side-view camera; this is not limited in this embodiment. The front-view camera is primarily mounted on the vehicle's windshield and is used for visual perception and recognition during driving. Based on function, it can be further divided into a front-view main camera, a front-view narrow-angle camera, and a front-view wide-angle camera. The number of front-view narrow-angle cameras is not fixed. Their main function is to identify targets such as traffic lights and pedestrians. They typically use narrow-angle lenses, around 30°-40°, and the pixel count is generally the same as that of the front-view main camera. This narrow-angle camera has higher pixel density and a longer detection distance, typically up to 250 m or even further. For example, the front-view narrow-angle camera has a field of view (FOV) of 28° and a minimum monitoring distance of at least 400 meters.

[0033] In some implementations, the vehicle can acquire raw images of its direction of travel captured by a camera from an associated cloud or electronic device via wireless communication technologies (such as WiFi, Bluetooth, ZigBEE, etc.). Alternatively, the vehicle can acquire these raw images from the camera via a serial communication interface (such as a serial LVDS interface). The method by which the vehicle acquires these raw images of its direction of travel is not limited here.

[0034] The original image can be understood as an image captured by the camera in the vehicle in the direction of the vehicle's travel. This image may include road conditions in the direction of the vehicle's travel, oncoming traffic conditions, and lights such as lights in the direction of the vehicle's travel, streetlights, signs, and other lights.

[0035] In one implementation, the processor in the vehicle can acquire the raw image of the vehicle's direction of travel captured by the vehicle's forward-facing narrow-angle camera.

[0036] Step S120: Preprocess the original image based on preset pixel values ​​to obtain the target position of the target light cluster in the original image.

[0037] In some implementations, after the vehicle acquires the original image of its travel direction captured by a camera, it can preprocess the original image based on preset pixel values ​​to obtain the target position of the target light cluster in the original image. Specifically, the vehicle may have preset pixel values. Furthermore, the vehicle can perform image segmentation, edge detection, image binarization, and connected component algorithm processing on the original image based on the preset pixel values ​​to obtain the target position of the target light cluster in the original image.

[0038] The target light cluster can be a light cluster in the original image captured by the camera during vehicle movement; the target light cluster can be a light cluster emitted by vehicle headlights, building lights, streetlights, or reflective signs, and is not limited here. The number of target light clusters in the original image can be one or more, and is not limited here.

[0039] As one implementation method, the vehicle can, based on a preset pixel value, i.e., a binarization threshold, traverse the differences between each pixel in each C channel of the original image and its surrounding pixels, thereby obtaining a binarized image of the original image. Here, each C channel in the original image can be understood as the white portion of the original image, or as the portions of the original image containing colors such as red, green, and blue. For an example, please refer to [link to example]. Figure 2 and Figure 3 , Figure 2 It shows the pixels of the C channel of the original image. In the original image, the white pixels represent the C channel pixels. Furthermore, Figure 3 The image shown is a binarized version of the original image obtained by traversing the differences between each pixel in the C channel and its surrounding pixels. Here, "0" represents white in the original image and "1" represents other colors in the original image.

[0040] Understandably, to avoid vehicles failing to switch to low beams in time when meeting oncoming traffic, thus interfering with the visibility of oncoming vehicles, and to meet the national standard requirement of 400 meters, it is common practice to determine whether there are strong glare clusters in the vehicle's direction of travel. To improve the accuracy and real-time performance of the vehicle's judgment of oncoming traffic conditions, this embodiment uses a GPU-accelerated algorithm for processing glare clusters in the image, thereby increasing image processing efficiency and reducing vehicle power consumption.

[0041] Furthermore, after obtaining the binarized image of the original image, the vehicle can perform light cluster detection on the image to obtain the target location of the target light cluster in the image. Specifically, the vehicle can perform light cluster detection on the binary image by applying a connected component detection algorithm to extract the size and position of the light cluster in the binary image.

[0042] For example, please refer to Figure 4 This diagram illustrates the size and location of light clusters in a binary image obtained by applying a connected component algorithm to the binary image. The dashed lines represent the light clusters within the binary image.

[0043] Optionally, the vehicle can identify all light clusters in the binarized image of the original image as target light clusters, or it can identify light clusters whose size is greater than a size threshold as target light clusters, or light clusters whose size is less than or equal to the size threshold as target light clusters, or it can identify target light clusters based on their positions; these are not limited here. The target position of the target light cluster can be the center position of each target light cluster, or it can be the position of a focal point of each target light cluster; these are not limited here.

[0044] Step S130: Process the original image based on a preset algorithm to obtain the road outline and skyline in the original image.

[0045] In some implementations, after the vehicle obtains the original image of its direction of travel captured by the camera, it can process the original image based on a preset algorithm to obtain the road outline and skyline in the original image. The road outline represents the boundary lines on both sides of the road; the skyline can represent the boundary line where the road disappears in the original image, or it can be understood as the boundary line between the ground and non-ground areas (such as the ground and sky, the ground and buildings, the ground and signs).

[0046] It is understandable that there are boundary lines on both sides of the road in a vehicle's travel segment. The corresponding road outline can represent the extent of the road. The road outline can be a straight line or a curve, without limitation here. The skyline represents the boundary between the ground and non-ground areas in the image. It can also serve as the boundary between the side closer to the road outline and the side farther away from the road outline. It can be a curve or a straight line, without limitation here.

[0047] In some implementations, a road contour detection model can be pre-set in the vehicle, which can identify the road contour in the original image and obtain the complete road contour in the original image.

[0048] For example, after inputting the original image into the road contour detection model, the road contour detection model performs grayscale conversion on the original image, denoises the image based on Gaussian smoothing, extracts the edges of the image based on the Canny algorithm, and finally obtains the road contour of the original image.

[0049] In some implementations, a skyline detection model can be pre-set in the vehicle, which can identify the skyline in the original image and obtain the complete skyline in the original image.

[0050] In some implementations, the camera that acquires the original image is configured with different calibration parameters depending on the vehicle size, the camera's installation location, and the model. The vehicle can then calculate the skyline in the original image based on the calibration parameters.

[0051] Step S140: Obtain the target relative position with the road outline and the skyline.

[0052] In some implementations, the vehicle can obtain the target position relative to the road outline and the skyline based on the target position of the obtained target light cluster, the road outline in the original image, and the skyline.

[0053] In some implementations, the vehicle can fuse the target location of the target light cluster, the road outline in the original image, and the skyline into the same coordinate system, i.e., the normalized plane, to obtain the target location relative to the road outline and the skyline.

[0054] In this context, within the same coordinate system, the target location of the target light cluster can be a (x, y) coordinate point within the two-dimensional normalized range of the original image. The road outline can be a curve or a straight line within the two-dimensional normalized range of the original image, and the skyline can be a straight line within the two-dimensional normalized range of the original image. Furthermore, obtaining the target's relative position to the road outline and the skyline within the same coordinate system can be achieved by calculating the shortest distance from the target location to the road outline and the positional relationship between the target location and the road outline and the skyline. The shortest distance from the target location to the road outline can be calculated using an algorithm for finding the shortest distance from a point to any curve. The positional relationship between the target location and the road outline and the skyline can be determined by whether the target location is within the road area represented by the road outline and whether the target location is on the side of the skyline closer to the road outline. The specific algorithm used to calculate the relative positional relationship is not limited.

[0055] Step S150: Filter the target light cluster based on the target's relative position.

[0056] In some implementations, after the vehicle obtains the target location relative to the road outline and the skyline, it can filter the target light cluster based on the target relative location.

[0057] Optionally, the vehicle can filter the target light cluster based on the target's relative position in the following ways: if the target position is on the side of the skyline that is far from the road outline, then the target light cluster is filtered out; or if the shortest distance from the target position to the road outline is greater than a preset distance, then the target light cluster is filtered out; or if the target position is outside the road area represented by the road outline and the shortest distance from the road outline is greater than a distance threshold, then the target light cluster is filtered out. No limitation is imposed here.

[0058] Understandably, the vehicle filters the target light cluster based on the relative position of the target. Based on the three elements of light source intensity, size, and position, the detection of the light cluster is optimized based on the road outline and skyline, which improves the accuracy of detecting light clusters in the direction of vehicle travel. This also improves the accuracy for users to judge whether there are oncoming traffic conditions in the direction of vehicle travel based on the light clusters in the original image after filtering out the target light clusters, thus improving the user experience.

[0059] An embodiment of this application provides an image processing method that acquires an original image of a vehicle's travel direction captured by a camera; preprocesses the original image based on preset pixel values ​​to obtain the target position of a target light cluster in the original image; processes the original image based on a preset algorithm to obtain the road outline and skyline in the original image; obtains the target position relative to the road outline and skyline; and filters the target light cluster based on the target relative position, thereby filtering out the light clusters in the vehicle's travel direction based on the road outline and skyline, improving the accuracy of detecting light clusters in the vehicle's travel direction and enhancing the user experience.

[0060] Please see Figure 5 , Figure 5 A schematic flowchart of an image processing method according to an embodiment of this application is shown. This method is applied to the aforementioned vehicle, which includes a camera. The following will focus on... Figure 5 The process shown will be described in detail. The image processing method may specifically include the following steps:

[0061] Step S210: Obtain the original image of the vehicle's direction of travel captured by the camera.

[0062] For a detailed description of step S210, please refer to step S110, which will not be repeated here.

[0063] Step S220: Binarize and process the connected components of the original image based on the preset pixel values ​​to obtain the target position of the target light cluster in the original image.

[0064] In some implementations, after acquiring the original image, the vehicle can binarize and perform connected component processing on the original image based on preset pixel values ​​to obtain the target position of the target light cluster in the original image. Specifically, the vehicle can binarize the C channel of the original image based on a preset pixel threshold to obtain a binary image; wherein the binary image includes the light cluster and the background (non-light cluster); and further, perform connected component processing on the binary image to obtain the target position and size of the target light cluster in the original image.

[0065] In some implementations, please refer to Figure 6Step S220 may include steps S221-S223.

[0066] Step S221: Binarize and process the connected components of the original image based on the preset pixel values ​​to obtain the size and position of the light cluster in the original image.

[0067] The vehicle can binarize the C-channel pixel values ​​of the original image based on a preset pixel threshold to obtain a binary image, which includes light clusters and background (non-light clusters). Furthermore, the vehicle can perform connected component processing on the binary image to obtain the size and location of the light clusters in the original image.

[0068] The position of the light cluster can be determined based on the position of the light cluster in the original image and the size of the light cluster. As one implementation, the center position of the determined light cluster size can be used as the position of the light cluster, or a focal position of the determined light cluster size can be used as the position of the light cluster.

[0069] Step S222: Determine a light cluster from the original image whose size is larger than a preset size, and designate it as the target light cluster.

[0070] The vehicle can be pre-set with a preset size, which can be used as a criterion for determining whether a light cluster is a target light cluster or a strong light cluster. For example, if the size of the light cluster is larger than the preset size, then the light cluster is determined to be a target light cluster and a strong light cluster.

[0071] Furthermore, after the vehicle acquires the size of the light cluster in the original image, it can compare the size of the light cluster with a preset size, and determine the light cluster in the original image whose size is larger than the preset size as the target light cluster.

[0072] Step S223: Determine the position of the target light cluster from the position of the light cluster in the original image as the target position.

[0073] In this process, after the vehicle acquires the position of the light cluster in the original image and determines the target light cluster in the original image, it can determine the position of the target light cluster from the position of the light cluster in the original image as the target position of the target light cluster.

[0074] Step S230: Process the original image based on a preset algorithm to obtain the road outline and skyline in the original image.

[0075] In some implementations, a road contour and skyline detection model is pre-set in the vehicle. After the vehicle obtains the original image, it can input the original image into the road contour and skyline detection model to obtain the road contour and skyline of the original image output by the road contour and skyline detection model.

[0076] In some implementations, please refer to Figure 7 Step S230 may include steps S231-S232.

[0077] Step S231: Process the original image based on a convolutional neural network to obtain the road outline in the original image.

[0078] In some implementations, the original image can be processed by the convolutional neural network to generate lane lines and road outlines in the original image.

[0079] Step S232: Calculate the skyline in the original image based on the calibration parameters of the camera.

[0080] In some implementations, the vehicle can use the calibration parameters of the camera to calculate the skyline in the original image after acquiring the raw image. Then, the initial skyline is offset by an offset threshold to obtain the final skyline.

[0081] In some implementations, a convolutional neural network model (e.g., generative adversarial network, recurrent neural network, etc.) that generates the skyline in an image based on the image can be pre-set in the vehicle. The vehicle can also generate the skyline corresponding to the original image through this convolutional neural network model.

[0082] Step S240: Obtain the target relative position with respect to the road outline and the skyline.

[0083] In some implementations, after the vehicle obtains the target location of the target light cluster, the road outline, and the skyline from the original image, it can acquire the shortest distance between the target location and the road outline, the positional relationship between the target location and the road outline, and the positional relationship between the target location and the skyline. Here, the target location can be understood as a point in the original image; the positional relationship between the target location and the road outline can be understood as whether the target location is within the area of ​​the road represented by the road outline; and the positional relationship between the target location and the skyline can be understood as whether the target location is within the sky area defined by the skyline in the original image.

[0084] In some implementations, please refer to Figure 8 Step S240 may include steps S241-S242.

[0085] Step S241: Map the target location, the road outline, and the skyline to the target coordinate system.

[0086] In some implementations, after the vehicle acquires the target location, road outline, and skyline, it can map these elements to a target coordinate system. This target coordinate system can be a coordinate system established by normalizing the original image in two dimensions, or it can be a single coordinate system into which the vehicle fuses the target location, road outline, and skyline based on the original image.

[0087] Step S242: Based on the target coordinate system, obtain the target relative position with the road outline and the skyline.

[0088] In some implementations, after the vehicle maps the target location, road outline, and skyline to the target coordinate system, it can obtain the target relative position between the target location and the road outline and skyline based on the target coordinate system.

[0089] Specifically, obtaining the target's relative position to the road outline and skyline based on the target coordinate system can involve obtaining the shortest distance between the target position and the road outline, the positional relationship between the target position and the road outline, and the positional relationship between the target position and the skyline. Here, the target position can be understood as a coordinate point in the target coordinate system; the positional relationship between the target position and the road outline can be understood as whether the target position is within the road area represented by the road outline; and the positional relationship between the target position and the skyline can be understood as whether the target position is within the non-ground area of ​​the original image where the skyline is defined.

[0090] In some implementations, please refer to Figure 9 Step S242 may include steps S2421-S2422.

[0091] Step S2421: Based on the target coordinate system, obtain the first relative position between the target position and the road outline, wherein the first relative position includes a first distance between the target position and the road outline, and the positional relationship between the target position and the road represented by the road outline, wherein the positional relationship includes the target position being within the road area or the target position being outside the road area.

[0092] The vehicle can obtain a first relative position between the target location and the road contour line based on the target coordinate system. This first relative position includes a first distance between the target location and the road contour line in the target coordinate system, and the positional relationship between the target location and the road represented by the road contour line. This positional relationship includes whether the target location is within the road area or outside the road area. The first distance can be the shortest distance from the target location to the road contour line, or it can be the average, median, or mode of the shortest distances from the target location to the road contour line representing the various road boundary lines; no limitation is imposed here.

[0093] For example, please refer to Figure 10 It shows the target location of the target light cluster in the original image, the road outline, and the skyline. The first distance (L) is the shortest distance from the target location to the road outline.

[0094] Step S2422: Based on the target coordinate system, obtain the second relative position between the target position and the skyline, wherein the second relative position includes the target position on the side of the skyline away from the road outline or the target position on the side of the skyline close to the road outline.

[0095] The vehicle can obtain a second relative position between the target position and the skyline based on the target coordinate system. The second relative position includes the target position being on the side of the skyline away from the road outline or the target position being on the side of the skyline closer to the road outline.

[0096] In this context, "target location on the side of the skyline away from the road outline" can be understood as the target location of the target light cluster being above the skyline, that is, the target location being in the non-ground area of ​​the original image divided by the skyline; "target location on the side of the skyline closer to the road outline" can be understood as the target location of the target light cluster being below the skyline, that is, the target location being in the ground area of ​​the original image divided by the skyline.

[0097] For example, please refer to Figure 11 It shows the target location of the target light cluster, the road outline, and the skyline in the original image.

[0098] Step S250: Filter the target light cluster based on the target's relative position.

[0099] In some implementations, after the vehicle obtains the relative position of the target, it can distinguish the light source corresponding to the target light cluster based on the relative position of the target, such as oncoming vehicle headlights, house lights, street lights, etc., and further filter the target light cluster based on the light source corresponding to the target light cluster.

[0100] Understandably, in the process of obtaining target light clusters by processing the original image based on preset pixel values, the electronic device filters the light clusters in the original image based on the light intensity and size of the light cluster, retaining those whose light intensity meets the preset pixel values ​​and whose size meets the preset size as target light clusters. Then, further, the target light clusters are filtered based on their target position relative to the road outline and the skyline. Based on the three elements of light intensity, size, and position, the light cluster detection is optimized based on the road outline curve and the skyline, which improves the judgment of the light source of the light clusters in the direction of vehicle travel, reduces the false detection rate of light source objects (such as streetlights, house lights, reflective signs, etc.) in the direction of vehicle travel, optimizes the judgment of oncoming traffic conditions in the direction of vehicle travel, and improves the user experience.

[0101] In some implementations, step S250 may include step S251.

[0102] Step S251: If the target location is on the side of the skyline away from the road outline, then the target light cluster is filtered out; or if the target location is on the side of the skyline close to the road outline, the target location is outside the road area and the first distance is greater than the distance threshold, then the target light cluster is filtered out.

[0103] In some implementations, the vehicle may filter the target light cluster based on the target's relative position by filtering out the target light cluster if the target position is on the side of the skyline away from the road outline; or by filtering out the target light cluster if the target position is on the side of the skyline close to the road outline, the target position is outside the road area, and the first distance is greater than a distance threshold.

[0104] In some implementations, if the target location is on the side of the skyline far from the road outline, it can be understood that the target location is in a non-ground area, and the light source of the target light cluster can be identified as a building lamp, street lamp, reflective sign, etc., that is, the target light cluster is identified as an invalid light cluster. If the target location is on the side of the skyline close to the road outline, the target location is outside the road area, and the first distance is greater than a distance threshold, the light source of the target light cluster can be identified as a building lamp, street lamp, reflective sign, etc., that is, the target light cluster is identified as an invalid light cluster. Furthermore, invalid light clusters are filtered out to avoid the influence of light emitted by building lamps, street lamps, reflective signs, etc., on the judgment of oncoming traffic conditions during vehicle movement, thereby improving the user experience.

[0105] Furthermore, if the target location is on the horizon closer to the road outline, it can be understood that the target location is on the ground, and if the target location is within or outside the road area at a distance less than or equal to a distance threshold, the light source of the target light cluster can be identified as the headlights of oncoming vehicles, that is, the target light cluster is identified as an invalid light cluster. This allows for judging the road conditions when vehicles are traveling based on the valid light clusters, timely switching to low beams to avoid interfering with the vision of oncoming vehicles, and meeting the national standard's 400-meter requirement, thus improving the user experience.

[0106] For example, please refer to Figure 12 The diagram illustrates a flowchart of an image processing method provided in one embodiment of this application. The method is applied to a vehicle, which includes a forward-looking narrow-angle camera. The vehicle can acquire a raw image of its travel direction captured by the forward-looking narrow-angle camera and calculate the skyline in the raw image based on preset parameters of the camera. The vehicle can also perform binarization and connected component detection on the raw image from the forward-looking narrow-angle camera using preset pixel values ​​to extract the size and position of the target light cluster in the raw image. Furthermore, a convolutional neural network is pre-installed in the vehicle, allowing it to generate lane lines and road contour lines in the raw image from the forward-looking narrow-angle camera. Further, the vehicle can fuse the target position of the target light cluster, lane lines, road contour lines, and skyline into the same coordinate system to obtain the target relative position of the target light cluster with respect to the skyline and road contour lines. Then, it determines whether the target location is on the side of the skyline away from the road outline. If so, it determines that the target light cluster is likely a street lamp, house lamp, decorative light, or reflective sign, that is, it determines that the target light cluster is invalid and filters the target light cluster. Alternatively, it determines whether the target location is on the side of the skyline closer to the road outline, and outside the road area represented by the road outline, and the shortest distance from the road outline is greater than a distance threshold. If so, it determines that the target light cluster is likely a street lamp, house lamp, decorative light, or reflective sign, that is, it determines that the target light cluster is invalid and filters the target light cluster. This improves the judgment of light sources in the direction of vehicle travel, reduces the false detection rate of light source objects (such as street lamps, house lamps, reflective signs, etc.) in the direction of vehicle travel, optimizes the judgment of oncoming traffic conditions in the direction of vehicle travel, and improves the user experience.

[0107] If the target location is on the side of the skyline close to the road outline, within the road area represented by the road outline, or the shortest distance from the road outline is less than or equal to the distance threshold, then the target light cluster is determined to be the headlight of an oncoming vehicle, that is, the target light cluster is determined to be a valid light cluster.

[0108] An embodiment of this application provides an image processing method, relative to... Figure 1The image processing method shown also performs binarization and connected component processing on the original image based on preset pixel values ​​to obtain the target location of the target light cluster in the original image. Compared to Figure 1 The image processing method shown in this embodiment can further optimize the detection of light clusters based on the road contour curve and skyline, in addition to the three elements of light cluster light intensity, size and position. This improves the judgment of light sources of light clusters in the direction of vehicle travel, and filters out light clusters that are invalid for judging the road conditions of oncoming vehicles, thereby reducing the false detection rate of light sources in the direction of vehicle travel, optimizing the judgment of the road conditions of oncoming vehicles, and improving the user experience.

[0109] Please see Figure 13 , Figure 13 A block diagram of an image processing apparatus according to an embodiment of this application is shown. The image processing apparatus 200 is applied to the aforementioned vehicle, which includes a camera. The following will describe... Figure 10 The process is described in detail below. The image processing device 200 includes: an image acquisition module 210, a light spot position acquisition module 220, a road outline and skyline acquisition module 230, a target relative position acquisition module 240, and a light spot filtering module 250, wherein:

[0110] The image acquisition module 210 is used to acquire the original image of the vehicle's direction of travel captured by the camera.

[0111] The light cluster location acquisition module 220 is used to preprocess the original image based on preset pixel values ​​to obtain the target location of the target light cluster in the original image.

[0112] The road outline and skyline acquisition module 230 is used to process the original image based on a preset algorithm to obtain the road outline and skyline in the original image.

[0113] The target relative position acquisition module 240 is used to acquire the target relative position with respect to the road outline and the skyline.

[0114] The light cluster filtering module 250 is used to filter the target light cluster based on the relative position of the target.

[0115] Furthermore, the light cluster location acquisition module 220 includes: a preprocessing module, wherein:

[0116] The preprocessing module is used to binarize and process the connected components of the original image based on the preset pixel values ​​to obtain the target position of the target light cluster in the original image.

[0117] Furthermore, the preprocessing module includes: a light cluster size acquisition unit, a target light cluster determination unit, and a target light cluster position determination unit, wherein:

[0118] The light cluster size acquisition unit is used to perform binarization and connected component processing on the original image based on the preset pixel value to obtain the size and position of the light cluster in the original image.

[0119] The target light cluster determination unit is used to determine light clusters in the original image whose size is greater than a preset size, and to designate them as the target light clusters.

[0120] The target light cluster position determination unit is used to determine the position of the target light cluster from the position of the light cluster in the original image as the target position.

[0121] Furthermore, the target relative position acquisition module 240 includes: a coordinate mapping module and a coordinate system relative position acquisition module, wherein:

[0122] The coordinate mapping module is used to transform the target location, the road outline, and the skyline into the target coordinate system.

[0123] The coordinate system relative position acquisition module is used to acquire the target position relative to the road outline and the skyline based on the target coordinate system.

[0124] Furthermore, the coordinate system relative position acquisition module includes: a first relative position acquisition unit and a second relative position acquisition unit, wherein:

[0125] The first relative position acquisition unit is used to acquire a first relative position between the target position and the road outline based on the target coordinate system, wherein the first relative position includes a first distance between the target position and the road outline, and the positional relationship between the target position and the road represented by the road outline, wherein the positional relationship includes the target position being within the road area or the target position being outside the road area.

[0126] The second relative position acquisition unit is used to acquire the second relative position between the target position and the skyline based on the target coordinate system, wherein the second relative position includes the target position on the side of the skyline away from the road outline or the target position on the side of the skyline close to the road outline.

[0127] Furthermore, the light cluster filtering module 250 includes: a first filtering unit and a second filtering unit, wherein:

[0128] A first filtering unit is configured to filter out the target light cluster if the target location is on the side of the skyline that is far from the road outline.

[0129] The second filtering unit is used to filter out the target light cluster if the target location is on the side of the skyline close to the road outline, the target location is outside the road range, and the first distance is greater than a distance threshold.

[0130] Furthermore, the road outline and skyline acquisition module 230 includes: a road outline acquisition unit and a skyline acquisition unit, wherein:

[0131] The road outline acquisition unit is used to process the original image based on a convolutional neural network to obtain the road outline in the original image.

[0132] The skyline acquisition unit is used to process the original image based on the preset calibration parameters of the camera to obtain the skyline in the original image.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0134] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0135] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0136] Please see Figure 14 This document illustrates a structural block diagram of a vehicle according to an embodiment of this application. The vehicle 100 can be a bicycle, electric vehicle, motorcycle, or other vehicle capable of running applications. The vehicle 100 in this application may include one or more components such as a camera 110, a processor 120, a memory 130, and one or more applications. The one or more applications may be stored in the memory 130 and configured to be executed by one or more processors 120. The one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0137] The camera 110 may include one or more cameras for capturing raw images of the vehicle 100 in its direction of travel. For example, the camera 110 may be a front-view camera, a side-view camera, or an in-vehicle camera of the vehicle 100, etc., and is not limited here.

[0138] For example, please refer to Figure 15 The diagram shows the installation location of the camera. Camera 101A can be installed in front of the windshield of vehicle 101 to capture raw images of the vehicle 101's direction of travel within the field of view in front of vehicle 101.

[0139] Processor 120 may include one or more processing cores. Processor 120 connects to various parts within vehicle 100 via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory 130, and calling data stored in memory 130 to perform various functions and process data within vehicle 100. Optionally, processor 120 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 120 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 120 and may be implemented separately using a communication chip.

[0140] The memory 130 may include random access memory (RAM) or read-only memory (ROM). The memory 130 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 130 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the vehicle 100 during use (such as phonebooks, audio and video data, chat log data, etc.).

[0141] Please see Figure 16This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 300 stores program code that can be invoked by a processor to execute the methods described in the above method embodiments.

[0142] The computer-readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 310 may be compressed, for example, in a suitable form.

[0143] In summary, the image processing method, apparatus, vehicle, and storage medium provided in this application acquire an original image of the vehicle's travel direction captured by a camera; preprocess the original image based on preset pixel values ​​to obtain the target position of the target light cluster in the original image; process the original image based on a preset algorithm to obtain the road outline and skyline in the original image; obtain the target position relative to the road outline and skyline; and filter the target light cluster based on the target relative position. The above method of this application filters out light clusters in the vehicle's travel direction based on the road outline and skyline, improving the accuracy of detecting light clusters in the vehicle's travel direction and enhancing the user experience.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, Applied to a vehicle, the vehicle including a camera, the method includes: Acquire the original image of the vehicle's direction of travel captured by the camera; The original image is preprocessed based on preset pixel values ​​to obtain the target position of the target light cluster in the original image. The original image is processed based on a preset algorithm to obtain the road outline and skyline in the original image; Obtain the target relative position with respect to the road outline and the skyline, wherein the target relative position includes the shortest distance from the target position to the road outline in the same coordinate system and the positional relationship between the target position and the road outline and the skyline, and the positional relationship includes whether the target position is within the road area represented by the road outline and whether the target position is on the side of the skyline closer to the road outline; The target light cluster is filtered based on its relative position.

2. The method according to claim 1, characterized in that, The step of preprocessing the original image based on preset pixel values ​​to obtain the target location of the target light cluster in the original image includes: Based on the preset pixel values, the original image is binarized and connected component processing is performed to obtain the target position of the target light cluster in the original image.

3. The method according to claim 2, characterized in that, The step of binarizing and processing the original image based on the preset pixel values ​​to obtain the target location of the target light cluster in the original image includes: Based on the preset pixel values, the original image is binarized and connected component processing is performed to obtain the size and position of the light cluster in the original image; A light cluster whose size is larger than a preset size is identified from the original image and designated as the target light cluster; The position of the target light cluster is determined from the position of the light cluster in the original image as the target position.

4. The method according to claim 1, characterized in that, The step of obtaining the target location relative to the road outline and the skyline includes: Map the target location, the road outline, and the skyline to the target coordinate system; Based on the target coordinate system, the target position relative to the road outline and the skyline is obtained.

5. The method according to claim 4, characterized in that, The step of obtaining the target position relative to the road outline and the skyline based on the target coordinate system includes: Based on the target coordinate system, a first relative position between the target location and the road outline is obtained, wherein the first relative position includes a first distance between the target location and the road outline, and the positional relationship between the target location and the road represented by the road outline, wherein the positional relationship includes the target location being within the road area or the target location being outside the road area; Based on the target coordinate system, a second relative position between the target position and the skyline is obtained, wherein the second relative position includes the target position on the side of the skyline away from the road outline or the target position on the side of the skyline close to the road outline.

6. The method according to claim 5, characterized in that, The filtering of the target light cluster based on the target's relative position includes: If the target location is on the side of the skyline that is far from the road outline, then the target light cluster is filtered out; or If the target location is on the side of the skyline closer to the road outline, the target location is outside the road area, and the first distance is greater than a distance threshold, then the target light cluster is filtered out.

7. The method according to any one of claims 1-6, characterized in that, The step of processing the original image based on a preset algorithm to obtain the road outline and skyline in the original image includes: The original image is processed using a convolutional neural network to obtain the road outline in the original image; The original image is processed based on the preset calibration parameters of the camera to obtain the skyline in the original image.

8. An image processing apparatus, characterized in that, Applied to a vehicle, the vehicle including a camera, the device includes: The image acquisition module is used to acquire the original image of the vehicle's direction of travel captured by the camera; The light cluster location acquisition module is used to preprocess the original image based on preset pixel values ​​to obtain the target location of the target light cluster in the original image; The road outline and skyline acquisition module is used to process the original image based on a preset algorithm to obtain the road outline and skyline in the original image; The target relative position acquisition module is used to acquire the target relative position with respect to the road outline and the skyline. The target relative position includes the shortest distance from the target position to the road outline in the same coordinate system and the positional relationship between the target position and the road outline and the skyline. The positional relationship includes whether the target position is within the road area represented by the road outline and whether the target position is on the side of the skyline closer to the road outline. A light cluster filtering module is used to filter the target light cluster based on the target's relative position.

9. A vehicle, characterized in that, The vehicles include: A camera is used to capture raw images of the vehicle's direction of travel; One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-7.

Images