Image processing method, storage medium, controller, vehicle and program product

Abstract

The invention relates to an image processing method, a storage medium, a controller, a vehicle and a program product, and the method comprises the steps: determining a first depth image in front of a target vehicle; performing noise reduction processing on the first depth image to obtain a second depth image; and converting the second depth image into a three-dimensional point cloud, and determining a target obstacle in front of the target vehicle according to the three-dimensional point cloud. The noise in the first depth image can be eliminated from the two-dimensional image by performing noise reduction processing on the first depth image, so that when the second depth image is converted into the three-dimensional point cloud, most irrelevant point clouds can be filtered compared with direct processing of the point clouds in the related technology, and the accuracy of the point clouds is improved. Therefore, the time for generating the point cloud can be reduced, and the point cloud density of the three-dimensional point cloud can be reduced. And then the target obstacle is determined according to the three-dimensional point cloud, so that the speed of processing the three-dimensional point cloud and the efficiency of determining the target obstacle can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to an image processing method, a storage medium, a controller, a vehicle, and a program product. Background Art

[0002] Lurking AGV (Automated Guided Vehicle) mainly uses laser, infrared or other sensors for navigation, and can travel freely anywhere.

[0003] In the related art, a laser radar is used to obtain three-dimensional point cloud data in front of the latent AGV, and a target obstacle is determined based on the three-dimensional point cloud data, or a target detection method is used to determine the target obstacle in front of the latent AGV. Summary of the invention

[0004] The purpose of the present disclosure is to provide an image processing method, a storage medium, a controller, a vehicle and a program product to solve the technical problems existing in the above-mentioned related technologies.

[0005] In order to achieve the above object, the present disclosure provides an image processing method, comprising:

[0006] Determine a first depth image in front of the target vehicle;

[0007] Performing noise reduction processing on the first depth image to obtain a second depth image;

[0008] The second depth image is converted into a three-dimensional point cloud, and a target obstacle in front of the target vehicle is determined according to the three-dimensional point cloud.

[0009] Optionally, performing noise reduction processing on the first depth image to obtain a second depth image includes:

[0010] The first noise and / or the second noise in the first depth image are removed to obtain the second depth image, wherein the first noise is used to characterize the noise whose depth information of the pixel point in the first depth image is greater than the preset depth threshold, and the second noise is used to characterize the first depth image.

[0011] Optionally, removing first noise in the first depth image to obtain the second depth image includes:

[0012] Eliminate the first noise in the first depth image by the following calculation formula to obtain the second depth image;

[0013]

[0014] Wherein, d is the depth information of the pixel point of the first depth image, and depth(d) is the depth information of the pixel point of the second depth image.

[0015] Optionally, removing second noise in the first depth image to obtain the second depth image includes:

[0016] The second depth image is obtained by removing the second noise in the first depth image through at least one of a median filtering method, a pseudo-median filtering method and an abnormality detection method.

[0017] Optionally, converting the second depth image into a three-dimensional point cloud comprises:

[0018] Convert the second depth image into the three-dimensional point cloud by the following calculation formula;

[0019] z = depth(d) / depth_scale

[0020] x=(uc x )*z / f x

[0021] y=(vc y )*z / f y

[0022] Wherein, depth(d) is the depth information of the pixel of the second depth image, depth_scale is the preset depth scale, u is the u-axis pixel value of the pixel of the second depth image in the pixel coordinate system, v is the v-axis pixel value of the pixel of the second depth image in the pixel coordinate system, and f x is the scaling factor of the camera focal length on the x-axis, f y is the scaling factor of the camera focal length on the y-axis, c y is the coordinate of the optical center on the y-axis, c x is the optical center coordinate on the x-axis, and (x, y, z) is the point cloud coordinate of the three-dimensional point cloud.

[0023] Optionally, determining a target obstacle ahead of the target vehicle according to the three-dimensional point cloud includes:

[0024] Reducing the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud;

[0025] Eliminate the ground point cloud in the first sub-three-dimensional point cloud to obtain a second sub-three-dimensional point cloud;

[0026] The second sub-three-dimensional point cloud is subjected to denoising processing to obtain a target obstacle ahead of the target vehicle.

[0027] Optionally, reducing the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud includes:

[0028] The point cloud density of the three-dimensional point cloud is reduced by at least one of a random downsampling method, a voxel downsampling method, a curvature downsampling method and a uniform downsampling method to obtain the first sub-three-dimensional point cloud.

[0029] Optionally, removing the ground point cloud from the sub-three-dimensional point cloud to obtain a second sub-three-dimensional point cloud includes:

[0030] Determine a target point cloud in the first sub-3D point cloud whose y-axis coordinate value is greater than a preset height threshold, multiply each axial coordinate value of the target point cloud by 0, and multiply each axial coordinate value of the remaining point cloud by 1 to obtain the second sub-3D point cloud, where the remaining point cloud is the point cloud in the first sub-3D point cloud except the target point cloud.

[0031] Optionally, the determining of a target point cloud whose y-axis coordinate value in the first sub-three-dimensional point cloud is greater than a preset height threshold, multiplying each axial coordinate value of the target point cloud by 0, and multiplying each axial coordinate value of the remaining point cloud by 1, to obtain the second sub-three-dimensional point cloud, includes:

[0032] The second sub-three-dimensional point cloud is obtained by the following calculation formula;

[0033]

[0034] X=x1*y filter

[0035] Y=y1*y filter

[0036] Z=z1*y filter

[0037] Among them, (x1, y1, z1) is the point cloud coordinates in the sub-3D point cloud, (X, Y, Z) is the point cloud coordinates in the second sub-3D point cloud, y filter is a value of 0 or 1, n is an index value, and y[n] is the coordinate value of the point cloud in the sub-3D point cloud on the y-axis.

[0038] Optionally, performing point cloud denoising processing on the second sub-three-dimensional point cloud to obtain a target obstacle in front of the target vehicle includes:

[0039] The second sub-three-dimensional point cloud is denoised by at least one method selected from voxel filtering, statistical filtering and radius filtering to obtain a target obstacle ahead of the target vehicle.

[0040] Optionally, determining a first depth image in front of the target vehicle includes:

[0041] Acquire a first depth image in front of the target vehicle by a depth camera arranged in front of the target vehicle;

[0042] Alternatively, a first depth image in front of the target vehicle is acquired by a depth camera disposed behind the target vehicle.

[0043] Optionally, the image processing method further includes:

[0044] The driving path of the target vehicle is controlled according to the target obstacle.

[0045] Optionally, controlling the driving path of the target vehicle according to the target obstacle includes:

[0046] Generate an obstacle bounding box according to the target obstacle, where the obstacle bounding box is a minimum circumscribed rectangle of the target obstacle;

[0047] According to the obstacle bounding box, determine the target width and target center coordinate information, wherein the target width is the width of the plane of the obstacle bounding box closest to the target vehicle, and the target center coordinate information is the center point cloud coordinate of the plane of the obstacle bounding box closest to the target vehicle;

[0048] The driving path of the target vehicle is controlled by the target width and the target center coordinate information.

[0049] In a second aspect, the present disclosure provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods provided in the first aspect of the present disclosure.

[0050] In a third aspect, the present disclosure provides a controller, comprising:

[0051] a memory having a computer program stored thereon;

[0052] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods provided in the first aspect of the present disclosure.

[0053] In a fourth aspect, the present disclosure provides a vehicle, comprising the controller provided in the third aspect of the present disclosure.

[0054] A fifth method, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method provided in the first aspect of the present disclosure when executed by a processor.

[0055] Through the above technical solution, the second depth image is obtained by performing noise reduction processing on the first depth image in front of the target vehicle; then the second depth image is converted into a three-dimensional point cloud, and the target obstacle in front of the target vehicle is determined based on the three-dimensional point cloud. By performing noise reduction processing on the first depth image, the noise in the first depth image can be eliminated in the two-dimensional image, and then when the second depth image is converted into a three-dimensional point cloud, compared with directly processing the point cloud in the related technology, most of the irrelevant point clouds can be filtered, thereby reducing the time for generating point clouds and reducing the point cloud density of the three-dimensional point cloud. When the target obstacle is determined by the three-dimensional point cloud, the speed of processing the three-dimensional point cloud and the efficiency of determining the target obstacle can be improved.

[0056] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0058] Figure 1 is a schematic diagram showing an image processing method according to an exemplary embodiment of the present disclosure.

[0059] Figure 2 is a schematic diagram showing removal of the first noise according to an exemplary embodiment of the present disclosure.

[0060] Figure 3 is a schematic diagram showing median filtering according to an exemplary embodiment of the present disclosure.

[0061] Figure 4 is a schematic diagram showing statistical filtering according to an exemplary embodiment of the present disclosure.

[0062] Figure 5 is a schematic diagram showing an obstacle bounding box according to an exemplary embodiment of the present disclosure.

[0063] Figure 6 is a flowchart showing an image processing method according to an exemplary embodiment of the present disclosure.

[0064] Figure 7 is a flowchart showing an image processing method according to an exemplary embodiment of the present disclosure.

[0065] Figure 8 is a schematic diagram showing an image processing device according to an exemplary embodiment of the present disclosure.

[0066] Fig. 9 is a block diagram showing a vehicle according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0068] The inventor discovered that when the target obstacle in front of the latent AGV is obtained by processing the three-dimensional point cloud acquired by the laser radar, the three-dimensional point cloud data includes multiple point clouds that will not affect the travel of the latent AGV. Directly processing the acquired three-dimensional point cloud results in a large amount of processed data and a long processing time, which makes the efficiency of determining the target obstacle low. When the target obstacle in front of the latent AGV is determined by the target detection method, a large number of training samples are required to train the model. However, when an unseen object appears in front of the latent AGV, the object cannot be identified, which poses a great safety hazard.

[0069] In view of this, the present disclosure provides an image processing method, a storage medium, a controller, a vehicle, and a program product to solve the problems existing in the above-mentioned related technologies.

[0070] like Figure 1 As shown, Figure 1 is a schematic diagram showing an image processing method according to an exemplary embodiment of the present disclosure, referring to Figure 1 ,include:

[0071] S101: Determine a first depth image in front of a moving target vehicle;

[0072] S102: Perform noise reduction processing on the first depth image to obtain a second depth image;

[0073] S103: Convert the second depth image into a three-dimensional point cloud, and determine a target obstacle in front of the target vehicle according to the three-dimensional point cloud.

[0074] Through the above technical solution, the second depth image is obtained by performing noise reduction processing on the first depth image in front of the target vehicle; then the second depth image is converted into a three-dimensional point cloud, and the target obstacle in front of the target vehicle is determined based on the three-dimensional point cloud. By performing noise reduction processing on the first depth image, the noise in the first depth image can be eliminated in the two-dimensional image, and then when the second depth image is converted into a three-dimensional point cloud, compared with directly processing the point cloud in the related technology, most of the irrelevant point clouds can be filtered, thereby reducing the time for generating point clouds and reducing the point cloud density of the three-dimensional point cloud. Then, the target obstacle is determined based on the three-dimensional point cloud, which can improve the speed of processing the three-dimensional point cloud and the efficiency of determining the target obstacle.

[0075] In addition, compared with the method of detecting obstacles through target detection methods in related technologies, the technical solution provided by the present disclosure does not require a large number of training samples for model training, so that any obstacle can be identified, thereby reducing safety hazards during the movement of the target vehicle.

[0076] In order to enable those skilled in the art to better understand the image processing method provided by the present disclosure, the above steps are described in detail with examples below.

[0077] For example, the target vehicle may be a vehicle that needs to be automatically navigated. In the disclosed embodiment, the target vehicle may be a lurking AGV. The front of the movement may be the front of the front of the target vehicle when it is moving forward, or the front of the rear of the target vehicle when it is reversing. The first depth image in front of the target vehicle may be the first depth image in front of the front of the target vehicle when it is moving forward, or the first depth image in front of the rear of the target vehicle when it is reversing. The disclosed embodiment does not specifically limit this. For example, when the moving direction of the target vehicle is the direction in which the target vehicle is moving forward, the first depth image may be the depth image in front of the front of the target vehicle; when the moving direction of the target vehicle is the direction in which the target vehicle is reversing backward, the first depth image may be the depth image in front of the rear of the target vehicle.

[0078] During the driving process of the target vehicle, it is necessary to detect whether there is an obstacle in front of the target vehicle, and then the driving path of the target vehicle can be planned according to the target obstacle in front of the vehicle. Thus, a first depth image in front of the target vehicle can be determined, and then the target obstacle can be determined according to the first depth image. The first depth image can be a depth image containing the depth information of each pixel to the camera.

[0079] In a possible manner, determining a first depth image in front of the target vehicle includes:

[0080] Acquire a first depth image in front of the target vehicle by a depth camera arranged in front of the target vehicle;

[0081] Alternatively, a first depth image in front of the target vehicle is acquired by a depth camera disposed behind the target vehicle.

[0082] It should be understood that the first depth image can be acquired by a depth camera, and the depth camera can be set at the front of the target vehicle and the rear of the target vehicle at the same time, and then the depth camera for acquiring the first depth image can be determined according to the running front of the target vehicle. For example, when the moving front is the front of the front of the vehicle moving forward, the first depth image can be acquired by the depth camera set in front of the target vehicle, and when the moving front is the front of the rear of the vehicle moving backward, the first depth image can be acquired by the depth camera set behind the target vehicle. This embodiment of the disclosure does not make specific limitations on this.

[0083] For example, the noise reduction process may be a process of removing noise from the first depth image. In the first depth image, there may be noise that increases the time to determine the target obstacle or reduces the accuracy of determining the target obstacle. Therefore, the embodiment of the present disclosure performs noise reduction on the first depth image to obtain a second depth image, and then converts the second depth image into a three-dimensional point cloud for subsequent operations. Compared with directly processing the point cloud in the related art, a large number of irrelevant point clouds can be filtered out, thereby reducing the time to generate a three-dimensional point cloud and reducing the point cloud density of the three-dimensional point cloud. Among them, the noise reduction process can be at least one filtering method such as statistical filtering, median filtering, mean filtering, Gaussian filtering, etc., and the embodiment of the present disclosure does not make specific limitations on this.

[0084] In a possible manner, performing noise reduction processing on the first depth image to obtain the second depth image includes:

[0085] The first noise and / or the second noise in the first depth image are removed to obtain the second depth image, where the first noise is used to characterize noise in which depth information of pixels in the first depth image is greater than a preset depth threshold, and the second noise is used to characterize salt and pepper noise in the first depth image.

[0086] It should be understood that the first depth image may include a first noise and a second noise. The first noise may be noise that is irrelevant to determining the target obstacle. The first noise may be noise in which the depth information of a pixel in the first depth image is greater than a preset depth threshold. The preset depth threshold may be expressed as a critical threshold between a range area that affects the operation of the target vehicle and a range area that does not affect the operation of the target vehicle in front of the target vehicle. The second noise may be the salt and pepper noise that comes with the depth image when the first depth image is acquired. Salt and pepper noise may be bright or dim pixels that appear randomly in the first depth image.

[0087] In the embodiment of the present disclosure, the second depth image can be obtained by removing the first noise in the first depth image, the second depth image can be obtained by removing the second noise in the first depth image, or the second depth image can be obtained by removing the first noise and the second noise in the first depth image.

[0088] By removing the first noise and / or the second noise to obtain a second depth image, and then converting the second depth image into a three-dimensional point cloud for processing, compared with directly processing the acquired point cloud in the related art, irrelevant point clouds can be filtered out, thereby reducing the time for generating a three-dimensional point cloud and reducing the point cloud density in the three-dimensional point cloud.

[0089] It should be understood that for each pixel in the first depth image, when the depth information of the pixel is less than the preset depth threshold, it can be represented whether there is an obstacle at the pixel that will affect the driving path of the target vehicle. When the depth information of the pixel is greater than the preset depth threshold, it can be represented whether there is an obstacle at the pixel that will not affect the driving path of the target vehicle. The second depth image can be obtained by eliminating the pixels whose depth information is less than the preset depth threshold in the first depth image. Figure 2 As shown, Figure 2 In the figure, depth cameras 1 are provided at both ends of the target vehicle 2, and the two dotted boxes can represent the range where the second depth image is located. For example, the second depth image can be obtained by the following calculation formula:

[0090]

[0091] Wherein, d can be the depth information of the pixel point, and the preset depth threshold can be set according to user needs. By eliminating the pixel points whose depth information in the first depth image is less than the preset depth threshold, the second depth image can be obtained, and the noise information in the first depth image can be reduced. When the second depth image is converted into a three-dimensional point cloud, irrelevant point clouds can be filtered out, thereby reducing the time for generating a three-dimensional point cloud and reducing the point cloud density in the three-dimensional point cloud.

[0092] In a possible manner, removing the second noise in the first depth image to obtain the second depth image includes:

[0093] The second depth image is obtained by removing the second noise in the first depth image through at least one of a median filtering method, a pseudo-median filtering method and an abnormality detection method.

[0094] It should be understood that the median filtering method can be a method that uses the median of all pixels in the pixel domain to replace the grayscale value of the central pixel, and can be used to remove salt and pepper noise or impulse noise. The pseudo-median filtering method can be a method that requires weighted averaging of the median of the pixel domain based on the median filtering method. The anomaly detection method can be used to identify abnormal pixels or abnormal areas in an image.

[0095] When the sub-depth image is processed by median filtering, the process of median filtering can be expressed by the following expression:

[0096] filter_depth[n]=median(depth[nk],...,depth[n],...,depth[n+k])

[0097] Among them, depth is the depth information of the pixel, filter_depth is the depth information after median filtering, n is the current position, and k is the window size of the selected pixel.

[0098] For example, one can Figure 3 The processing is performed in the manner shown. When window k=3, a 3×3 window is arbitrarily selected. In the window of the 3×3 core, the depth information values ​​from top to bottom are 117, 56, 38, 201, 58, 69, 91, 75, 88, and the depth information is sorted from small to large as 38, 56, 58, 69, 75, 88, 91, 117, 201, and the median is 75, which is different from the middle value of the window of the 3×3 core. According to the median filtering method, 58 in the window of the 3×3 core is replaced by 75. Then, according to this method, the depth information of the pixels in all windows is subjected to median filtering, and the salt and pepper noise in the sub-depth image can be removed to obtain the second depth image.

[0099] By removing the first noise and / or the second noise in the first depth image to obtain the second depth image, and then converting the second depth image into a three-dimensional point cloud for processing, compared with directly processing the point cloud in the related art, most of the irrelevant point clouds can be filtered, thereby reducing the time for generating the three-dimensional point cloud and reducing the point cloud density of the three-dimensional point cloud. When the three-dimensional point cloud is used to confirm the target obstacle, the speed of processing the three-dimensional point cloud and the efficiency of determining the target obstacle can be improved.

[0100] For example, a three-dimensional point cloud can be a data set of a combination of discrete point coordinates in space. The second depth image in the two-dimensional plane can be converted into a three-dimensional point cloud, and then the target obstacle is determined based on the three-dimensional point cloud. Compared with the three-dimensional point cloud directly collected by the lidar in the related art, the three-dimensional point cloud filters out a large number of irrelevant point clouds, thereby reducing the time for generating the three-dimensional point cloud and reducing the point cloud density of the three-dimensional point cloud. When the three-dimensional point cloud is used to confirm the target obstacle, the processing speed of the three-dimensional point cloud and the efficiency of determining the target obstacle can be improved.

[0101] In a possible manner, when the second depth image is converted into a three-dimensional point cloud, the three-dimensional point cloud can be determined based on the camera intrinsic parameters, the depth information of each pixel in the second depth image, and the pixel value of each pixel in the pixel coordinate system. Among them, the camera intrinsic parameters can be camera focal length parameters and camera optical center parameters, which are not specifically limited in the embodiments of the present disclosure. The camera focal length parameter can be the distance measured between the imaging plane and the object. The camera optical center parameter can be the center point of the depth camera optical system. The camera intrinsic parameters can be the intrinsic parameters of the depth camera corresponding to the first depth image. For example, when it is determined that the depth camera of the first depth image is the depth camera of the front of the target vehicle, the camera intrinsic parameters can be the camera intrinsic parameters of the depth camera of the front of the target vehicle; when it is determined that the depth camera of the first depth image is the depth camera of the rear of the target vehicle, the camera intrinsic parameters can be the camera intrinsic parameters of the depth camera of the rear of the target vehicle.

[0102] For each pixel in the second depth image, when the second depth image is converted into a three-dimensional point cloud according to the camera intrinsic parameters, the depth information of each pixel in the second depth image, and the pixel value of each pixel in the pixel coordinate system, the conversion can be performed by the following calculation formula.

[0103] z=d / depth_scale

[0104] x=(uc x )*z / f x

[0105] y=(vc y )*z / f y

[0106] Wherein, depth_scale can be a preset depth scale, u can be the pixel value of the u axis of each pixel in the pixel coordinate system, v can be the pixel value of the v axis of each pixel in the pixel coordinate system, and f x Can be the scaling factor of the camera focal length on the x-axis, f y Can be the scaling factor of the camera focal length on the y-axis, c y can be the coordinate of the optical center on the y-axis, c x It can be the coordinate of the optical center on the x-axis, and (x, y, z) can be converted into point cloud coordinates in a three-dimensional coordinate system for each pixel point.

[0107] Then, the target obstacle in front of the target vehicle can be determined based on the three-dimensional point cloud obtained in the above manner.

[0108] Among possible ways, the target obstacle ahead of the target vehicle is determined based on the three-dimensional point cloud, including:

[0109] Reduce the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud;

[0110] Eliminate the ground point cloud in the first sub-3D point cloud to obtain the second sub-3D point cloud;

[0111] The second sub-3D point cloud is denoised to obtain the target obstacle in front of the target vehicle.

[0112] It should be understood that when the second depth image is converted into a three-dimensional point cloud, the point cloud density in the three-dimensional point cloud is still relatively complex, and it is necessary to reduce the point cloud density of the three-dimensional point cloud and improve the processing speed of the three-dimensional point cloud. In the embodiment of the present disclosure, the point cloud density of the three-dimensional point cloud can be reduced by a downsampling method to obtain a first sub-three-dimensional point cloud, thereby improving the speed of processing the first sub-three-dimensional point cloud and the efficiency of determining the target obstacle.

[0113] The first sub-3D point cloud includes both the point cloud of the obstacle to be determined and the ground point cloud. The ground point cloud can be the point cloud of the ground part included in the point cloud data. The ground point cloud has no effect on the driving path of the target vehicle, and the ground point cloud can be removed from the first sub-3D point cloud to obtain the second sub-3D point cloud. The second sub-3D point cloud may contain outlier point clouds, and the target obstacle can be obtained by denoising the second sub-3D point cloud.

[0114] The target obstacle is obtained by reducing the point cloud density of the three-dimensional point cloud, eliminating the ground point cloud of the first sub-three-dimensional point cloud, and denoising the second sub-three-dimensional point cloud. Compared with directly processing the acquired point cloud in the related art, the speed of point cloud processing can be improved and the efficiency of determining the target obstacle can be improved.

[0115] In a possible manner, reducing the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud includes:

[0116] The point cloud density of the three-dimensional point cloud is reduced by at least one of a random downsampling method, a voxel downsampling method, a curvature downsampling method and a uniform downsampling method to obtain the first sub-three-dimensional point cloud.

[0117] It should be understood that when reducing the point cloud density of the three-dimensional point cloud to obtain the first sub-three-dimensional point cloud, the first sub-three-dimensional point cloud can be obtained by processing through at least one method among random downsampling, voxel downsampling, curvature downsampling and uniform downsampling. Among them, random downsampling can be a method of downsampling the signal according to randomly selected sampling points to reduce the amount of data. Voxel downsampling can be a sampling method that reduces the complexity and storage space of the model by reducing the number of voxels in the three-dimensional model. Curvature downsampling can be a downsampling method when the curvature is small, thereby reducing the amount of data of the point cloud or surface model. Uniform downsampling can be sampling the data at fixed intervals or in a regular manner to reduce the amount of data.

[0118] When the point cloud density in the three-dimensional point cloud is reduced by a random downsampling method to obtain a first sub-three-dimensional point cloud, random downsampling can be performed according to a preset ratio. The preset ratio can be to randomly sample one pixel out of ten pixels, or it can be to randomly sample five pixels out of twenty pixels, and the embodiments of the present disclosure do not make specific limitations on this. When the three-dimensional point cloud is randomly downsampled according to a preset ratio, the point cloud density of the three-dimensional point cloud can be reduced, thereby increasing the speed of processing the first sub-three-dimensional point cloud and the efficiency of determining target obstacles. In other words, the first sub-three-dimensional point cloud is obtained by randomly downsampling the three-dimensional point cloud, and the point cloud density of the first sub-three-dimensional point cloud is less than the point cloud density of the three-dimensional point cloud, thereby increasing the speed of subsequent processing of the first sub-three-dimensional point cloud.

[0119] By reducing the point cloud density of the three-dimensional point cloud to obtain the first sub-three-dimensional point cloud, the speed of processing the first sub-three-dimensional point cloud can be increased when the first sub-three-dimensional point cloud is applied to subsequent processing steps, thereby improving the efficiency of determining the target obstacle.

[0120] In a possible manner, the removing of the ground point cloud in the sub-three-dimensional point cloud to obtain the second sub-three-dimensional point cloud includes:

[0121] Determine a target point cloud in the first sub-3D point cloud whose y-axis coordinate value is greater than a preset height threshold, multiply each axial coordinate value of the target point cloud by 0, and multiply each axial coordinate value of the remaining point cloud by 1 to obtain the second sub-3D point cloud, where the remaining point cloud is the point cloud in the first sub-3D point cloud except the target point cloud.

[0122] It should be understood that the preset height threshold may be a critical threshold for distinguishing whether the point cloud is a ground point cloud. The target point cloud may be a ground point cloud.

[0123] When the point cloud coordinates of the depth camera corresponding to the first depth image are positive numbers downward, the point cloud whose y-axis coordinate value in the point cloud coordinates is greater than the preset height threshold can belong to the ground point cloud, and the point cloud whose y-axis coordinate value in the point cloud coordinates is less than the preset height threshold can be a point cloud representing an obstacle. Then, the target point cloud whose y-axis coordinate value in the first sub-3D point cloud is greater than the preset height threshold can be determined, and each axial coordinate value in the target point cloud is multiplied by 0 respectively, so that all coordinate values ​​in the target point cloud can be changed to 0, and each axial coordinate value of the remaining point cloud is multiplied by 1 respectively to obtain the second sub-3D point cloud. After the multiplication operation, the coordinate values ​​of all point clouds in the target point cloud are 0, and the coordinate values ​​of the remaining point clouds are the same as the point cloud coordinate values ​​in the first sub-3D point cloud, and then the target point cloud is filtered out to obtain the second sub-3D point cloud. Among them, the calculation process can be expressed by the following calculation formula.

[0124]

[0125] X=x1*y filter

[0126] Y=y1*y filter

[0127] Z=z1*y filter

[0128] Among them, (x1, y1, z1) can be the target point cloud, (X, Y, Z) can be the second sub-3D point cloud, and y filter It can be a value of 0 or 1, and n can be an index value.

[0129] When the point cloud coordinates of the depth camera corresponding to the first depth image are positive upward, the point cloud whose y-axis coordinate value is less than the preset height threshold in the point cloud coordinates can belong to the ground point cloud, and the point cloud whose y-axis coordinate value is greater than the preset height threshold in the point cloud coordinates can be a point cloud representing an obstacle. The target point cloud whose y-axis coordinate value is less than the preset height threshold in the first sub-3D point cloud can be determined, and each axial coordinate value in the target point cloud can be multiplied by 0, so that all coordinate values ​​in the target point cloud can be changed to 0, and each axial coordinate value of the remaining point cloud can be multiplied by 1 to obtain the second sub-3D point cloud. After the multiplication operation, the coordinate values ​​of all point clouds in the target point cloud are 0, and the coordinate values ​​of the remaining point clouds are the same as the point cloud coordinate values ​​in the first sub-3D point cloud, and then the target point cloud is filtered out to obtain the second sub-3D point cloud.

[0130] By setting a preset height threshold to compare with the y-axis coordinate, and performing a product operation with each axial coordinate value of the corresponding point cloud according to the comparison result, the second sub-3D point cloud is obtained. Compared with directly processing the point cloud in the related art, the processing time of irrelevant point clouds is reduced, thereby improving the processing speed of the 3D point cloud and the efficiency of determining the target obstacle.

[0131] In a possible manner, performing point cloud denoising processing on the second sub-three-dimensional point cloud to obtain the target obstacle in front of the target vehicle includes:

[0132] The second sub-three-dimensional point cloud is denoised by at least one method selected from voxel filtering, statistical filtering and radius filtering to obtain a target obstacle in front of the target vehicle.

[0133] It should be understood that when the second sub-three-dimensional point cloud is denoised to obtain the target obstacle, the denoising process can be performed by at least one of voxel filtering, statistical filtering and radius filtering to obtain the target obstacle, and the embodiments of the present disclosure do not specifically limit this. Among them, voxel filtering can divide the point cloud into a regular voxel grid, and then calculate the average value or median of the points within the voxel to obtain a smooth point cloud. Statistical filtering can filter out abnormal points or outliers in the point cloud in order to calculate the statistical features of the points in the field. Statistical filtering can include filtering methods such as Gaussian filtering, mean rate and median filtering. Radius filtering can be a point cloud filtering method based on a distance range.

[0134] When the second sub-3D point cloud is denoised by statistical filtering, such as Figure 4 As shown, Figure 4 The left side of is the second sub-3D point cloud with outlier point cloud. Figure 4 The right side of is the target obstacle obtained after statistical filtering of the second sub-3D point cloud, and there is no outlier point cloud in the target obstacle. By processing the second sub-3D point cloud through statistical filtering to obtain the target obstacle, the outlier point cloud in the second sub-3D point cloud can be removed to prevent inaccurate judgment of the target obstacle, thereby improving the accuracy of determining the target obstacle.

[0135] In a possible manner, the image processing method further includes:

[0136] The driving path of the target vehicle is controlled according to the target obstacle.

[0137] It should be understood that when the target obstacle is determined to exist in front of the target vehicle through the image processing method, the target vehicle needs to avoid the target obstacle during driving. The driving path of the target vehicle can be controlled by the target obstacle. For example, when the target vehicle is moving forward in a straight line, the image processing method determines that the target vehicle has a target obstacle within a preset depth threshold range, then the target vehicle can avoid the target obstacle from the left or right side of the target obstacle and continue to drive forward. Then, according to the target obstacle, the driving path of the target vehicle can be controlled, thereby reducing safety hazards.

[0138] In a possible manner, controlling the driving path of the target vehicle according to the target obstacle includes:

[0139] Generate an obstacle bounding box according to the target obstacle, where the obstacle bounding box is a minimum circumscribed rectangle of the target obstacle;

[0140] According to the obstacle bounding box, determine the target width and target center coordinate information, wherein the target width is the width of the plane of the obstacle bounding box closest to the target vehicle, and the target center coordinate information is the center point cloud coordinate of the plane of the obstacle bounding box closest to the target vehicle;

[0141] The driving path of the target vehicle is controlled by the target width and the target center coordinate information.

[0142] It should be understood that the obstacle bounding box can be a geometric body in a three-dimensional space, which is used to form a minimum bounding box surrounding the target obstacle. The obstacle bounding box can be a cube or a sphere, which is not specifically limited in this embodiment. Then, the target width and target center coordinate information can be determined based on the obstacle encirclement, and the driving path of the target vehicle can be controlled through the target width and target center coordinate information.

[0143] When the obstacle bounding box is a cube, the target width may be the width of the plane closest to the target vehicle among the six planes of the obstacle bounding box. The target center coordinate information may be the center point cloud coordinates of the plane closest to the target vehicle among the six planes of the obstacle bounding box. The coordinate information on the target width may be determined based on the target center coordinate information, and then the driving path of the target vehicle may be controlled based on the target width and the target center coordinate information, thereby reducing safety hazards. Figure 5 As shown, the rectangular frame 6 above is the obstacle bounding box, and the quadrilateral 7 below can be the ground point cloud.

[0144] When the obstacle bounding box is a sphere, the target width may be the diameter of the sphere. The target center coordinate information may be the center point cloud coordinate of the point closest to the target vehicle. The coordinates on the target width may be determined based on the target center coordinate information, and the driving path of the target vehicle may be controlled based on the target width and the target center coordinate information.

[0145] By controlling the target vehicle's driving path through the target width and the target center point cloud coordinates, the target measurement can avoid obstacles and thus improve the accuracy of the target vehicle's driving path control.

[0146] Figure 6 is a flowchart of an image processing method according to another exemplary embodiment. Figure 6 As shown, the process of displaying the door closing countdown on the display system includes the following steps.

[0147] Step S601: acquiring a first depth image through a depth image.

[0148] Step S602: performing noise reduction processing on the first depth image to obtain a second depth image.

[0149] Step S603: Generate a three-dimensional point cloud according to the second depth image.

[0150] Step S604: reducing the density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud.

[0151] Step S605: Eliminate the ground point cloud to obtain a second sub-3D point cloud.

[0152] Step S606: Perform noise reduction processing on the second sub-3D point cloud to obtain the target Zhang Zhiwu.

[0153] Step S607: Generate an obstacle bounding box according to the target obstacle.

[0154] Step S608: Calculate the target width and the target center point cloud coordinates.

[0155] Step S609: Control the driving path of the target vehicle.

[0156] Figure 7 is a flowchart of an image processing method according to another exemplary embodiment. Figure 7In the figure, the image processing process of Figure (a) to Figure (g) is included. Figure (a) can be the first depth image obtained, in which the ground 5, the target obstacle 4 and the pixel 3 are included. After removing the first noise in the first depth image, the image in Figure (a) becomes the image in Figure (b). By comparing Figure (a) with Figure (b), it can be seen that the pixel 3 in Figure (b) is less than the pixel 3 in Figure (a). Then, the second noise in Figure (b) can be removed to obtain Figure (c). By comparing Figure (b) with Figure (c), it can be seen that the pixel 3 in Figure (c) is less than the pixel 3 in Figure (b). Then, Figure (c) can be converted into a three-dimensional point cloud, and the three-dimensional point cloud is subjected to noise reduction processing to obtain Figure (d), and the point cloud density of Figure (d) is less than the point cloud density of the three-dimensional point cloud. Then, the ground point cloud in Figure (d) can be extracted to obtain Figure (e). In Figure (e), there may be an outlier point cloud 8. By performing statistical filtering on Figure (e), the outlier point cloud 8 in Figure (e) can be removed to obtain Figure (f). Then, an obstacle bounding box 9 can be generated according to Figure (f) to obtain Figure (g). Finally, the target width and target center coordinate information can be calculated according to Figure (g), and the driving path of the target vehicle can be controlled according to the target width and target center information.

[0157] The specific implementation methods of the above-mentioned process steps have been illustrated in detail above and will not be repeated here. In addition, it should be understood that for the above-mentioned system embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited to the order of actions described above. Secondly, those skilled in the art should also know that the embodiments described above belong to preferred embodiments, and the steps involved are not necessarily required by the present disclosure.

[0158] In the above manner, the first depth image in front of the target vehicle is subjected to noise reduction processing to obtain a second depth image; the second depth image is then converted into a three-dimensional point cloud, and the target obstacle in front of the target vehicle is determined based on the three-dimensional point cloud. The driving path of the target vehicle is then controlled based on the target obstacle.

[0159] By performing noise reduction processing on the first depth image to obtain the second depth image, the first noise and the second noise can be removed in the two-dimensional plane image. When the second depth image is converted into a three-dimensional point cloud, compared with directly processing the acquired point cloud in the related art, the time for processing the three-dimensional point cloud can be reduced and the point cloud density of the three-dimensional point cloud can be reduced. Then, the target obstacle is determined by the three-dimensional point cloud, thereby improving the efficiency of the three-dimensional point cloud processing and the speed of determining the target obstacle. The target vehicle's driving path is controlled by the target obstacle, thereby improving the target vehicle's obstacle avoidance efficiency and planning the target vehicle's driving path.

[0160] Based on the same concept, this embodiment also provides an image processing device 800, referring to Figure 8 , Figure 8 is a schematic diagram showing an image processing device 800 according to an exemplary embodiment of the present disclosure, such as Figure 8 As shown, including:

[0161] A first determination module 801 is used to determine a first depth image in front of the target vehicle;

[0162] A first noise reduction processing module 802, configured to perform noise reduction processing on the first depth image to obtain a second depth image;

[0163] The conversion module 803 is used to convert the second depth image into a three-dimensional point cloud, and determine the target obstacle in front of the target vehicle according to the three-dimensional point cloud.

[0164] Optionally, the first noise reduction processing module 802 is used to:

[0165] The first noise and / or the second noise in the first depth image are removed to obtain the second depth image, where the first noise is used to characterize noise in which depth information of pixels in the first depth image is greater than a preset depth threshold, and the second noise is used to characterize salt and pepper noise in the first depth image.

[0166] Optionally, the first noise reduction processing module 802 is further configured to:

[0167] Eliminate the first noise in the first depth image by the following calculation formula to obtain the second depth image;

[0168]

[0169] Wherein, d is the depth information of the pixel point of the first depth image, and depth(d) is the depth information of the pixel point of the second depth image.

[0170] Optionally, the first noise reduction processing module 802 is further configured to:

[0171] The second depth image is obtained by removing the second noise in the first depth image through at least one of a median filtering method, a pseudo-median filtering method and an abnormality detection method.

[0172] Optionally, the conversion module 803 is used to:

[0173] Convert the second depth image into the three-dimensional point cloud by the following calculation formula;

[0174] z = depth(d) / depth_scale

[0175] x=(uc x )*z / f x

[0176] y=(vc y )*z / f y

[0177] Wherein, depth(d) is the depth information of the pixel of the second depth image, depth_scale is the preset depth scale, u is the u-axis pixel value of the pixel of the second depth image in the pixel coordinate system, v is the v-axis pixel value of the pixel of the second depth image in the pixel coordinate system, and f x is the scaling factor of the camera focal length on the x-axis, f y is the scaling factor of the camera focal length on the y-axis, c y is the coordinate of the optical center on the y-axis, c x is the optical center coordinate on the x-axis, and (x, y, z) is the point cloud coordinate of the three-dimensional point cloud.

[0178] Optionally, the conversion module 803 includes:

[0179] A point cloud processing module, used to reduce the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud;

[0180] A removal module, used for removing the ground point cloud in the first sub-three-dimensional point cloud to obtain a second sub-three-dimensional point cloud;

[0181] The second noise reduction processing module is used to perform noise reduction processing on the second sub-three-dimensional point cloud to obtain a target obstacle in front of the target vehicle.

[0182] Optionally, the point cloud processing module is further used for:

[0183] The point cloud density of the three-dimensional point cloud is reduced by at least one of a random downsampling method, a voxel downsampling method, a curvature downsampling method and a uniform downsampling method to obtain the first sub-three-dimensional point cloud.

[0184] Optionally, the rejection module is further used for:

[0185] Determine a target point cloud in the first sub-3D point cloud whose y-axis coordinate value is greater than a preset height threshold, multiply each axial coordinate value of the target point cloud by 0, and multiply each axial coordinate value of the remaining point cloud by 1 to obtain the second sub-3D point cloud, where the remaining point cloud is the point cloud in the first sub-3D point cloud except the target point cloud.

[0186] Optionally, the rejection module is further used for:

[0187] The second sub-three-dimensional point cloud is obtained by the following calculation formula;

[0188]

[0189] X=x1*y filter

[0190] Y=y1*y filter

[0191] Z=z1*y filter

[0192] Among them, (x1, y1, z1) is the point cloud coordinates in the sub-3D point cloud, (X, Y, Z) is the point cloud coordinates in the second sub-3D point cloud, y filter is a value of 0 or 1, n is an index value, and y[n] is the coordinate value of the point cloud in the sub-3D point cloud on the y-axis.

[0193] Optionally, the second noise reduction processing module is further used for:

[0194] The second sub-three-dimensional point cloud is denoised by at least one method selected from voxel filtering, statistical filtering and radius filtering to obtain a target obstacle in front of the target vehicle.

[0195] Optionally, the first determining module 801 is used to:

[0196] Acquire a first depth image in front of the target vehicle by a depth camera arranged in front of the target vehicle;

[0197] Alternatively, a first depth image in front of the target vehicle is acquired by a depth camera disposed behind the target vehicle.

[0198] Optionally, the image processing device 800 further includes:

[0199] A control module is used to control the driving path of the target vehicle according to the target obstacle.

[0200] Optionally, the control module includes:

[0201] A generating module, configured to generate an obstacle bounding box according to the target obstacle, wherein the obstacle bounding box is a minimum circumscribed rectangle of the target obstacle;

[0202] A second determination module is used to determine the target width and target center coordinate information according to the obstacle bounding box, wherein the target width is the width of the plane of the obstacle bounding box closest to the target vehicle, and the target center coordinate information is the center point cloud coordinate of the plane of the obstacle bounding box closest to the target vehicle;

[0203] The sub-control module is used to control the driving path of the target vehicle through the target width and the target center coordinate information.

[0204] Based on the same concept, this embodiment also provides a non-temporary computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the image processing method provided by this embodiment are implemented.

[0205] Based on the same concept, this embodiment also provides a controller, including:

[0206] a memory having a computer program stored thereon;

[0207] The processor is used to execute the computer program in the memory to implement the steps of the image processing method provided in this embodiment.

[0208] In another exemplary embodiment, a controller is also provided. The controller may be an integrated circuit (IC) or a chip, wherein the integrated circuit may be an IC or a collection of multiple ICs; the chip may include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC, system on chip or system-level chip), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned image processing method. The executable instructions may be stored in the integrated circuit or chip, or may be obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions may be stored in the memory, and the above-mentioned image processing method may be implemented when the executable instructions are executed by the processor; or, the integrated circuit or chip may receive the executable instructions through the interface and transmit them to the processor for execution to implement the above-mentioned image processing method.

[0209] Based on the same concept, this embodiment also provides a vehicle, including the controller provided by this embodiment.

[0210] In a possible embodiment, the vehicle may be a latent AGV (Automated Guided Vehicle).

[0211] Fig. 9 FIG. 9 is a block diagram showing a vehicle 900 according to an exemplary embodiment. Fig. 9 As shown, the vehicle is provided with a depth camera 902 and a controller 901 provided in this embodiment.

[0212] Regarding the vehicle 900 in the above embodiment, the specific method in which the controller 901 performs the operation has been described in detail in the relevant image processing method, and will not be elaborated here.

[0213] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned image processing method when executed by the programmable device.

[0214] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0215] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0216] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. An image processing method, characterized in that: include: Determine a first depth image in front of the target vehicle; Performing noise reduction processing on the first depth image to obtain a second depth image; The second depth image is converted into a three-dimensional point cloud, and a target obstacle in front of the target vehicle is determined according to the three-dimensional point cloud.

2. The image processing method according to claim 1, characterized in that: The performing noise reduction processing on the first depth image to obtain a second depth image includes: The first noise and / or the second noise in the first depth image are removed to obtain the second depth image, where the first noise is used to characterize noise in which depth information of pixels in the first depth image is greater than a preset depth threshold, and the second noise is used to characterize salt and pepper noise in the first depth image.

3. The image processing method according to claim 2, characterized in that: Eliminating first noise in the first depth image to obtain the second depth image includes: Eliminate the first noise in the first depth image by the following calculation formula to obtain the second depth image; Wherein, d is the depth information of the pixel point of the first depth image, and depth(d) is the depth information of the pixel point of the second depth image.

4. The image processing method according to claim 2, characterized in that: Eliminating second noise in the first depth image to obtain the second depth image includes: The second depth image is obtained by removing the second noise in the first depth image through at least one of a median filtering method, a pseudo-median filtering method and an abnormality detection method.

5. The image processing method according to claim 1, characterized in that: The converting the second depth image into a three-dimensional point cloud comprises: Convert the second depth image into the three-dimensional point cloud by the following calculation formula; z = depth(d) / depth_scale x=(u-c x )*z / f x y=(v-c y )*z / f y Wherein, depth(d) is the depth information of the pixel of the second depth image, depth_scale is the preset depth scale, u is the u-axis pixel value of the pixel of the second depth image in the pixel coordinate system, v is the v-axis pixel value of the pixel of the second depth image in the pixel coordinate system, and f x is the scaling factor of the camera focal length on the x-axis, f y is the scaling factor of the camera focal length on the y-axis, c y is the coordinate of the optical center on the y-axis, c x is the optical center coordinate on the x-axis, and (x, y, z) is the point cloud coordinate of the three-dimensional point cloud.

6. The image processing method according to claim 1, characterized in that: Determining the target obstacle ahead of the target vehicle according to the three-dimensional point cloud includes: Reducing the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud; Eliminate the ground point cloud in the first sub-three-dimensional point cloud to obtain a second sub-three-dimensional point cloud; The second sub-three-dimensional point cloud is subjected to denoising processing to obtain a target obstacle ahead of the target vehicle.

7. The image processing method according to claim 6, characterized in that: The step of reducing the point cloud density of the three-dimensional point cloud to obtain a first sub-three-dimensional point cloud includes: The point cloud density of the three-dimensional point cloud is reduced by at least one of a random downsampling method, a voxel downsampling method, a curvature downsampling method and a uniform downsampling method to obtain the first sub-three-dimensional point cloud.

8. The image processing method according to claim 6, characterized in that: The step of removing the ground point cloud from the sub-three-dimensional point cloud to obtain a second sub-three-dimensional point cloud includes: Determine a target point cloud in the first sub-3D point cloud whose y-axis coordinate value is greater than a preset height threshold, multiply each axial coordinate value of the target point cloud by 0, and multiply each axial coordinate value of the remaining point cloud by 1 to obtain the second sub-3D point cloud, where the remaining point cloud is the point cloud in the first sub-3D point cloud except the target point cloud.

9. The image processing method according to claim 8, characterized in that: The step of determining a target point cloud in the first sub-three-dimensional point cloud whose y-axis coordinate value is greater than a preset height threshold, multiplying each axial coordinate value of the target point cloud by 0, and multiplying each axial coordinate value of the remaining point cloud by 1, to obtain the second sub-three-dimensional point cloud, comprises: The second sub-three-dimensional point cloud is obtained by the following calculation formula; X=x1*y filter Y=y1*y filter Z=z1*y filter Among them, (x1, y1, z1) is the point cloud coordinates in the sub-3D point cloud, (X, Y, Z) is the point cloud coordinates in the second sub-3D point cloud, y filter is a value of 0 or 1, n is an index value, and y[n] is the coordinate value of the point cloud in the sub-3D point cloud on the y-axis.

10. The image processing method according to claim 6, characterized in that: The performing point cloud denoising processing on the second sub-three-dimensional point cloud to obtain the target obstacle in front of the target vehicle includes: The second sub-three-dimensional point cloud is denoised by at least one method selected from voxel filtering, statistical filtering and radius filtering to obtain a target obstacle ahead of the target vehicle.

11. The image processing method according to any one of claims 1 to 10, characterized in that: The step of determining a first depth image in front of the target vehicle includes: Acquire a first depth image in front of the target vehicle by a depth camera arranged in front of the target vehicle; Alternatively, a first depth image in front of the target vehicle is acquired by a depth camera disposed behind the target vehicle.

12. The image processing method according to any one of claims 1 to 10, characterized in that: The image processing method further comprises: The driving path of the target vehicle is controlled according to the target obstacle.

13. The image processing method according to claim 12, characterized in that: The controlling the driving path of the target vehicle according to the target obstacle includes: Generate an obstacle bounding box according to the target obstacle, where the obstacle bounding box is a minimum circumscribed rectangle of the target obstacle; According to the obstacle bounding box, determine the target width and target center coordinate information, wherein the target width is the width of the plane of the obstacle bounding box closest to the target vehicle, and the target center coordinate information is the center point cloud coordinate of the plane of the obstacle bounding box closest to the target vehicle; The driving path of the target vehicle is controlled by the target width and the target center coordinate information.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 13 are implemented.

15. A controller, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 13.

16. A vehicle, characterized in that: Includes the controller as claimed in claim 15.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images