Target segmentation method, device, electronic device, storage medium and program product

Abstract

The present invention provides a target segmentation method, which includes: converting a three-dimensional point cloud corresponding to an acquired target object into a depth image; calculating a first angle value formed by two adjacent pixels in each column of all columns of pixels in the depth image, and determining the pixel points belonging to ground points according to the first angle value; removing the pixel points belonging to ground points from the depth image to obtain a target image; calculating a second angle value formed by two adjacent pixels in the target image, and determining the pixel points belonging to the same target according to the second angle value to complete the confirmation of the same target; segmenting the same target, and back-projecting the target's depth image into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image. The present invention makes full use of the depth projection image, reduces the time consumption of sorting using the DBSCAN algorithm, and improves the algorithm rate.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to a method, device, electronic device, storage medium and program product for target segmentation. Background Art

[0002] In the autonomous driving system, the point cloud output by the LiDAR sensor can obtain effective information such as target size, distance, heading angle, etc. The point cloud needs to be accurately segmented and the bounding box needs to be fitted to provide reliable information of the target for subsequent tracking, fusion and other modules, so as to achieve environmental perception and ensure the safety of autonomous driving technology. For traditional algorithms, the LiDAR point cloud first needs to filter the ground point cloud to reduce the impact on target segmentation. The common DBSCAN algorithm solves the problems of ground filtering and target segmentation and is suitable for situations with a small number of point clouds, such as millimeter-wave radars with a point cloud of about two to three hundred. If it is also used to process LiDAR point clouds with a number of more than 100,000, the CPU occupancy and calculation time will increase greatly, and it cannot meet the requirements of the actual vehicle operation of the software. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, the present invention provides a target segmentation method, device, electronic device, storage medium and program product to solve the above-mentioned technical problems.

[0004] The present invention provides a target segmentation method, the method comprising:

[0005] Convert the acquired three-dimensional point cloud corresponding to the target object into a depth image;

[0006] Calculate a first angle value formed by two adjacent pixel points in each column of all columns of pixels in the depth image, and compare the first angle values ​​of all columns of pixels in the depth image with a first specified threshold value to obtain a first comparison result, and determine the pixel point belonging to the ground point according to the first comparison result;

[0007] Removing the pixel points belonging to the ground points from the depth image to obtain a target image;

[0008] Calculating a second angle value formed by two adjacent pixels in the target image, and comparing the second angle values ​​of all columns of pixels in the target image with a second specified threshold value to obtain a second comparison result, and determining the pixels belonging to the same target according to the second comparison result to complete the confirmation of the same target;

[0009] The same target is segmented, and the depth image of the target is back-projected into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image.

[0010] In an exemplary embodiment of the present invention, after obtaining all target points belonging to the same target, the method further includes:

[0011] Acquiring the distance between the target and the point cloud data acquisition device;

[0012] According to the distance, searching for a pixel threshold value corresponding to the distance in a correlation relationship table between distance ranges and pixel threshold values; wherein different pixel threshold values ​​correspond to different distance ranges;

[0013] Delete pixels whose value is less than the pixel threshold.

[0014] In an exemplary embodiment of the present invention, in the step of converting three-dimensional point cloud data into a depth image, at least one point corresponds to a pixel point; if each pixel point corresponds to multiple points, the point with the largest distance from the laser radar among the multiple points is converted to obtain the depth image data corresponding to the point.

[0015] In an exemplary embodiment of the present invention, after the step of converting the three-dimensional point cloud into a depth image, the method further includes:

[0016] Determine whether there is a pixel point without a depth value in each column of pixels of the depth image, and record it as a pixel to be repaired;

[0017] Determine adjacent pixel points of the point to be repaired as repair reference points, and use an average value of depth values ​​of the repair reference points as the depth value of the point to be repaired; wherein a difference in the depth values ​​of the repair reference points is greater than a depth difference threshold.

[0018] In an exemplary embodiment of the present invention, after the first angle value is calculated, the method further includes: constructing an angle image using the first angle value as a matrix element, and performing smoothing filtering on the angle image.

[0019] In an exemplary embodiment of the present invention, determining the pixel point belonging to the ground point according to the first comparison result includes:

[0020] If the first comparison result is that the first angle value is less than the first specified threshold, two adjacent pixel points corresponding to the first angle value are marked as ground points.

[0021] In an exemplary embodiment of the present invention, determining the pixel points belonging to the same target according to the second comparison result includes:

[0022] If the second comparison result is that the second angle value is greater than the second specified threshold, two adjacent pixel points corresponding to the second angle value are marked as points of the same target.

[0023] In an exemplary embodiment of the present invention, before converting the three-dimensional point cloud into a depth image, the method further includes:

[0024] Setting projection parameters, wherein the projection parameters include projection angles in the vertical direction and the horizontal direction;

[0025] The three-dimensional point cloud is projected onto the depth image according to the projection parameters to convert the three-dimensional point cloud into a depth image.

[0026] The present invention provides a target segmentation device, the device comprising:

[0027] A depth image acquisition module, used to convert the acquired three-dimensional point cloud corresponding to the target object into a depth image;

[0028] A ground point determination module, used to calculate a first angle value formed by two adjacent pixel points in each column of all columns of pixels in the depth image, and compare the first angle values ​​of all columns of pixels in the depth image with a first specified threshold value to obtain a first comparison result, and determine the pixel point belonging to the ground point according to the first comparison result;

[0029] A removal module, used to remove the ground point from the depth image to obtain a target image;

[0030] A target determination module is used to calculate a second angle value formed by two adjacent pixel points in the target image, and compare the second angle values ​​of all columns of pixels in the target image with a second specified threshold value to obtain a second comparison result, and determine the pixel points belonging to the same target according to the second comparison result to complete the confirmation of the same target;

[0031] The target segmentation module is used to segment the same target and back-project the target's depth image into point cloud data based on the mapping relationship between the three-dimensional point cloud and the depth image.

[0032] The present invention provides an electronic device, comprising:

[0033] one or more processors;

[0034] The storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the target segmentation method.

[0035] A computer-readable storage medium provided by the present invention is characterized in that computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is enabled to execute the target segmentation method.

[0036] A computer program product provided by the present invention is characterized in that it includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the target segmentation method.

[0037] Beneficial effects of the present invention:

[0038] A target segmentation method in the present invention includes: converting the acquired three-dimensional point cloud corresponding to the target object into a depth image; calculating the first angle value formed by two adjacent pixels in each column of all columns of pixels in the depth image, and comparing the first angle values ​​of all columns of pixels in the depth image with a first specified threshold value to obtain a first comparison result, and determining the pixel points belonging to the ground point according to the first comparison result; removing the pixel points belonging to the ground point from the depth image to obtain the target image; calculating the second angle value formed by two adjacent pixels in the target image, and comparing the second angle values ​​of all columns of pixels in the target image with a second specified threshold value to obtain a second comparison result, and determining the pixel points belonging to the same target according to the second comparison result to complete the confirmation of the same target; segmenting the same target, and back-projecting the target's depth image into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image. The present invention uses the depth image to first identify the ground in the depth image, then remove the ground to obtain an image including the target, and then identify the pixels belonging to the same target in the target image, thereby identifying the target, and finally segmenting the target. The present invention makes full use of the depth projection image, reduces the time consumption of sorting using the DBSCAN algorithm, and improves the algorithm rate.

[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0041] Figure 1 is a schematic diagram of an implementation environment of a target segmentation method shown in an exemplary embodiment of the present application;

[0042] Figure 2 is a flow chart of a target segmentation method shown in an exemplary embodiment of the present application;

[0043] Figure 3is a schematic diagram showing a process of image restoration according to an exemplary embodiment of the present application;

[0044] Figure 4 is a flowchart showing determination of ground pixel points according to an exemplary embodiment of the present application;

[0045] Figure 5 is a schematic diagram showing the angle of two adjacent pixels according to an exemplary embodiment of the present application;

[0046] Figure 6 is another exemplary embodiment of the present application showing an angle diagram of two adjacent pixels;

[0047] Figure 7 is another exemplary embodiment of the present application showing a flow chart for determining the same target;

[0048] Figure 8 is a schematic diagram of the angle between two adjacent pixels in a target image shown in another exemplary embodiment of the present application;

[0049] Fig. 9 is a comparison result diagram of another exemplary embodiment of the present application showing the target segmentation method of the present invention;

[0050] Fig.10 is a comparison diagram of target segmentation results without setting a segmentation threshold and after setting a segmentation threshold, shown in another exemplary embodiment of the present application;

[0051] Fig.11 is a time efficiency comparison diagram of the target segmentation method of the present invention and a general segmentation algorithm based on depth projection, shown in another exemplary embodiment of the present application;

[0052] Fig.12 is a block diagram of a target segmentation device shown in an exemplary embodiment of the present application;

[0053] Fig.13 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0055] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0056] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0057] Figure 1 is a schematic diagram of an exemplary target segmentation method implementation environment of the present application. Figure 1 The implementation environment includes a terminal device 101 and a server 102, and the terminal device 101 and the server 102 communicate with each other through a wired or wireless network.

[0058] It should be understood that Figure 1 The number of terminal devices 101 and servers 102 in the embodiment is only for illustration. Any number of terminal devices 101 and servers 102 may be provided according to actual needs.

[0059] Among them, the terminal device 101 corresponds to the client, which can be any electronic device with a user input interface, including but not limited to smart phones, tablets, laptops, computers, car computers, etc., among which the user input interface includes but is not limited to touch screens, keyboards, physical buttons, audio pickup devices, etc.

[0060] Among them, server 102 corresponds to the service end, which can be a server that provides various services. It can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network) and basic cloud computing services such as big data and artificial intelligence platforms. This is not restricted here.

[0061] The terminal device 101 can communicate with the server 102 through wireless networks such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), and 5G (fifth generation mobile information technology), and this is not restricted here.

[0062] Since there are problems in the prior art that target segmentation is time-consuming and has low accuracy, in order to solve these problems, the embodiments of the present application respectively propose a target segmentation method, a software segmentation device, an electronic device, a computer-readable storage medium and a computer program product, and these embodiments will be described in detail below.

[0063] See also Figure 2 , Figure 2 is a flowchart of a target segmentation method shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is specifically executed by the terminal device 101 in the implementation environment. It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0064] See also Figure 2 , Figure 2 This is a flowchart of an exemplary target segmentation method of the present application, and the target segmentation method at least includes steps S210 to S250, including:

[0065] Step S210 converts the acquired three-dimensional point cloud corresponding to the target object into a depth image;

[0066] Step S220 calculates a first angle value formed by two adjacent pixel points in each column of all columns of pixels in the depth image, and compares the first angle values ​​of all columns of pixels in the depth image with a first specified threshold value to obtain a first comparison result, and determines the pixel point belonging to the ground point according to the first comparison result;

[0067] Step S230 removes the pixel points belonging to the ground points from the depth image to obtain a target image;

[0068] Step S240 calculates a second angle value formed by two adjacent pixels in the target image, and compares the second angle values ​​of all columns of pixels in the target image with a second specified threshold value to obtain a second comparison result, and determines the pixels belonging to the same target according to the second comparison result to complete the confirmation of the same target;

[0069] Step S250 segments the same target and back-projects the target's depth image into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image.

[0070] The present invention utilizes the depth image, first identifies the ground in the depth image, then removes the ground to obtain an image including the target, then identifies the pixels in the target image that belong to the same target, thereby identifying the target, and finally segmenting the target. The present invention makes full use of the depth projection image, reduces the time consumption of sorting using the DBSCAN algorithm, and improves the algorithm rate.

[0071] The detailed steps of the target segmentation method of the present invention are described below through specific embodiments.

[0072] In step S210, the acquired three-dimensional point cloud corresponding to the target object is converted into a depth image;

[0073] It should be noted that the target segmentation method in the embodiment of the present application can be applied in, for example, an autonomous driving system. Autonomous vehicles (Self-driving automobile) is also known as unmanned driving or computer driving, which is a type of unmanned driving achieved through a computer system. Autonomous driving relies on the collaboration of artificial intelligence, visual computing, radar, monitoring devices, and global positioning systems, allowing computers to operate automatically and safely without any active human operation.

[0074] The target object may be an object of any form or shape. In an exemplary embodiment, the target object may be a vehicle. The acquisition of the 3D point cloud corresponding to the target object may be a device that can directly collect the 3D point cloud, such as a 3D laser scanner, a 3D optical detector, or a 3D laser radar, or a component of the above device, such as a processor or microprocessor inside the device. The 3D point cloud is a collection of a large number of points that represent the surface characteristics of the target object.

[0075] In an exemplary embodiment, a three-dimensional laser radar can be used to obtain a three-dimensional point cloud corresponding to a target object. Specifically, three-dimensional laser radar is widely used in driverless cars, unmanned small aircraft, unmanned trucks, mobile mapping vehicles, street view vehicles, yachts, airports, and other fields. The working principle of a three-dimensional laser radar is very similar to that of a radar. It uses laser as a signal source. The pulsed laser emitted by the laser hits the trees, roads, bridges, and buildings on the ground, causing scattering. A portion of the light waves will be reflected to the laser radar receiver. According to the laser ranging principle, the distance from the three-dimensional laser radar to the target point is calculated. The pulsed laser continuously scans the target object to obtain the data of all target points on the target object. After imaging processing with this data, an accurate three-dimensional stereo image can be obtained.

[0076] A depth image, also called a distance image, refers to an image that uses the distance (depth) value from the image collector to each point in the scene as a pixel value. It directly reflects the geometric shape of the visible surface of the scene, and it can be used to easily solve many problems in 3D target description. When a laser beam is irradiated onto the surface of an object, the reflected laser will carry information such as direction and distance. If the laser beam is scanned along a certain trajectory, the reflected laser point information will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained, thus forming a three-dimensional point cloud. The depth image can be calculated as three-dimensional point cloud data after coordinate transformation, and of course the three-dimensional point cloud data can also be inversely calculated as depth image data.

[0077] In an exemplary embodiment of the present application, since the number of pixel points corresponding to the target object is much smaller than the number of point clouds of the target object, 2 to 3 point clouds are projected to one pixel point during the process of projecting the point cloud to the depth image. In addition, in order to form a reasonable projection of the point cloud on the image and prevent an increase in pixel restoration caused by uneven projection, before converting the three-dimensional point cloud into a depth image, projection parameters can be set according to the distribution of the statistical point cloud in the horizontal and vertical directions, including projection angles in the vertical and horizontal directions; the three-dimensional point cloud is projected onto the depth image according to the projection parameters to convert the three-dimensional point cloud into a depth image.

[0078] Specifically, according to the distribution of the 3D laser radar point cloud, it is ensured that it is projected relatively evenly into the pixel, the horizontal pixel angle is set to 0.2°, the number of pixels is FOV / 0.2°=701, the vertical angle is 0.4°*8+0.3*15+0.2°*27+0.15°*8+0.2°*20+0.3°*8+0.4°*5, and the number of pixels is 91. Among them, 8 means that there are 8 pixels in the scanning area of ​​0.4°, 15 means that there are 15 pixels in the scanning area of ​​0.3°, 27 means that there are 27 pixels in the scanning area of ​​0.2°, 8 means that there are 8 pixels in the scanning area of ​​0.15°, 20 means that there are 8 pixels in the scanning area of ​​0.2°, 8 means that there are 8 pixels in the scanning area of ​​0.3°, and 5 means that there are 5 pixels in the scanning area of ​​0.4°.

[0079] In an exemplary embodiment of the present application, the three-dimensional coordinate system (x, y, z) of the three-dimensional point cloud is converted to the (row, col, depth) coordinates of the depth image. Specifically, the points in the three-dimensional space are first projected onto a cylindrical surface that can be unfolded to make it planar, and then any three-dimensional data point (x, y, z) in the three-dimensional point cloud is converted into an element (Row, Col) of the two-dimensional depth map by the following formula, and the distance dis from the three-dimensional data point to the laser radar is used as the pixel value of the point. Calculate the depth of the point Calculate the horizontal angle Calculate vertical angle According to the projection parameters, the pixels (Row, Col) of each point cloud projection are obtained.

[0080] In an exemplary embodiment of the present application, in the step of converting the three-dimensional point cloud data into a depth image, at least one point corresponds to a pixel point; if each pixel point corresponds to multiple points, the point among the multiple points with the largest distance from the laser radar is converted to obtain the depth image data corresponding to the point.

[0081] In this embodiment, 2 to 3 point clouds are projected onto the depth image. During the projection process, the indexes of all point clouds are not recorded, but only the index of the point cloud with the largest depth value is recorded to achieve downsampling processing, which can reduce the number of point clouds by about 30%.

[0082] See also Figure 3 , Figure 3 is a flowchart of image restoration. Since it is very likely that there is no point cloud projection on the depth projection image pixel, therefore, in an exemplary embodiment of the present invention, after the step of converting the three-dimensional point cloud into a depth image, the method further includes:

[0083] S310 determines whether there are any pixels without depth values ​​in each column of pixels of the depth image, and records them as points to be repaired;

[0084] S320 determines adjacent pixel points of the point to be repaired as repair reference points, and uses an average value of the depth values ​​of the repair reference points as the depth value of the point to be repaired.

[0085] It should be noted that the difference in depth value of the repair reference point is greater than the depth difference threshold. Specifically, for the current pixel without depth, find the four adjacent pixels in the vertical direction (the same column of pixels), and under the premise of meeting the depth difference threshold of the upper and lower point clouds, the current pixel is repaired by the average depth of the adjacent points.

[0086]

[0087] Among them, curDepth is the depth value of the point to be repaired, counter is the number of points involved in the repair, prevDepth1 is the first point above the repair point and adjacent to the point to be repaired, prevDepth2 is the second point above the repair point and adjacent to the point to be repaired, nextDepth1 is the first point below the repair point and adjacent to the point to be repaired, and nextDepth2 is the second point below the repair point and adjacent to the point to be repaired.

[0088] In step S220, the first angle value formed by two adjacent pixel points in each column of all columns of pixels in the depth image is calculated, and the first angle values ​​of all columns of pixels in the depth image are compared with the first specified threshold to obtain a first comparison result, and the pixel point belonging to the ground point is determined according to the first comparison result.

[0089] The step of determining the pixel points belonging to the ground points according to the first comparison result includes: if the first comparison result is that the first angle value is less than the first specified threshold, marking two adjacent pixel points corresponding to the first angle value as ground points.

[0090] In an exemplary embodiment of the present invention, please refer to Figure 4 , Figure 4 The specific steps for determining ground pixel points are shown in the figure. The process of determining pixel points belonging to ground points includes:

[0091] S410 randomly selects two adjacent pixels in a column of pixels of the depth image, calculates a first angle value of the two adjacent pixels, and compares the first angle value with a first specified threshold.

[0092] It should be noted that when selecting two adjacent pixel points, search from the bottom to the top first. Since the angle values ​​of the pixels in the bottom row are all 0, you can select the starting point from the second to last row.

[0093] See also Figure 5 , 6 , Figure 5 is a schematic diagram of the angles of two adjacent pixels in one embodiment, Figure 6 Schematic diagram of the angle between two adjacent pixels in another embodiment, that is, the first angle value = arctan (dy / dx); wherein dx is the difference between the horizontal coordinates of the two adjacent pixels, and dy is the difference between the vertical coordinates of the two adjacent pixels.

[0094] S420 marks the corresponding pixel point whose first angle value is less than the first specified threshold as the current ground point;

[0095] The first specified threshold can be 45°, that is, when the first angle value is less than 45°, it indicates that the angle difference between the two corresponding pixels in the depth image is small enough to meet the ground conditions, and the corresponding pixel is considered to be a ground point and used as the current ground point. When performing specific marking, the point can be marked as 1.

[0096] S430: Starting from the current ground point, searching toward unmarked adjacent points to determine all pixel points that are ground points;

[0097] S440 traverses all column pixels and repeats the above steps to complete the marking of all ground points.

[0098] In step S230, the pixel points belonging to the ground points are removed from the depth image to obtain a target image.

[0099] Since the depth image obtained by projecting the point cloud data of the three-dimensional lidar includes the ground and the target object, the target object must be determined from the depth image before the target is segmented. Therefore, the ground points need to be removed from the depth image to obtain a depth image belonging to the target object.

[0100] In step S240, the second angle value formed by two adjacent pixel points in the target image is calculated, and the second angle values ​​of all column pixels of the target image are compared with the second specified threshold value to obtain a second comparison result, and the pixel points belonging to the same target are determined according to the second comparison result to complete the confirmation of the same target.

[0101] Among them, determining the pixel points belonging to the same target based on the second comparison result includes: if the second comparison result is that the second angle value is greater than the second specified threshold, marking the two adjacent pixel points corresponding to the second angle value as points of the same target.

[0102] See also Figure 7 , Figure 7 The process for determining the same goal is:

[0103] First, select the starting pixel, traverse each pixel, and determine whether the pixel is greater than zero. If it is greater than, it means that the pixel has been marked as a target point, and then continue to search for the next pixel. If the pixel is not marked as a target point, determine whether the pixel has depth information. If it is not marked and has projected depth information, use the starting pixel as the current pixel to search for the angle distance of the pixels adjacent to the current pixel, that is, the second angle value.

[0104] See also Figure 8 , Figure 8 is a schematic diagram of the angle between two adjacent pixels in the target image. If the two adjacent pixels are adjacent pixels in the same row, then α is the horizontal angle, that is, the angle between the two laser lines; if the two adjacent pixels are adjacent pixels in the same column, then α is the vertical angle. Figure 7 As shown, dy = d2*sin(α), dx = d1-d2*cos(α), then the second angle value β = arctan(dy / dx). When the β angle is greater than the second specified threshold, it indicates that the distance between the two pixels is close to the same target point, and the pixels that meet this condition are marked as points of the same target.

[0105] In step S250, the same target is segmented, and the depth image of the target is back-projected into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image.

[0106] See also Fig. 9 , Fig. 9 The figure is a comparison result diagram of the target segmentation method of the present invention. It can be seen from the figure that the number of point clouds of the obtained target is greatly reduced by using the present invention to process the original point cloud, which reduces the amount of data and improves the processing efficiency.

[0107] In an exemplary embodiment of the present invention, after obtaining all target points belonging to the same target, the method further includes:

[0108] The distance between the target and the point cloud data acquisition device is obtained; according to the distance, a pixel threshold corresponding to the distance is searched in an association relationship table between a distance range and a pixel threshold; and pixel points whose pixels are smaller than the pixel threshold are deleted.

[0109] It should be noted that the association table between distance range and pixel threshold records the relationship between distance range and pixel threshold. For example, the distance range A to B corresponds to pixel threshold C, the distance range A1 to B1 corresponds to pixel threshold C1, the distance range A2 to B2 corresponds to pixel threshold C2, and so on. Finding the pixel threshold corresponding to the distance can specifically mean that if the distance between the target and the radar is between A and B, the corresponding pixel threshold is C1, then when marking the pixel points belonging to the same target, the pixel points whose pixels are less than the pixel threshold C1 are deleted, that is, the minimum pixel value of the retained pixel points is C1.

[0110] Specifically, three distance ranges of near, medium and far are set with corresponding pixel thresholds. 0 to 20m is a near distance, and the minimum threshold for retaining the same marked target is 40 pixels; 20 to 70m is a medium distance, and the minimum threshold for retaining is 25 pixels; greater than 70m is a far distance, and the minimum threshold for retaining is 10 pixels; when the number of pixels of the target is greater than the segmentation threshold, the target is retained, otherwise it does not meet the minimum number of target pixels and is not retained as a target. Please refer to Fig.10 , Fig.10 The figure shows the comparison of the target segmentation results without setting the segmentation threshold and after setting the segmentation threshold. Fig.10 It can be seen that there is no target loss for long-distance targets.

[0111] In an exemplary embodiment of the present invention, after the first angle value is calculated, the method further includes: constructing an angle image using the first angle value as a matrix element, and performing smoothing filtering on the angle image.

[0112] Specifically, the angle image can be smoothed by the SavitskyGolay filtering algorithm. More specifically, the filtering parameters are first set according to the image size, and then the filtering is completed by the filter2D function of openCV.

[0113] The Savitzky-Golay filter fitting method determines the appropriate filtering parameters according to the average trend of the NDVI time series curve, and uses a polynomial to achieve the least squares fitting within the sliding window; the Savitzky-Golay filter method (based on the least squares convolution fitting algorithm) is used for iterative calculations to simulate the entire NDVI time series data to obtain the long-term change trend.

[0114] See also Fig.11 , Fig.11 FIG. 4 is a time efficiency comparison diagram of the target segmentation method of the present invention and the general segmentation algorithm based on depth projection. Fig.11 This is a time efficiency comparison chart between the segmentation method of the present invention and the general segmentation algorithm based on depth projection under the condition of the same point cloud test data. The average processing time of the segmentation method of the present invention is about 55ms, and the average time consumption of the general method is about 95ms. The computing speed of the segmentation method of the present invention is higher.

[0115] See also Fig.12 , Fig.12 This is a principle block diagram of an object segmentation device according to an exemplary embodiment of the present application, and the object segmentation device includes:

[0116] The depth image acquisition module 1201 is used to convert the acquired three-dimensional point cloud corresponding to the target object into a depth image;

[0117] A ground point determination module 1202 is used to calculate a first angle value formed by two adjacent pixel points in each column of all columns of pixels in the depth image, and compare the first angle values ​​of all columns of pixels in the depth image with a first specified threshold to obtain a first comparison result, and determine a pixel point belonging to a ground point according to the first comparison result;

[0118] A removal module 1203, configured to remove the ground point from the depth image to obtain a target image;

[0119] The target determination module 1204 is used to calculate a second angle value formed by two adjacent pixels in the target image, and compare the second angle values ​​of all columns of pixels in the target image with a second specified threshold value to obtain a second comparison result, and determine the pixels belonging to the same target according to the second comparison result to complete the confirmation of the same target;

[0120] The target segmentation module 1205 is used to segment the same target and back-project the target's depth image into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image.

[0121] It should be noted that the segmentation device provided in the above embodiment and the segmentation method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the segmentation device provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0122] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the segmentation method provided in the above-mentioned embodiments.

[0123] Fig.13 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0124] like Fig.13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303, such as executing the method described in the above embodiment. In the RAM 1303, various programs and data required for system operation are also stored. The CPU 1301, the ROM 1302 and the RAM 1303 are connected to each other through the bus 1304. The input / output (I / O) interface 1305 is also connected to the bus 1304.

[0125] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0126] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, various functions defined in the system of the present application are executed.

[0127] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0128] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0130] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the target segmentation method as described above is implemented. The computer-readable storage medium can be included in the electronic device described in the above embodiment, or it can exist independently without being assembled into the electronic device.

[0131] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the target segmentation method provided in each of the above embodiments.

[0132] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A target segmentation method, characterized in that: The method comprises: Convert the acquired three-dimensional point cloud corresponding to the target object into a depth image; Calculate a first angle difference between two adjacent pixels in each column of all columns of the depth image, and compare the first angle differences of all columns of the depth image with a first specified threshold value to obtain a first comparison result, and determine the pixel points belonging to the ground point according to the first comparison result; when the first angle difference is less than the first specified threshold value, the two adjacent pixels corresponding to the first angle difference are marked as ground points; wherein the first angle difference is arctan (dy / dx), dx is the difference between the horizontal coordinates of the two adjacent pixels, and dy is the difference between the vertical coordinates of the two adjacent pixels; Removing the pixel points belonging to the ground points from the depth image to obtain a target image; Calculate the second angle difference formed by two adjacent pixel points in the target image, and compare the second angle difference values ​​of all columns of pixels in the target image with the second specified threshold value to obtain a second comparison result, and determine the pixel points belonging to the same target according to the second comparison result to complete the confirmation of the same target; when the second angle difference value is greater than the second specified threshold value, the two adjacent pixel points corresponding to the second angle difference value are marked as points of the same target; wherein, the second angle difference value β = arctan(dy1 / dx1), dy1 = d2 *sin(α), dx1 = d1-d2 *cos(α), d1 represents the Euclidean distance between the first pixel point and the origin, d2 represents the Euclidean distance between the second pixel point and the origin, represents the angle between the line connecting the first pixel point and the origin and the line connecting the second pixel point and the origin; wherein the first pixel point is one of the two adjacent pixel points in the same column, and the second pixel point is the other pixel point of the two adjacent pixel points in the same column; Segmenting the same target, and back-projecting the target's depth image into point cloud data according to a mapping relationship between the three-dimensional point cloud and the depth image; After obtaining the pixels belonging to the same target, it also includes: Acquiring the distance between the target and the point cloud data acquisition device; According to the distance, searching for a pixel threshold value corresponding to the distance in a correlation relationship table between distance ranges and pixel threshold values; wherein different pixel threshold values ​​correspond to different distance ranges; Delete pixels whose value is less than the pixel threshold.

2. The target segmentation method according to claim 1, characterized in that: In the step of converting three-dimensional point cloud data into a depth image, at least one point corresponds to a pixel point; if each pixel point corresponds to multiple points, the point with the largest distance from the laser radar among the multiple points is converted to obtain the depth image data corresponding to the point.

3. The target segmentation method according to claim 1 or 2, characterized in that: After the step of converting the three-dimensional point cloud into a depth image, the method further comprises: Determine whether there is a pixel point without a depth value in each column of pixels of the depth image, and record it as a pixel to be repaired; Determine adjacent pixel points of the point to be repaired as repair reference points, and use the average value of the depth values ​​of the repair reference points as the depth value of the point to be repaired; wherein, the difference in the depth values ​​of the repair reference points is greater than a depth difference threshold, the repair reference point and the point to be repaired are located in the same column of pixels, and the repair reference points include: a first point above the point to be repaired and adjacent to the point to be repaired, a second point above the point to be repaired and adjacent to the point to be repaired, a first point below the point to be repaired and adjacent to the point to be repaired, and a second point below the point to be repaired and adjacent to the point to be repaired; wherein, the first point and the second point above the point to be repaired and the first point and the second point below the point to be repaired meet the depth difference threshold of the upper and lower point clouds.

4. The target segmentation method according to claim 1, characterized in that: After the first angle difference is calculated, the method further includes: constructing an angle image using the first angle difference as a matrix element, and performing smoothing filtering on the angle image.

5. The target segmentation method according to claim 1, characterized in that: Before converting the three-dimensional point cloud into a depth image, the method further includes: Setting projection parameters, wherein the projection parameters include projection angles in the vertical direction and the horizontal direction; The three-dimensional point cloud is projected onto the depth image according to the projection parameters to convert the three-dimensional point cloud into a depth image.

6. A target segmentation device, characterized in that: The device comprises: A depth image acquisition module, used to convert the acquired three-dimensional point cloud corresponding to the target object into a depth image; A ground point determination module, used to calculate a first angle difference formed by two adjacent pixel points in each column of all columns of pixels in the depth image, and compare the first angle differences of all columns of pixels in the depth image with a first specified threshold value to obtain a first comparison result, and determine the pixel points belonging to the ground point according to the first comparison result; when the first angle difference is less than the first specified threshold value, the two adjacent pixel points corresponding to the first angle difference are marked as ground points; wherein the first angle difference is arctan (dy / dx), dx is the difference between the horizontal coordinates of two adjacent pixel points, and dy is the difference between the vertical coordinates of two adjacent pixel points; A removal module, used to remove the ground point from the depth image to obtain a target image; A target determination module is used to calculate the second angle difference formed by two adjacent pixel points in the target image, and compare the second angle difference values ​​of all column pixels of the target image with the second specified threshold value to obtain a second comparison result, and determine the pixel points belonging to the same target according to the second comparison result to complete the confirmation of the same target; when the second angle difference value is greater than the second specified threshold value, the two adjacent pixel points corresponding to the second angle difference value are marked as points of the same target; wherein, the second angle difference value β = arctan(dy1 / dx1), dy1 = d2 *sin(α), dx1 = d1-d2 *cos(α), d1 represents the Euclidean distance between the first pixel point and the origin, d2 represents the Euclidean distance between the second pixel point and the origin, represents the angle between the line connecting the first pixel point and the origin and the line connecting the second pixel point and the origin; wherein the first pixel point is one of the two adjacent pixel points in the same column, and the second pixel point is the other pixel point of the two adjacent pixel points in the same column; A target segmentation module, used to segment the same target and back-project the target's depth image into point cloud data according to the mapping relationship between the three-dimensional point cloud and the depth image; After obtaining the pixel points belonging to the same target, the distance between the target and the point cloud data acquisition device is obtained; according to the distance, the pixel threshold corresponding to the distance is searched in the association relationship table between the distance range and the pixel threshold; wherein different pixel thresholds correspond to different distance ranges; and the pixel points whose pixels are smaller than the pixel threshold are deleted.

7. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the target segmentation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the target segmentation method according to any one of claims 1 to 5.

9. A computer program product, characterized in that Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the target segmentation method according to any one of claims 1 to 5.

Images