Point cloud feature extraction method and system, electronic equipment and medium

Abstract

The invention relates to the technical field of image processing, and provides a point cloud feature extraction method and system, electronic equipment and a medium, and the point cloud feature extraction method comprises the following steps: obtaining a picture shot by a camera and a point cloud shot by a laser radar; the view field range of the laser radar is larger than that of the camera; obtaining effective point clouds in the view field range of the camera; calculating a mapping table from the image to the point cloud; feature contour points in the image are obtained, and feature points of the corresponding point cloud are calculated; according to the scheme, the image and the point cloud can be combined, the mapping table of the point cloud and the image is established, the feature points extracted from the image are matched in the point cloud, and the corresponding point cloud feature points are obtained.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a point cloud feature extraction method and system, electronic device and medium. Background Technology

[0002] With societal development, technologies such as intelligent driving, machine vision, and photogrammetry are being increasingly widely applied across various industries. The types of sensors used for data collection are also diverse, including LiDAR, cameras, and millimeter-wave radar. Each of these sensors has its own advantages and disadvantages, and they complement each other. Therefore, efficiently fusing data from multiple sensors is crucial for improving the effectiveness of perception algorithms.

[0003] Cameras generate 2D data, which can capture the color, shape, and other characteristics of objects, and the data is dense. Therefore, it is easy to find the outline of a target in an image, but a single camera cannot determine the actual size and volume of the target. LiDAR, on the other hand, generates 3D data, which can accurately determine the distance to an object, but the data is sparse. LiDAR cannot perceive information such as the color of the target, nor can it use color information to segment the target.

[0004] Since camera and LiDAR data are complementary, the image data from the camera is fused with the point cloud data from the LiDAR. First, features are extracted from the image, and then the corresponding point clouds from the LiDAR are matched. This allows for the calculation of the target's volume and area.

[0005] Existing point cloud feature extraction algorithms are all based on the point cloud itself, using point cloud extraction and segmentation algorithms to extract features such as lines and planes from the point cloud.

[0006] Existing point cloud feature extraction methods are mainly based on the distribution and statistical characteristics of point clouds. They can only be extracted normally when the features to be extracted are significantly different from the background in the point cloud. Furthermore, because point clouds are relatively sparse and the color of the target cannot be obtained, feature extraction algorithms based on point clouds are not effective in extracting regions caused by color differences in point clouds.

[0007] Therefore, there is a need to provide a point cloud feature extraction method, system, electronic device, and medium that can combine images and point clouds to establish a mapping table between point clouds and images, and match the feature points extracted from the image in the point cloud to obtain the corresponding point cloud feature points.

[0008] The information disclosed in the background section is only intended to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0009] The main objective of this invention is to overcome the problem of inaccurate point cloud feature extraction and to provide a point cloud feature extraction method, system, electronic device, and medium that can combine images and point clouds to establish a mapping table between point clouds and images. For feature points extracted from images, they are matched in the point cloud to obtain the corresponding point cloud feature points.

[0010] To achieve the above objectives, the first aspect of the present invention provides a point cloud feature extraction method, comprising the following steps: It acquires images captured by the camera and point clouds captured by the LiDAR; the field of view of the LiDAR is larger than that of the camera. Acquire the effective point cloud within the camera's field of view; Calculate the mapping table from the image to the point cloud; Obtain the feature contour points in the image and calculate the corresponding feature points of the point cloud.

[0011] According to an exemplary embodiment of the present invention, the effective point cloud within the camera's field of view is the point cloud located within the camera's field of view.

[0012] According to an exemplary embodiment of the present invention, acquiring the effective point cloud within the camera's field of view includes: Obtain the normalized phase plane positions of the midpoints of the four edges of the image; Calculate the camera's horizontal and vertical field of view based on four points; Mark the point cloud in the opposite direction of the camera's shooting direction as invalid points; Calculate the angle between the point cloud and the camera optical axis in the horizontal direction and the angle between the point cloud and the camera optical axis in the vertical direction. Mark point clouds with a horizontal angle greater than half of the horizontal field of view as invalid points, and mark point clouds with a vertical angle greater than half of the vertical field of view as invalid points. The remaining point clouds, excluding invalid points, are marked as valid point clouds.

[0013] According to an exemplary embodiment of the present invention, the image-to-point cloud mapping table includes: Create an empty two-dimensional mapping table based on the pixel coordinate system of the image; Project the point cloud onto a mapping table of the image.

[0014] According to an exemplary embodiment of the present invention, the mapping table for projecting point clouds onto images includes: The point cloud in 3D space is projected one by one into the image mapping table. If multiple point clouds are projected into the same pixel, the point cloud closest to the optical center is selected and projected into the mapping table.

[0015] According to an exemplary embodiment of the present invention, the step of acquiring feature contour points in an image and calculating the corresponding feature points of a point cloud includes: Detect point features and / or line features and / or contour features of the image; Look up the corresponding point cloud in the mapping table; If there is no point cloud at the location of a feature point in the image, then search for a point cloud within the 8-neighborhood of the mapping table. The found point cloud is filtered to obtain the feature points of the point cloud.

[0016] According to an exemplary embodiment of the present invention, the detection point features are detection SIFT features, SURF features, or ORB features; The detection line feature uses the LSD algorithm, EDLine algorithm, or Hough line detection method; The detection of circles or ellipses uses the ED ellipse detection algorithm or a circle detection algorithm based on Hough transform. Contour feature detection involves calculating image edges and using contour finding methods or manually drawing contours on the image.

[0017] According to an exemplary embodiment of the present invention, the filtering of the found point cloud includes: Denoise the point cloud and exclude discrete point clouds; Extract straight lines from the point cloud on the line feature; Obtain point clouds on the same plane, perform plane segmentation, and remove points outside the plane.

[0018] As a second aspect of the present invention, the present invention provides a point cloud feature extraction system, comprising: a data acquisition module, an effective point cloud filtering module, a mapping table calculation module, and a feature point calculation module; The data acquisition module is used to acquire images captured by the camera and point clouds captured by the lidar; the field of view of the lidar is larger than that of the camera. The effective point cloud filtering module is used to obtain effective point clouds within the camera's field of view. The mapping table calculation module is used to calculate the mapping table from the image to the point cloud; The feature point calculation module is used to obtain feature contour points in the image and calculate the corresponding feature points of the point cloud.

[0019] As a third aspect of the present invention, the present invention provides an electronic device comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the point cloud feature extraction method.

[0020] As a fourth aspect of the present invention, the present invention provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the point cloud feature extraction method described above.

[0021] The advantages of this invention are: This scheme first calculates the camera's field of view and transforms the point cloud into the camera coordinate system. It then identifies the valid point cloud within the camera's field of view, enabling the filtering of point clouds within the field of view. Finally, it establishes a mapping table from the image to the point cloud, providing a point cloud search method based on the mapping table, which can match corresponding point cloud features for directional feature points. Attached Figure Description

[0022] The above and other objects, features, and advantages of this application will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 The diagram illustrates the structure of a point cloud feature extraction system.

[0024] Figure 2 The diagram illustrates the steps of a point cloud feature extraction method.

[0025] Figure 3 A schematic diagram illustrating point cloud filtering is shown.

[0026] Figure 4 A schematic diagram of the mapping table is shown.

[0027] Figure 5 The illustration shows an image taken by a camera.

[0028] Figure 6 The diagram illustrates a point cloud acquisition process using lidar.

[0029] Figure 7 The illustration shows a schematic diagram of contour extraction from an image.

[0030] Figure 8 A schematic diagram of the feature points of the corresponding contour in the point cloud is shown.

[0031] Figure 9 A schematic diagram of the electronic device is shown.

[0032] Figure 10 A schematic diagram of the structure of a computer medium is shown. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0034] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0035] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0038] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0039] According to a first specific embodiment of the present invention, the present invention provides a point cloud feature extraction system, such as... Figure 1 As shown, it includes: a data acquisition module, an effective point cloud filtering module, a mapping table calculation module, and a feature point calculation module.

[0040] The data acquisition module is used to acquire images captured by the camera and point clouds captured by the lidar; the lidar's field of view is larger than that of the camera.

[0041] The effective point cloud filtering module is used to obtain the effective point cloud within the camera's field of view.

[0042] The mapping table calculation module is used to calculate the mapping table from the image to the point cloud.

[0043] The feature point calculation module is used to obtain feature contour points in the image and calculate the corresponding feature points of the point cloud.

[0044] According to a second specific embodiment of the present invention, the present invention provides a point cloud feature extraction method, such as... Figure 2 As shown, it includes the following steps: S1: Acquire images captured by the camera and point clouds captured by the LiDAR; the LiDAR's field of view is larger than the camera's field of view.

[0045] S2: Obtain the effective point cloud within the camera's field of view.

[0046] Before acquiring the effective point cloud within the camera's field of view, the extrinsic parameters, intrinsic parameters, and distortion of the camera relative to the lidar are obtained. Based on the extrinsic parameters of the camera relative to the lidar, the point cloud is transformed from the lidar coordinate system to the camera coordinate system.

[0047] The effective point cloud within the camera's field of view is the point cloud located within the camera's field of view.

[0048] Obtaining the effective point cloud within the camera's field of view includes: S21: Obtain the position of the normalized phase plane of the midpoint of the four edges of the image.

[0049] First, select the midpoints of the four edges (top, bottom, left, and right) in the image. Then, based on the distortion coefficients in the intrinsic parameter calibration, obtain the pixel positions of these four points before distortion. Finally, based on the pixel positions of the four points, calculate the positions of these four points in the normalized phase plane.

[0050] S22: Calculate the horizontal and vertical field of view of the camera based on four points.

[0051] Connecting the four points to the camera's optical center yields four rays: up, down, left, and right. Calculating the angles of the left and right rays gives the camera's horizontal field of view θ. H The vertical field of view θ of the camera is obtained by calculating the angle between the upper and lower rays. V These two field-of-view angles represent the camera's field of view.

[0052] S23: Mark the point cloud in the opposite direction of the camera's shooting direction as invalid points.

[0053] (x) c y c , z c () represents the coordinates of the point cloud in the camera coordinate system.

[0054] The x-axis represents the horizontal direction of the camera, the y-axis represents the vertical direction of the camera, and the z-axis represents the optical axis of the camera.

[0055] z c A value less than 0 indicates the opposite direction of the camera's shooting direction. When the z-coordinate of a point cloud is less than 0, the point cloud is marked as an invalid point. Figure 3 Point 3 in the diagram is the point whose z-coordinate is less than 0, and is therefore an invalid point.

[0056] S24: Calculate the angle between the point cloud and the camera optical axis in the horizontal direction and the angle between the point cloud and the camera optical axis in the vertical direction. Mark point clouds with a horizontal angle greater than half of the horizontal field of view as invalid points, and mark point clouds with a vertical angle greater than half of the vertical field of view as invalid points.

[0057] ; .

[0058] In the above formula, θ x Let θ be the angle between the point cloud and the camera's optical axis in the horizontal direction. y Let x be the angle between the point cloud and the camera's optical axis in the vertical direction. c y c , z c () represents the coordinates of the point cloud in the camera coordinate system.

[0059] like Figure 3 As shown, A, B, C, and D are four points on the edge of the image, and 0 is the optical center of the camera. ∠A0C is the horizontal field of view of the camera, and ∠BOD is the vertical field of view of the camera. Points 1, 2, and 3 are three points in the radar point cloud. Point 1 is a valid point, point 2 is outside the field of view and is an invalid point, and point 3 is behind the camera and is also an invalid point.

[0060] If the point cloud from the LiDAR is not filtered to select valid point clouds within the camera's field of view, then point 3 can also be projected into the image when building the mapping table, which will affect the extraction of features from the point cloud. In reality, the camera cannot see point 3, so no matter what location of the feature point is selected in the image, it cannot be mapped to an invalid point like point 3 when mapping feature points in the point cloud.

[0061] S25: Point clouds other than invalid points are marked as valid point clouds.

[0062] S3: Calculate the mapping table from the image to the point cloud.

[0063] The process of calculating the mapping table from image to point cloud includes: S31: Create an empty two-dimensional mapping table based on the pixel coordinate system of the image.

[0064] The size of the mapping table is the same as the image size, and the number of tables is the same as the number of pixels. The initial values ​​in the mapping table are all 0.

[0065] S32: Project the point cloud onto the image's mapping table.

[0066] like Figure 4 As shown, using the OpenCV function `projectPoints()`, the camera's intrinsic, extrinsic, and distortion parameters, along with the valid point cloud coordinates in space, are input into the function to obtain the projected pixel coordinates. Then, based on the projected pixel coordinates, the mapping table is updated, projecting each point cloud in 3D space onto the image's mapping table. If multiple point clouds are projected into the same pixel, the point cloud closest to the optical center is selected and projected into the mapping table. Figure 4 The three-dimensional coordinates of the point cloud are written into the mapping table.

[0067] The specific method is as follows: For each projected pixel coordinate, check if the value of this coordinate in the mapping table is 0. If the value is 0, it means that no point cloud has been projected to this position before, so update the value of this position in the mapping table to the corresponding point cloud coordinate; if it is not 0, read the value of this position, which is the 3D coordinate of the point cloud previously projected to this position, calculate the distance from the coordinate to the optical center, i.e., the modulus, and compare it with the modulus of the current point cloud coordinate projected to this position. If the current point cloud is closer to the optical center, i.e., the modulus is smaller, update the value in the mapping table to the current point cloud coordinate; otherwise, do not update and continue to check the next point.

[0068] The mapping table determines the 3D position of a point cloud by the coordinates of that pixel within which it is projected. If multiple point clouds are projected into the same pixel, the point cloud closest to the LiDAR (i.e., the one closest to the optical center) is selected. Once the mapping table is established, given an image point, it can determine whether that point has a corresponding LiDAR point cloud and what its coordinates are.

[0069] S4: Obtain the feature contour points in the image and calculate the corresponding feature points of the point cloud.

[0070] Obtaining feature contour points in an image and calculating the corresponding feature points of the point cloud includes: S41: Detect point features, and / or line features, and / or circle or ellipse features, and / or contour features of the image.

[0071] For point features, a feature point extraction algorithm can be used directly in the image to calculate the pixel coordinates of the feature points in the image, and then the corresponding point cloud feature points can be found using a mapping table.

[0072] The detection point features are achieved using detection SIFT features, SURF features, or ORB features.

[0073] SIFT, or Scale Invariant Feature Transform, is a descriptor used in image processing. This descriptor is scale invariant and can detect key points in an image; it is a local feature descriptor. SIFT features are local features of an image, invariant to rotation, scaling, and brightness changes, and also maintain a certain degree of stability against viewpoint changes, affine transformations, and noise.

[0074] SURF (Speeded Up Robust Features) is an image feature detection and description algorithm in the field of computer vision. As an improved version of the SIFT algorithm, it significantly improves computational efficiency by introducing integral images and box filters to approximate Gaussian filtering. Its core process includes feature point detection, orientation assignment, and descriptor generation, and it is suitable for tasks such as object recognition and 3D reconstruction. The SURF algorithm uses an approximate image of the Hessian matrix determinant instead of the DOG image of SIFT for feature point detection, and uses integral images to accelerate the calculation of the second derivative. In scale space construction, multi-scale analysis is achieved by adjusting the filter size instead of the image size. The algorithm's feature descriptor is constructed based on the Haar wavelet response within a 20s×20s region, generating a 64-dimensional vector (containing 16 sub-regions × 4 feature components), achieving dimensionality reduction compared to the SIFT algorithm. Feature matching is determined by combining Euclidean distance and the sign of the Hessian matrix trace.

[0075] ORB (Oriented FAST and Rotated BRIEF) is an efficient feature extraction algorithm that combines FAST keypoint detection and rotation-invariant BRIEF descriptors, and is widely used in the field of computer vision.

[0076] For line features, the LSD algorithm, EDLine algorithm, or Hough line detection method are used to detect line features.

[0077] Specifically, the ED line detection algorithm, LSD line detection algorithm, or Hough line detection algorithm are first used to detect lines in the image, and the detection results are the two endpoints of the line segment. To obtain the coordinates of the pixels traversed by the line segment, the Bresenham algorithm can be used to calculate the pixels, or the OpenCV function LineIterator can be used directly to calculate the pixels traversed by the line. Finally, the corresponding point cloud features are looked up in the mapping table.

[0078] The Edge Rendering (ED) algorithm, also known as the edge rendering algorithm, is an edge detection algorithm. Unlike traditional edge detection algorithms that determine edge points through gradient threshold magnitude clustering, the ED algorithm first finds anchor points with large gradients and then connects these anchor points using a heuristic edge tracking method; hence the name "edge rendering." The edges calculated by the ED algorithm have good continuity and high quality. Based on the edges detected by the ED algorithm, more advanced edge features, such as lines, circles, and ellipses, can be extracted, leading to algorithms such as the ED line detection algorithm and the ED circle detection algorithm.

[0079] The LSD algorithm is a line detection algorithm that can obtain high-precision line segment detection results in a short time. The LSD algorithm first calculates the gradient magnitude and direction of all points in the image. Then, it treats adjacent points with small gradient direction changes as a connected component. Next, it determines whether each component needs to be broken according to rules to form multiple components with larger rectangularity based on its rectangularity. Finally, it improves and filters all generated components, retaining those that meet the conditions; these are the final line detection results. The advantages of this method are its speed, lack of parameter tuning, and improved accuracy through error control.

[0080] Both the Hough line detection algorithm and the Hough circle detection algorithm are based on the Hough transform. The Hough transform is a feature extraction technique in image processing. It detects objects with specific shapes using a voting algorithm. This process calculates the local maxima of the accumulated results in a parameter space to obtain a set of objects conforming to that specific shape as the result of the Hough transform. It maps curves or lines with the same shape in one space to a point in another space, forming a peak value, thus transforming the problem of detecting arbitrary shapes into a problem of statistical peak values. The Hough transform algorithm has strong anti-interference capabilities, is insensitive to incomplete lines or circles in the image, and has strong noise resistance.

[0081] For circular or elliptical features, the ED ellipse detection algorithm or the circle detection algorithm based on Hough transform is used to detect circular or elliptical features.

[0082] Specifically, the ED ellipse detection algorithm or a circle detection algorithm based on the Hough transform is used to calculate ellipses in the image. The detection result is not pixels, but parameters of the circle or ellipse, such as the origin, radius, or major and minor axes. To obtain the pixels that the circle or ellipse passes through, the OpenCV ellipse2Poly() function can be used to calculate the pixels it passes through based on the parameters of the circle or ellipse. Alternatively, OpenCV's ellipse drawing functions can be used to draw the circle or ellipse on a blank image, and then the pixels it passes through can be found one by one. Finally, the corresponding point cloud features are looked up in a mapping table.

[0083] For contour features, contour feature detection is achieved by calculating image edges and using contour lookup methods or manually drawing contours on the image.

[0084] Specifically, edge detection is first performed in the image, then the contour is obtained, and then the contour is filtered based on the area of ​​the contour, the bounding rectangle and other information to obtain the required contour features. Since the contour is a collection of pixels, the point cloud features corresponding to the contour points can be directly looked up in the mapping table.

[0085] S42: Look up the corresponding point cloud in the mapping table.

[0086] S43: If there is no point cloud at the location of a feature point in the image, search for a point cloud within the 8-neighborhood of the mapping table. If a point cloud is found at the location of a feature point in the image, the search is successful, and the point is recorded.

[0087] The 8-neighborhood refers to the eight table points of the mapping table surrounding the feature point.

[0088] for Figure 4 Given a mapping table, if a feature in the image is a straight line located in the second row, then based on the mapping table, the corresponding point cloud points are: (x0, y0, z0), (x1, y1, z1), and (x2, y2, z2). Among these, (x1, y1, z1) is not on the straight line but was found in the neighborhood.

[0089] S44: Filter the found point cloud to obtain the feature points of the point cloud.

[0090] Filtering the found point cloud includes: S441: Denoise point clouds and exclude discrete point clouds.

[0091] If the number of point clouds around a point is too small, the point is considered a discrete point and is excluded.

[0092] S442: Extract straight lines from the point cloud on the line feature.

[0093] S443: For point clouds on circular or elliptical features and contour features, obtain point clouds on the same plane, perform planar segmentation, and remove points outside the plane.

[0094] When the corresponding point cloud feature is found from the mapping table, filtering is also required to remove mismatched point cloud feature points.

[0095] Since point features in an image are discrete and spatially discrete with no discernible pattern, filtering is not required.

[0096] Line features in an image are also straight lines in space. Therefore, for the found point cloud, spatial line fitting is required, and points that are far from the spatial line are excluded. Thus, step S442 is necessary. For circular or elliptical features in an image, they should lie in a plane in space. Therefore, for the found point cloud, spatial plane fitting can be performed, and point cloud feature points that are far from the plane can be excluded.

[0097] Since contour features in an image cannot be guaranteed to lie in a single plane in space, plane fitting can only be performed to eliminate mismatched points if it is determined that these contours originate from the same plane. Therefore, step S443 is required.

[0098] The decision of which features to detect in an image, and how to filter them after finding the corresponding point cloud features, depends on the specific target being detected.

[0099] This solution associates camera images with point cloud data acquired by LiDAR to obtain a mapping table between the two. Based on this mapping table, the point cloud feature points corresponding to feature points in the image can be calculated. Subsequently, the area, volume, etc. of the target can be calculated based on the point cloud feature points, realizing multi-source sensor data fusion and joint measurement.

[0100] Figure 5 Images taken by a camera.

[0101] Figure 6 Point cloud acquired by lidar.

[0102] Figure 7 The contours extracted from the image.

[0103] Figure 8 The extracted point cloud feature points are the corresponding contour points in the point cloud.

[0104] according to Figures 5-8 It can be observed that more feature points are extracted from the image, while fewer are extracted from the point cloud. This is because the pixel density of the image is much greater than the number of point clouds in the LiDAR, resulting in far fewer point cloud feature points being found than image feature points. Furthermore, due to depth of field and calibration errors, mismatches can occur between image points and point cloud points, and these points will be filtered out. Therefore, this solution first calculates the camera's field of view, transforming the point cloud into the camera coordinate system. It then identifies the effective point cloud within the camera's field of view, filtering the point cloud within the field of view. Finally, it establishes a mapping table from the image to the point cloud, providing a point cloud lookup method based on this table. This method can accurately match directional feature points with corresponding point cloud features, laying a solid foundation for subsequent target volume and area calculations.

[0105] According to a third specific embodiment of the present invention, the present invention provides an electronic device, such as... Figure 9 As shown, Figure 9 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0106] The following reference Figure 9 To describe an electronic device 100 according to this embodiment of the present application. Figure 9 The electronic device 100 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0107] like Figure 9 As shown, the electronic device 100 is presented in the form of a general-purpose computing device. The components of the electronic device 100 may include, but are not limited to: at least one processing unit 110, at least one storage unit 120, a bus 130 connecting different system components (including storage unit 120 and processing unit 110), a display unit 140, etc.

[0108] The storage unit stores program code that can be executed by the processing unit 110, causing the processing unit 110 to perform the steps described in this specification according to various exemplary embodiments of this application. For example, the processing unit 110 can perform the steps shown in the second specific embodiment.

[0109] The storage unit 120 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1201 and / or a cache storage unit 1202, and may further include a read-only memory unit (ROM) 1203.

[0110] The storage unit 120 may also include a program / utility 1204 having a set (at least one) program module 1205, such program module 1205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of these examples or some combination of these may include an implementation of a network environment.

[0111] Bus 130 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0112] Electronic device 100 can also communicate with one or more external devices 100' (e.g., keyboard, pointing device, Bluetooth device, etc.), enabling users to communicate with devices that interact with electronic device 100, and / or any device that allows electronic device 100 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 150. Furthermore, electronic device 100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 160. Network adapter 160 can communicate with other modules of electronic device 100 via bus 130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0113] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware.

[0114] Therefore, according to a fourth specific embodiment of the present invention, the present invention provides a computer-readable medium. For example... Figure 10 As shown, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of the present invention.

[0115] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0116] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0117] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0118] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the functions of the second specific embodiment.

[0119] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0120] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of the present invention.

[0121] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A point cloud feature extraction method, characterized in that, The method comprises the following steps: acquiring an image captured by a camera and a point cloud captured by a laser radar, wherein a field of view range of the laser radar is larger than a field of view range of the camera; acquiring valid point clouds in the field of view range of the camera; calculating a mapping table from the image to the point cloud; acquiring feature contour points in the image and calculating feature points of the corresponding point cloud.

2. The point cloud feature extraction method of claim 1, wherein, The valid point clouds in the field of view range of the camera are point clouds located in the field of view range of the camera.

3. The method of claim 1, wherein, The calculation of the mapping table from the image to the point cloud comprises: establishing an empty two-dimensional mapping table according to a pixel coordinate system of the image; projecting the point cloud into the mapping table of the image.

4. The point cloud feature extraction method of claim 4, wherein, The projection of the point cloud into the mapping table of the image comprises: projecting the point cloud in a three-dimensional space into the mapping table of the image one by one, and if multiple point clouds are projected into the same pixel, selecting the point cloud close to the optical center to be projected into the mapping table.

5. The method of claim 1, wherein, The acquisition of the feature contour points in the image and the calculation of the feature points of the corresponding point cloud comprise: detecting point features, and / or line features, and / or circle or ellipse features, and / or contour features in the image; searching for the corresponding point cloud in the mapping table; if there is no point cloud in the position of the feature point in the image, searching for the point cloud in the 8-neighborhood in the mapping table; filtering the found point cloud to obtain the feature point of the point cloud.

6. The point cloud feature extraction method of claim 5, wherein, The detection of the point features adopts the detection of sift features, surf features or orb features; the detection of the line features adopts the LSD algorithm, the EDLine algorithm or the Hough straight line detection method; the detection of the circle or ellipse features adopts the ED ellipse detection algorithm or the circle detection algorithm based on the Hough transform; the detection of the contour features adopts the calculation of the image edge and the use of the contour search method or the manual drawing of the contour on the image.

7. The method of claim 5, wherein, The filtering of the found point cloud comprises: de-noising the point cloud and excluding discrete point clouds; extracting a straight line on the point cloud of the line feature; acquiring point clouds on the same plane, performing plane segmentation and removing points outside the plane.

8. A point cloud feature extraction system, characterized in that, The method comprises: a data acquisition module, a valid point cloud screening module, a mapping table calculation module and a feature point calculation module; the data acquisition module is used to acquire an image captured by a camera and a point cloud captured by a laser radar, wherein a field of view range of the laser radar is larger than a field of view range of the camera; the valid point cloud screening module is used to acquire valid point clouds in the field of view range of the camera; the mapping table calculation module is used to calculate a mapping table from the image to the point cloud; the feature point calculation module is used to acquire feature contour points in the image and calculate feature points of the corresponding point cloud.

9. An electronic device, comprising: The method comprises: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the point cloud feature extraction method according to any one of claims 1-7.

10. A computer readable medium having stored thereon a computer program, characterized in that, The programs are executed by the processor to implement the point cloud feature extraction method according to any one of claims 1-7.

Images