A feature extraction method and apparatus

Abstract

The embodiment of the application provides a feature extraction method and device. The scheme is as follows: obtaining three-dimensional point cloud data of a target to be extracted; the three-dimensional point cloud data comprises a plurality of three-dimensional points corresponding to the target to be extracted, and position information of each three-dimensional point; determining semantic labels of the three-dimensional points in the three-dimensional point cloud data according to the position information of each three-dimensional point; for each specified semantic label of the target to be extracted, determining a plurality of three-dimensional points corresponding to the specified semantic label as first anchor points based on the three-dimensional points in the three-dimensional point cloud data with the specified semantic label; and extracting a first feature of each first anchor point. Through the technical scheme provided by the embodiment of the application, the accuracy and comprehensiveness of feature extraction are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a feature extraction method and apparatus. Background Technology

[0002] With the continuous development of 3D imaging technology, image processing of 3D images has significant advantages over image processing of 2D images. For example, in the process of face recognition, the analysis and processing of 3D face images can effectively eliminate the influence of factors such as makeup and lighting on the face recognition results, thereby improving the accuracy of face recognition.

[0003] Currently, in the process of 3D image processing, feature extraction of targets in 3D images is the foundation for subsequent matching, recognition, or authentication. Summary of the Invention

[0004] The purpose of this application is to provide a feature extraction method and apparatus to improve the accuracy and comprehensiveness of feature extraction. The specific technical solution is as follows:

[0005] This application embodiment also provides a feature extraction method, the method comprising:

[0006] Acquire 3D point cloud data of the target to be extracted; the 3D point cloud data includes multiple 3D points corresponding to the target to be extracted, and the position information of each 3D point;

[0007] Based on the position information of each three-dimensional point, determine the semantic label of each three-dimensional point in the three-dimensional point cloud data;

[0008] For each specified semantic label of the target to be extracted, based on the three-dimensional points in the three-dimensional point cloud data whose semantic labels are specified semantic labels, multiple three-dimensional points corresponding to the specified semantic labels are determined as the first anchor points;

[0009] Extract the first feature of each first anchor point.

[0010] Optionally, the step of determining multiple 3D points corresponding to the specified semantic label as the first anchor point for each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic labels are the specified semantic labels, includes:

[0011] For each specified semantic label of the target to be extracted, 3D keypoint detection technology is used to determine the key 3D points in the 3D point cloud data that are the semantic labels of the specified semantic labels, which are then used as the first anchor points corresponding to the specified semantic labels; or

[0012] Based on the three-dimensional point cloud data, 3D modeling is performed to obtain a 3D model; for each specified semantic label of the target to be extracted, the vertex of the 3D model that matches the specified semantic label is determined as the first anchor point corresponding to the specified semantic label.

[0013] Optionally, the step of determining multiple 3D points corresponding to the specified semantic label as the first anchor point for each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic labels are the specified semantic labels, includes:

[0014] Based on the semantic labels of each three-dimensional point in the three-dimensional point cloud data, point cloud segmentation technology is used to segment the point cloud to obtain the point cloud region corresponding to each semantic label.

[0015] For each specified semantic label, multiple 3D points are obtained from the point cloud region corresponding to the specified semantic label according to a preset sampling strategy, and used as the first anchor point corresponding to the specified semantic label.

[0016] Optionally, the step of extracting the first feature of each first anchor point includes:

[0017] For each first anchor point, the first anchor point is input into the first model corresponding to the semantic label of the first anchor point, and the first feature of the first anchor point output by the first feature extraction unit in the first model is obtained; or

[0018] The three-dimensional point cloud data is input into the second model corresponding to each specified semantic label. The second feature extraction unit in the second model is used to extract the second features of each three-dimensional point. The feature fusion unit in the second model is used to perform feature fusion on the second features of each three-dimensional point within the preset receptive field range corresponding to each first anchor point, which is used as the first feature of each first anchor point.

[0019] The first model / second model is trained using a preset training set, which includes multiple three-dimensional sample image data and label information corresponding to each three-dimensional sample image data.

[0020] Optionally, before inputting the 3D point cloud data into the second model corresponding to each specified semantic label, the method further includes:

[0021] For each first anchor point, according to the pre-stored correspondence between receptive fields and semantic tags, the receptive field corresponding to the semantic tag of the first anchor point is determined as the preset receptive field of the first anchor point.

[0022] Optionally, the parameters of the first / second model may differ for different semantic labels.

[0023] Optionally, the method further includes:

[0024] The plurality of first anchor points are downsampled to obtain a plurality of second anchor points;

[0025] For each second anchor point, a third feature of the second anchor point is determined based on the first features of a preset number of first anchor points within a preset neighborhood of the second anchor point.

[0026] When the number of second anchor points is greater than a preset number threshold, the second anchor point is used as the first anchor point, and the process of downsampling the multiple first anchor points to obtain multiple second anchor points is repeated until the number of second anchor points is not greater than the preset number threshold. The third feature of each second anchor point at the current time is determined as the target feature of the target to be extracted in the three-dimensional point cloud data.

[0027] Optionally, the step of determining the third feature of each second anchor point based on the first features of a preset number of first anchor points within a preset neighborhood of the second anchor point includes:

[0028] The first anchor points with the same semantic label among the plurality of first anchor points are divided into the same anchor point set;

[0029] For each second anchor point, a predetermined number of first anchor points that are closest to the second anchor point are determined from the set of anchor points to which the second anchor point is located, and these first anchor points are used as the target anchor points corresponding to the second anchor point.

[0030] The third feature of the second anchor point is determined based on the first feature of each target anchor point corresponding to the second anchor point.

[0031] Optionally, the method further includes:

[0032] Calculate the similarity between the target features and the preset features of each target object stored in the preset database;

[0033] Perform the target operation that matches the similarity.

[0034] This application embodiment also provides a feature extraction device, the device further comprising:

[0035] The acquisition module is used to acquire the three-dimensional point cloud data of the target to be extracted; the three-dimensional point cloud data includes multiple three-dimensional points corresponding to the target to be extracted, and the position information of each three-dimensional point;

[0036] The first determining module is used to determine the semantic label of each three-dimensional point in the three-dimensional point cloud data based on the position information of each three-dimensional point.

[0037] The second determining module is used to determine, for each specified semantic label of the target to be extracted, multiple three-dimensional points corresponding to the specified semantic label in the three-dimensional point cloud data based on the three-dimensional points with the specified semantic label, and use them as the first anchor points;

[0038] The extraction module is used to extract the first feature of each first anchor point.

[0039] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0040] Memory, used to store computer programs;

[0041] When a processor executes a program stored in memory, it implements any of the feature extraction method steps described above.

[0042] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the feature extraction method steps described above.

[0043] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the feature extraction methods described above.

[0044] Beneficial effects of the embodiments in this application:

[0045] The technical solution provided in this application embodiment can, after acquiring the 3D point cloud data of the target to be extracted, determine the semantic label of each 3D point based on the position information of each 3D point in the 3D point cloud data. Then, for each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic label is that specified semantic label, determine multiple first anchor points corresponding to each specified semantic label, and then extract the first feature of the first anchor point. By accurately and effectively determining the semantic label of each 3D point through the position information of each 3D point in the 3D point cloud data, it is easier to select anchor points for each specified semantic label in the later stage. This ensures that the anchor points selected include all specified semantic labels, improving the comprehensiveness of the anchor point selection for the target to be extracted. Furthermore, the features extracted based on these anchor points include the features corresponding to all specified semantic labels, improving the accuracy and comprehensiveness of feature extraction.

[0046] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0048] Figure 1 This is a schematic diagram of the first process of the feature extraction method provided in the embodiments of this application;

[0049] Figure 2 This is a second flowchart illustrating the feature extraction method provided in the embodiments of this application;

[0050] Figure 3 This is a schematic diagram of a third process for the feature extraction method provided in the embodiments of this application;

[0051] Figure 4 This is a schematic diagram of the fourth process of the feature extraction method provided in the embodiments of this application;

[0052] Figure 5 A fifth flowchart illustrating the feature extraction method provided in this application embodiment;

[0053] Figure 6 A sixth flowchart illustrating the feature extraction method provided in this application embodiment;

[0054] Figure 7 A seventh flowchart illustrating the feature extraction method provided in this application embodiment;

[0055] Figure 8 The eighth flowchart illustrating the feature extraction method provided in this application embodiment;

[0056] Figure 9 A ninth flowchart illustrating the feature extraction method provided in this application embodiment;

[0057] Figure 10 A schematic diagram of the tenth process of the feature extraction method provided in the embodiments of this application;

[0058] Figure 11 A schematic diagram of the feature extraction device provided in the embodiments of this application;

[0059] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0061] In related technologies, multiple anchor points can be selected from the aforementioned 3D point cloud data using the Farthest Point Sampling (FPS) technique when selecting anchor points. However, the FPS-based anchor point selection process is highly sensitive to noise data, making it easy for 3D points corresponding to outliers in the 3D point cloud data to be selected as anchor points. This results in a reduction in the number of effective points among the selected anchor points, affecting the accuracy of the extracted features later.

[0062] To address the problems in related technologies, embodiments of this application provide a feature extraction method. This method can be applied to any electronic device.

[0063] like Figure 1 As shown, Figure 1 This is a schematic diagram of a first flowchart of a feature extraction method provided in an embodiment of this application. The method includes the following steps.

[0064] Step S101: Obtain the 3D point cloud data of the target to be extracted; the 3D point cloud data contains multiple 3D points corresponding to the target to be extracted, as well as the position information of each 3D point.

[0065] Step S102: Determine the semantic label of each three-dimensional point in the three-dimensional point cloud data based on the position information of each three-dimensional point.

[0066] Step S103: For each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic labels are specified semantic labels, determine multiple 3D points corresponding to the specified semantic label as the first anchor points.

[0067] Step S104: Extract the first feature of each first anchor point.

[0068] In this embodiment, the extracted features differ depending on the target to be extracted, and the uses of these features also differ. For example, when the target to be extracted is a face in a 3D face image, the extracted features can be the facial features corresponding to that face. After extracting the facial features, the electronic device can perform face matching, face recognition, etc., based on these facial features. In this case, the electronic device can be a device for face matching or a device for face recognition. Here, the target to be extracted, the first feature, and the electronic device are not specifically limited.

[0069] pass Figure 1 The method described above, after acquiring the 3D point cloud data of the target to be extracted, determines the semantic label of each 3D point based on its position information. Then, for each specified semantic label of the target, multiple first anchor points corresponding to that semantic label are determined based on the 3D points in the 3D point cloud data. The first feature of each first anchor point is then extracted. By accurately and effectively determining the semantic label of each 3D point through its position information, the method facilitates subsequent anchor point selection for each specified semantic label. This ensures that anchor points covering all specified semantic labels are selected, improving the comprehensiveness of anchor point selection for the target. Consequently, the features extracted based on these anchor points include features corresponding to all specified semantic labels, thus improving the accuracy and comprehensiveness of feature extraction.

[0070] The embodiments of this application will be described below through specific examples.

[0071] Regarding step S101 above, the three-dimensional point cloud data of the target to be extracted is obtained; the three-dimensional point cloud data includes multiple three-dimensional points corresponding to the target to be extracted, as well as the position information of each three-dimensional point.

[0072] In this embodiment of the application, the electronic device can acquire a three-dimensional image of the target to be extracted, thereby acquiring three-dimensional point cloud data of the target to be extracted in the three-dimensional image.

[0073] The aforementioned 3D images can be represented in 3D forms such as point clouds or meshes. A mesh is a 3D representation composed of point clouds and connected surfaces. Here, no specific limitation is made on the representation of the 3D images. For ease of understanding, the following explanation uses a point cloud network as an example and does not constitute any limitation.

[0074] The target to be extracted can be any object in the image acquisition scene, such as people, animals, or vehicles. No specific limitations are imposed on the target. For ease of understanding, the following explanation uses a person in the image acquisition scene as the target and a 3D image of the person's face as the 3D image of the target; this does not constitute any limitation.

[0075] In an optional embodiment, for the target to be extracted in the above-described image acquisition scenario, an RGBD (Red-Green-Blue-Depth) camera can be used to acquire images of the target in the image acquisition scenario, obtaining a Red-Green-Blue (RGB) image and a depth image. An electronic device can acquire the RGB and depth images acquired by the RGBD camera and convert them into a three-dimensional image to obtain a three-dimensional image of the target to be extracted.

[0076] In another optional embodiment, for the target to be extracted in the above-mentioned image acquisition scenario, a three-dimensional image of the target to be extracted can be directly acquired by a 3D scanner / stereo vision system, etc., and the electronic device acquires the three-dimensional image.

[0077] In another optional embodiment, for the target to be extracted in the above-described image acquisition scenario, an image of the target to be extracted can be acquired by a camera to obtain a two-dimensional (2D) image including the target to be extracted. The electronic device acquires the 2D image and uses machine learning methods such as neural networks to perform three-dimensional reconstruction on the acquired 2D image to obtain a three-dimensional image of the target to be extracted.

[0078] In this embodiment of the application, the method of acquiring the three-dimensional image of the target to be extracted is not specifically limited.

[0079] In one optional embodiment, after acquiring the three-dimensional image of the target to be extracted, the electronic device can directly obtain the three-dimensional point cloud data of the target to be extracted from the three-dimensional image.

[0080] The aforementioned 3D point cloud data includes the positional information of multiple 3D points corresponding to the target to be extracted. For example, the 3D coordinate information (also referred to as point cloud information) corresponding to each point in the point cloud network. In addition, the aforementioned 3D point cloud data may also include other additional information. For example, the normals of each 3D point in the point cloud network, the patch relationships in the point cloud network, the color information and curvature information of each point cloud coordinate, etc. Here, no specific limitations are made on the aforementioned 3D point cloud data.

[0081] In another optional embodiment, considering that there may be some differences in the pose of the target to be extracted in the three-dimensional image acquired by the electronic device, after acquiring the three-dimensional image of the target to be extracted, the electronic device can perform pose estimation, normalization and other processing on the target to be extracted in the acquired three-dimensional image.

[0082] For ease of understanding, a 3D face image is used as an example. After acquiring the 3D face image of the target to be extracted, the electronic device can project the 3D face image of the target to be extracted onto a preset coordinate system. This preset coordinate system includes the projection result corresponding to a standard face image (such as a 3D face image of a preset person facing forward). The electronic device can determine the angle information and displacement information of the target to be extracted relative to the preset person based on the position of the 3D face image in the preset coordinate system and the position of the standard face image in the preset coordinate system. The angle information represents the rotation angle of the target's face relative to the preset person's face. The displacement information represents the deviation between the position of the target's face and the position of the preset person's face.

[0083] The electronic device can perform normalization operations such as rotation, translation, scaling, denoising, and repair on the three-dimensional face image of the target to be extracted in a preset coordinate system based on the determined angle and displacement information, so that the error between the normalized three-dimensional face image of the target to be extracted and the standard face image is less than the preset error threshold in dimensions such as size and pose.

[0084] For example, the three-dimensional face image of the target to be extracted is a side profile image of the target. After processing such as pose estimation and normalization, a frontal face image of the target to be extracted can be obtained.

[0085] The specific processing procedures and methods for pose estimation and normalization are not described here.

[0086] Regarding step S102 above, that is, determining the semantic label of each three-dimensional point in the three-dimensional point cloud data based on the position information of each three-dimensional point.

[0087] In this step, after obtaining the three-dimensional point cloud data of the target to be extracted, the electronic device can label the semantic information of each three-dimensional point in the three-dimensional point cloud data according to the RGB image / 2D image, or use machine learning algorithms such as neural networks to determine the semantic information of each three-dimensional point in the three-dimensional point cloud data, and obtain the semantic label of each three-dimensional point in the three-dimensional point cloud data.

[0088] In this embodiment, the method for determining the semantic labels of each 3D point in the 3D point cloud data varies depending on the acquisition method of the 3D image. For example, if the 3D image is acquired by the RGBD camera and there is a clear mapping (one-to-one) relationship between RGB pixels and depth map pixels, the electronic device can determine the semantic information of the pixels in the RGB image as the semantic label of the corresponding 3D point in the 3D image through this mapping relationship. For example, the electronic device can determine the semantic information of the pixels in the eye region of the RGB image as an eye, and the semantic label of the 3D point corresponding to the pixel whose semantic information is eye in the 3D image is also eye. Here, the method for determining the semantic labels of each 3D point in the 3D point cloud data is not specifically limited.

[0089] Depending on the target to be extracted, the semantic labels corresponding to each 3D point in the above 3D point cloud data will also be different. For example, when the above 3D point cloud data is the face data of the above 3D face image, the semantic label of each 3D point in the 3D point cloud data can be one of the facial features (i.e., eyebrows, eyes, ears, nose, and mouth), cheeks, forehead, and chin. When the above 3D point cloud data is the body data of the above 3D animal image, the semantic label of each 3D point in the 3D point cloud data can be one of the animal's head, limbs, body, and tail. Here, no specific limitation is made on the above semantic labels.

[0090] In one optional embodiment, the semantic labels corresponding to each three-dimensional point in the same semantic region may be the same or different.

[0091] To facilitate understanding, we will use the 3D points of the semantic region containing the eyebrows in a 3D face image as an example. A person's eyebrows can include the inner brow region, the middle brow region, the peak brow region, and the outer brow region. We assume that each 3D point in each region has corresponding semantic information. That is, the semantic information of the 3D point located in the inner brow region is the inner brow, the semantic information of the 3D point located in the middle brow region is the middle brow, the semantic information of the 3D point located in the peak brow region is the peak brow, and the semantic information of the 3D point located in the outer brow region is the outer brow.

[0092] For each 3D point located in the eyebrow region, when determining the semantic label of that 3D point, the electronic device can assign the same semantic label—"eyebrow"—to the 3D points in the eyebrow head region, eyebrow waist region, eyebrow peak region, and eyebrow tail region. Alternatively, the electronic device can separately determine the corresponding semantic label for the 3D points included in the eyebrow head region, eyebrow waist region, eyebrow peak region, and eyebrow tail region based on the region where each 3D point is located. For example, the semantic label for a 3D point located in the eyebrow head region is "eyebrow head," and so on, determining the semantic label for each 3D point. Here, no specific limitation is made on the method for determining the semantic label of each 3D point in the above 3D point cloud data.

[0093] In the above embodiments, the semantic labels of each 3D point in the 3D point cloud data are determined solely based on the positional information of each 3D point in the 3D point cloud data. In addition, the electronic device can also comprehensively consider other additional information about each 3D point in the 3D point cloud data, such as the normals of each 3D point, the patch relationships in the point cloud network, the coordinate color information and curvature information of each point cloud, etc.

[0094] To facilitate understanding, let's take the iris of the human eye as an example. The color of the iris varies among different people; common iris colors include brown, dark brown, and tan. When electronic devices determine the semantic labels of 3D points located in the eye region of 3D point cloud data, in addition to considering the location of the iris, they can also determine the 3D points located on the iris based on its color, and then assign the semantic label of these 3D points as "iris."

[0095] The method for determining the semantic labels of each 3D point in the above 3D point cloud data will not be explained in detail here.

[0096] For step S103 above, that is, for each specified semantic label of the target to be extracted, based on the three-dimensional points in the three-dimensional point cloud data whose semantic label is the specified semantic label, multiple three-dimensional points corresponding to the specified semantic label are determined as the first anchor points.

[0097] In this step, multiple specified semantic tags are pre-defined for the target to be extracted. For each specified semantic tag of the target to be extracted, the electronic device can use key point detection technology, point cloud segmentation technology, etc., to determine multiple 3D points corresponding to the specified semantic tag based on the 3D points belonging to the same specified semantic tag in the above 3D point cloud data, and use them as the first anchor points. The method for determining the first anchor points is described below and will not be specifically explained here.

[0098] The first anchor point mentioned above can be a three-dimensional point included in the three-dimensional point cloud data, or it can be a three-dimensional point generated based on the three-dimensional point cloud data.

[0099] For example, the first anchor point mentioned above can be a 3D point in the above 3D point cloud data whose semantic label is a specified semantic label, such as a 3D point belonging to the facial features in a human face.

[0100] For example, the first anchor point mentioned above can also be a 3D point in the 3D point cloud data that is inferred / generated by preset rules based on the 3D points of each 3D point according to the specified semantic label. For example, based on the 3D points corresponding to the left and right corners of the eye, and the positions of the 3D points included in the eye, the 3D point on the midline of the two 3D points representing the left and right corners of the eye, and on the outline of the eye, can be determined as the first anchor point.

[0101] For each specified semantic label, the number of 3D points in the above 3D point cloud data whose semantic label is that specified semantic label can be greater than / equal to / less than the number of the first anchor points corresponding to that specified semantic label.

[0102] In this embodiment of the application, the number of three-dimensional points corresponding to different semantic labels in the three-dimensional point cloud data varies depending on the number of voxels occupied by different semantic labels in the three-dimensional image, and the number of first anchor points corresponding to different specified semantic labels also varies.

[0103] In the above embodiments, only a three-dimensional image of the target to be extracted is used as an example for illustration. For multiple three-dimensional point cloud data of the same type of target to be extracted, the first anchor point of each specified semantic label determined by the above step S103 has semantic consistency. That is, the first anchor point determined in each three-dimensional point cloud data has the same sorting position in each three-dimensional point cloud data, or has completely consistent semantic information, or represents the same specified semantic label.

[0104] The specified semantic labels mentioned above can be all semantic labels from the semantic labels of each 3D point in the 3D point cloud data of the target to be extracted, or they can be a portion of the semantic labels from the semantic labels of each 3D point in the 3D point cloud data of the target to be extracted, such as the semantic labels corresponding to the feature regions in the target to be extracted. The specified semantic labels mentioned above can be set according to the target to be extracted, user needs, etc., and no specific limitation is made here.

[0105] For step S104 above, that is, extracting the first feature of each first anchor point.

[0106] In this step, the electronic device can use a pre-trained neural network model to extract features from each of the first anchor points determined in step S103 above, obtaining the features of each first anchor point (i.e., point cloud features, denoted as the first feature). The extraction method of the first feature is described below and will not be specifically explained here.

[0107] In this embodiment, since the selection of the first anchor point is performed on the 3D points of the specified semantic label, the influence of outlier noise on the anchor point selection is reduced, the probability of the 3D points corresponding to outlier noise in the 3D point cloud data being selected as anchor points is greatly reduced, the number of effective 3D points among the multiple anchor points selected is increased, thereby improving the accuracy of the features extracted later.

[0108] In an optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of a second flowchart of the feature extraction method provided in an embodiment of this application. Figure 2 In the method shown, step S103 is specifically represented by the following steps, namely step S1031.

[0109] Step S1031: For each specified semantic label of the target to be extracted, use 3D key point detection technology to determine the key 3D point of the semantic label in the 3D point cloud data as the first anchor point corresponding to the specified semantic label.

[0110] To facilitate understanding, let's take the eye as an example, with the specified semantic label. Electronic devices can use 3D keypoint detection technology to determine key points in 3D point cloud data that belong to the eye. For example, by selecting 3D points in the 3D point cloud data that belong to the corner of the eye, upper eyelid, lower eyelid, iris, pupil, etc., the key points corresponding to the eye in the 3D point cloud data are obtained.

[0111] In this embodiment, the electronic device can determine each key point detected in the 3D point cloud data corresponding to a specified semantic label as a first anchor point. In related technologies, key points can be set by the user according to their needs. Typically, users will set more key points in areas with higher recognizability, such as the facial features or cheeks of a face; or key points can be set as corner points or equal division points on a contour, meaning that feature areas with corner points or contours have more key points. For example, the facial features or cheeks of a face have corresponding corner points or contours, thus having more key points. It is evident that the number of first anchor points detected in the facial features, cheeks, and other feature areas through the above step S1031 for a 3D face image is significantly higher than the number of first anchor points detected in feature areas such as the forehead. Therefore, through the above step S1031, the number of first anchor points in the 3D point cloud data with high recognizability feature areas is relatively large, thereby making the features extracted based on the first anchor points more concentrated in the high recognizability feature areas of the target to be extracted, effectively improving the accuracy of the features extracted based on the first anchor points. Here, no specific limit is made on the number of first anchor points determined in each feature region of the above three-dimensional point cloud data.

[0112] In another alternative embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of a third process for the feature extraction method provided in an embodiment of this application. Figure 3 In the method shown, step S103 is specifically represented by the following steps, namely step S1032-step S1033.

[0113] Step S1032: Perform 3D modeling based on 3D point cloud data to obtain a 3D model.

[0114] For ease of understanding, we will still use a 3D face image as an example. Electronic devices can construct a 3DMM model of the target to be extracted based on the 3D point cloud data in the 3D face image using 3D Morphable Model (3DMM) technology.

[0115] Step S1033: For each specified semantic label of the target to be extracted, determine the vertex of the 3D model that matches the specified semantic label as the first anchor point corresponding to the specified semantic label.

[0116] The aforementioned vertices can be partial vertices in the 3D model, or they can be called corner points.

[0117] Through steps S1032-S1033, the 3D model is constructed. The feature region of the target to be extracted has relatively many details, and therefore the number of vertices is also relatively large. Therefore, by determining the vertices matching the specified semantic label as the first anchor points, the determined first anchor points are concentrated in the feature region of the target to be extracted, effectively increasing the number of effective points among the extracted first anchor points, thereby improving the accuracy of the features extracted based on the first anchor points.

[0118] In another optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of the fourth process of the feature extraction method provided in the embodiments of this application. Figure 4 In the method shown, step S103 is specifically represented by the following steps, namely step S1034-step S1035.

[0119] Step S1034: Based on the semantic labels of each 3D point in the 3D point cloud data, point cloud segmentation technology is used to segment the point cloud to obtain the point cloud region corresponding to each semantic label.

[0120] For ease of understanding, we will still use a 3D human face image as an example. When electronic devices use point cloud segmentation technology to perform point cloud segmentation, they can perform point cloud segmentation for each semantic label of each target to be extracted, such as the facial features, cheeks, forehead, chin, and other regions mentioned above, to obtain the point cloud region corresponding to each semantic label.

[0121] Step S1035: For each specified semantic label, obtain multiple 3D points from the point cloud region corresponding to the specified semantic label according to a preset sampling strategy, and use them as the first anchor point corresponding to the specified semantic label.

[0122] In this step, for each specified semantic tag corresponding to a point cloud region, the electronic device can use a preset sampling strategy to acquire multiple 3D points from the point cloud region corresponding to each specified semantic tag, thus obtaining the first anchor point. For example, the electronic device can use FPS technology to acquire multiple 3D points from the point cloud region corresponding to a specified semantic tag, thus obtaining the first anchor point.

[0123] Besides FPS technology, the aforementioned preset sampling strategy can also be other sampling strategies, such as random sampling strategies, strategies to minimize quadratic surface error, and custom sampling strategies. No specific limitations are imposed on the aforementioned preset sampling strategy here.

[0124] In an optional embodiment, in order to further improve the accuracy of the features extracted in the later feature extraction, before performing the above step S1035, for each specified semantic tag, the correspondence between the specified semantic tag and the number of anchor points collected can be preset, so that when performing the above step S1035, the first anchor point matching the number of anchor points collected for each specified semantic tag can be obtained.

[0125] The correspondence between the specified semantic tags and the number of anchor points collected can be set according to user needs. For example, the number of anchor points collected for facial features and cheeks can be the first number, and the number of anchor points collected for the forehead and chin can be the second number. The first number can be greater than or equal to the second number.

[0126] The aforementioned anchor point collection count can be expressed as the anchor point collection count per unit voxel, or as the total anchor point collection count corresponding to each semantic tag. Here, no specific limitation is made on the aforementioned anchor point collection count.

[0127] Through the above steps S1034-S1035, the electronic device can acquire anchor points for each specified semantic tag, effectively improving the comprehensiveness of the acquired anchor points, thereby improving the accuracy of the features extracted based on the anchor points.

[0128] In an optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 5 As shown, Figure 5 This is a fifth flowchart illustrating the feature extraction method provided in this application embodiment. Figure 5 In the method shown, step S104 can be specifically represented as the following steps, namely step S1041.

[0129] Step S1041: For each first anchor point, input the first anchor point into the first model corresponding to the semantic label of the first anchor point, and obtain the first feature of the first anchor point output by the first feature extraction unit in the first model.

[0130] The first feature extraction unit in the first model described above is used for features of a single 3D point. The first model can be a neural network model such as a Multi-Layer Perceptron (MLP).

[0131] For ease of understanding, the above-mentioned MLP is used as an example to illustrate the concept. In this case, the first feature extraction unit in the first model can be a fully connected layer of the MLP. The electronic device can determine the first model corresponding to the specified semantic label of each first anchor point. The electronic device can input each first anchor point into its corresponding first model and obtain the output of the fully connected layer to obtain the first feature of each first anchor point.

[0132] The first model mentioned above is trained using a preset training set (denoted as the first training set). The preset training set includes multiple three-dimensional sample image data (denoted as the first sample image data) and label information corresponding to each three-dimensional sample image data (denoted as the first label information).

[0133] In this embodiment, the three-dimensional sample image data and the first label information vary depending on the intended use of the extracted first feature. For example, when the features extracted by the first model are used for face recognition, the three-dimensional sample image data can be image data from a three-dimensional face image corresponding to a specific person, and the first label information is used to indicate the person information corresponding to each three-dimensional sample image data. As another example, if each three-dimensional sample image data is image data corresponding to two sample three-dimensional face images, and the extracted features are used for face matching, then the first label information is used to indicate whether the person in the two three-dimensional face images corresponding to the three-dimensional sample image data is the same person. Here, the three-dimensional sample image data and label information in the first preset training set are not specifically limited.

[0134] In addition, the training method for the first model mentioned above can be referred to the training method for neural network models in related technologies, and will not be specifically explained here.

[0135] Through the above step S1041, the electronic device can use the first model to extract features for each first anchor point, effectively improving the convenience and accuracy of the first features of each first anchor point.

[0136] In an optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 6 As shown, Figure 6 This is a sixth flowchart illustrating the feature extraction method provided in this application embodiment. Figure 6In the method shown, step S104 can be specifically represented as the following steps, namely step S1042.

[0137] Step S1042: Input the 3D point cloud data into the second model corresponding to each specified semantic label, use the second feature extraction unit in the second model to extract the second features of each 3D point, and use the feature fusion unit in the second model to perform feature fusion on the second features of each 3D point within the preset receptive field range corresponding to each first anchor point, and use them as the first features of each first anchor point.

[0138] The second model mentioned above includes a second feature extraction unit and a feature fusion unit. The second feature extraction unit is used to extract features from a single three-dimensional point, and the feature fusion unit is used to fuse features from each three-dimensional point within a preset receptive field.

[0139] The second model mentioned above is a convolutional neural network model. This second model includes, but is not limited to, Graph Convolutional Networks (GCN), Point Convolutional Networks (PointConv), and Point Transformer. Among them, Point Transformer is a point cloud neural network based on a self-attention mechanism. No specific limitations are imposed on the second model mentioned above.

[0140] The second model mentioned above is trained using a preset training set (denoted as the second training set). The preset training set includes multiple three-dimensional sample image data (denoted as the second sample image data) and label information corresponding to each three-dimensional sample image data (denoted as the second label information).

[0141] The second training set mentioned above may be the same as or different from the first training set mentioned above. The description of the second sample image data and the second label information in the second training set can be found in the description of the first sample image data and the first label information mentioned above, and will not be elaborated upon here.

[0142] The training method for the second model mentioned above can refer to the training method for neural network models in related technologies, and will not be described in detail here.

[0143] Through the above step S1042, when the electronic device extracts features from each first anchor point, it can fuse the features of each three-dimensional point within the preset receptive field of each first anchor point, thereby enabling the first feature of each first anchor point to be fused with the features of its surrounding three-dimensional points, effectively improving the accuracy of the extracted features.

[0144] In the embodiments of this application, when the electronic device performs feature extraction using the first model / second model corresponding to each specified semantic tag, the parameters of the first model / second model may be the same or different.

[0145] In an optional embodiment, when the parameters of the first model / second model corresponding to different semantic tags are different, that is, the parameters of the first model / second model corresponding to each specified semantic tag are different, the electronic device can use the first model / second model with different parameters to extract features for the first anchor point belonging to different specified semantic tags, thereby effectively improving the model's feature extraction capability for different feature regions and improving the accuracy of the extracted features.

[0146] In an optional embodiment, for the first model and the second model described above, when the feature extraction unit in the first model or the second model includes a large number of network layers, the parameters of the first model or the second model can be the same; when the feature extraction unit in the first model or the second model includes a small number of network layers, the parameters of the first model or the second model can be different.

[0147] By adjusting the number of network layers included in the feature extraction units of the first and second models, when the number of network layers is small, using the first or second model with different parameters to extract features for the first anchor point of different specified semantic labels can effectively improve the model's feature extraction capability for different feature regions and improve the accuracy of the extracted features.

[0148] In an optional embodiment, according to the above... Figure 6 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 7 As shown, Figure 7 This is a seventh flowchart illustrating the feature extraction method provided in this application embodiment. Figure 7 In the method shown, before performing step S1042, the electronic device may also perform the following step, namely step S1043.

[0149] Step S1043: For each first anchor point, according to the pre-stored correspondence between receptive fields and semantic tags, the receptive field corresponding to the semantic tag of the first anchor point is determined as the preset receptive field of the first anchor point.

[0150] In one optional embodiment, the size of the receptive field corresponding to each semantic label is different in the pre-stored correspondence between receptive fields and semantic labels.

[0151] For ease of understanding, we will continue to use a 3D face image as an example. For semantic labels such as cheeks and forehead in the 3D point cloud data, the receptive field corresponding to that semantic label in the pre-stored correspondence between receptive fields and semantic labels is the first value. For semantic labels such as facial features and chin in the 3D point cloud data, the receptive field corresponding to that semantic label in the pre-stored correspondence between receptive fields and semantic labels is the second value.

[0152] The first and second values ​​mentioned above can be the same or different. No specific limitations are imposed on the first and second values ​​here.

[0153] In the embodiments of this application, the correspondence between the receptive field and the semantic label can be set according to the spatial size, curvature information or user requirements of each semantic label in the three-dimensional point cloud data. Here, the setting of the correspondence between the receptive field and the semantic label is not specifically limited.

[0154] In the above embodiments, by setting a receptive field corresponding to each semantic tag, the preset receptive field corresponding to each specified semantic tag is different when the electronic device performs the above step S1042, thereby improving the accuracy of the first features of each first anchor point extracted.

[0155] In an optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 8 As shown, Figure 8 This is an eighth flowchart illustrating the feature extraction method provided in this application embodiment. The method includes the following steps.

[0156] Step S801: Obtain the 3D point cloud data of the target to be extracted; the 3D point cloud data contains multiple 3D points corresponding to the target to be extracted, as well as the position information of each 3D point.

[0157] Step S802: Determine the semantic label of each three-dimensional point in the three-dimensional point cloud data based on the position information of each three-dimensional point.

[0158] Step S803: For each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic labels are specified semantic labels, determine multiple 3D points corresponding to the specified semantic label as the first anchor points.

[0159] Step S804: Extract the first feature of each first anchor point.

[0160] The steps S801-S804 described above are the same as the steps S101-S104 described above.

[0161] Step S805: Downsample multiple first anchor points to obtain multiple second anchor points.

[0162] In an optional embodiment, the electronic device may employ a sampling strategy similar to PointNet++ to perform FPS on the aforementioned plurality of first anchor points to obtain a plurality of second anchor points. The number of second anchor points is less than the number of first anchor points.

[0163] In the above embodiments, the sampling strategy used during downsampling can also be minimizing quadratic surface error, a custom sampling strategy, etc. Here, no specific limitation is made on the sampling strategy used in the downsampling process.

[0164] Step S806: For each second anchor point, determine the third feature of the second anchor point based on the first features of a preset number of first anchor points within the preset neighborhood of the second anchor point.

[0165] In an optional embodiment, for each second anchor point, the electronic device may use the K Nearest Neighbors (KNN) algorithm to aggregate the first features of the K first anchor points that have the smallest distance from the second anchor point, and determine the aggregated features as the features of the second anchor point (denoted as the third feature).

[0166] The above aggregation processing methods include, but are not limited to, max pooling, average pooling, and feature learning based on neighborhood information, such as GCN and PointConv.

[0167] In the above aggregation process, the value of K corresponding to the above KNN algorithm can be set according to the size of the feature region where the second anchor point is located, the number of first anchor points included in the feature region where the second anchor point is located, and user requirements, etc. Here, the value of K is not specifically limited.

[0168] Step S807: When the number of second anchor points is greater than the preset number threshold, the second anchor points are used as first anchor points, and the process of downsampling multiple first anchor points to obtain multiple second anchor points is repeated until the number of second anchor points is not greater than the preset number threshold. The third feature of each second anchor point at the current time is determined as the target feature of the target to be extracted in the three-dimensional point cloud data.

[0169] In this step, after obtaining the third features of multiple second anchor points, the electronic device can compare the number of second anchor points with a preset number threshold. If the number of second anchor points is greater than the preset number threshold, the electronic device can use the second anchor points as first anchor points and return to execute the above step S805, that is, return to execute the step of downsampling multiple first anchor points to obtain multiple second anchor points, until the number of second anchor points is not greater than the preset number threshold, and the third features of each second anchor point at the current time are determined as the target features of the target to be extracted in the 3D point cloud data.

[0170] The above-mentioned preset quantity threshold can be set based on user experience. Here, no specific limitation is made on the above-mentioned preset quantity threshold.

[0171] Through steps S805-S807, the electronic device can effectively improve the effectiveness of feature aggregation by downsampling the first anchor point and aggregating the first feature, and also increases the recognizability and interpretability of the extracted target features.

[0172] In an optional embodiment, according to the above... Figure 8 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 9 As shown, Figure 9 This is a ninth flowchart illustrating the feature extraction method provided in this application embodiment. Figure 9 In the method shown, step S806 is further refined into the following steps, namely step S8061-step S8063.

[0173] Step S8061: Divide the first anchors with the same semantic label among the multiple first anchors into the same anchor set.

[0174] exist Figure 9 In the illustrated embodiment, step S8061 is executed after step S605. Alternatively, step S8061 can be executed after any step between steps S803 and S805, or simultaneously with step S805. Here, the execution order of step S8061 and steps S803-S805 is not specifically limited.

[0175] Step S8062: For each second anchor point, determine a preset number of first anchor points that are closest to the second anchor point from the set of anchor points to which the second anchor point is located, and use them as the target anchor points corresponding to the second anchor point.

[0176] Step S8063: Determine the third feature of the second anchor point based on the first feature of each target anchor point corresponding to the second anchor point.

[0177] In steps S8062-S8063 above, for each second anchor point, the electronic device can determine a preset number of first anchor points (i.e., the aforementioned target anchor points) closest to the second anchor point in the anchor point set where the second anchor point is located, and then aggregate the first features of these preset number of target anchor points to obtain the third feature of the second anchor point. Since the target anchor point corresponding to each second anchor point is in the same anchor point set as the second anchor point, the second anchor point and the target anchor point corresponding to the second anchor point belong to the same specified semantic label, which improves the semantic consistency of the features processed by the aggregation, thereby improving the accuracy of the third feature obtained by aggregation.

[0178] In an optional embodiment, according to the above... Figure 8 The method shown in this application embodiment also provides a feature extraction method. For example... Figure 10 As shown, Figure 10 This is a tenth flowchart illustrating the feature extraction method provided in this application embodiment. The method includes the following steps.

[0179] Step S1001: Obtain the 3D point cloud data of the target to be extracted; the 3D point cloud data contains multiple 3D points corresponding to the target to be extracted, as well as the position information of each 3D point.

[0180] Step S1002: Determine the semantic label of each three-dimensional point in the three-dimensional point cloud data based on the position information of each three-dimensional point.

[0181] Step S1003: For each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic labels are specified semantic labels, determine multiple 3D points corresponding to the specified semantic label as the first anchor points.

[0182] Step S1004: Extract the first feature of each first anchor point.

[0183] Step S1005: Downsample multiple first anchor points to obtain multiple second anchor points.

[0184] Step S1006: For each second anchor point, determine the third feature of the second anchor point based on the first features of a preset number of first anchor points within the preset neighborhood of the second anchor point.

[0185] Step S1007: When the number of second anchor points is greater than the preset number threshold, the second anchor points are used as first anchor points, and the process is repeated to downsample multiple first anchor points to obtain multiple second anchor points until the number of second anchor points is not greater than the preset number threshold. The third feature of each second anchor point at the current time is determined as the target feature of the target to be extracted in the three-dimensional point cloud data.

[0186] The steps S1001-S1007 described above are the same as the steps S801-S807 described above.

[0187] Step S1008: Calculate the similarity between the target features and the preset features of each target object stored in the preset database.

[0188] The aforementioned similarity is expressed as the cosine distance, Euclidean distance, etc., between the target feature and the preset features. No specific limitations are placed on the calculation method for this similarity.

[0189] Step S1009: Perform the target operation that matches the similarity.

[0190] In an optional embodiment, if the target features are used for face recognition, the preset database may also store personnel information for each target. When performing step S1009, the electronic device can identify the personnel information corresponding to the preset feature in the preset database that has the highest similarity to the target features as the personnel information of the target to be extracted, and output this personnel information.

[0191] In another optional embodiment, if the target feature is used for face matching, the electronic device can determine the maximum similarity between the target feature and preset features in a preset database, and perform the corresponding target operation when the maximum similarity is greater than a preset similarity threshold. This could include outputting a face matching result or performing a face unlock operation.

[0192] In this embodiment, the target operation performed in step S1009 varies depending on the extracted target features. No specific limitation is made to the target operation described above.

[0193] Through the above steps S1008-S1009, the electronic device can apply the extracted features of the target to be extracted to fields such as face recognition and face matching, thereby improving the accuracy of face recognition or face matching.

[0194] Based on the same inventive concept, and according to the feature extraction method provided in the above embodiments of this application, this application also provides a feature extraction device. For example... Figure 11 As shown, Figure 11 This is a schematic diagram of a feature extraction device provided in an embodiment of this application. The device includes the following modules.

[0195] The acquisition module 1101 is used to acquire the three-dimensional point cloud data of the target to be extracted; the three-dimensional point cloud data includes multiple three-dimensional points corresponding to the target to be extracted, as well as the position information of each three-dimensional point;

[0196] The first determining module 1102 is used to determine the semantic label of each three-dimensional point in the three-dimensional point cloud data based on the position information of each three-dimensional point.

[0197] The second determining module 1103 is used to determine multiple three-dimensional points corresponding to the specified semantic label for each specified semantic label of the target to be extracted, based on the three-dimensional points in the three-dimensional point cloud data whose semantic label is the specified semantic label, and use them as the first anchor points;

[0198] Extraction module 1104 is used to extract the first feature of each first anchor point.

[0199] Optionally, the second determining module 1103 described above can be specifically used to, for each specified semantic label of the target to be extracted, utilize 3D key point detection technology to determine the key 3D points in the 3D point cloud data that are the semantic labels of the specified semantic labels, and use them as the first anchor points corresponding to the specified semantic labels; or

[0200] 3D modeling is performed based on 3D point cloud data to obtain a 3D model; for each specified semantic label of the target to be extracted, the vertex that matches the specified semantic label in the 3D model is determined as the first anchor point corresponding to the specified semantic label.

[0201] Optionally, the second determining module 1103 mentioned above can be used to perform point cloud segmentation based on the semantic labels of each three-dimensional point in the three-dimensional point cloud data, using point cloud segmentation technology to obtain the point cloud region corresponding to each semantic label.

[0202] For each specified semantic label, multiple 3D points are obtained from the point cloud region corresponding to the specified semantic label according to a preset sampling strategy, and used as the first anchor point corresponding to the specified semantic label.

[0203] Optionally, the extraction module 1104 described above can be specifically used to, for each first anchor point, input the first anchor point into the first model corresponding to the semantic label of the first anchor point, and obtain the first feature of the first anchor point output by the first feature extraction unit in the first model; or

[0204] The three-dimensional point cloud data is input into the second model corresponding to each specified semantic label. The second feature extraction unit in the second model is used to extract the second features of each three-dimensional point. The feature fusion unit in the second model is used to perform feature fusion on the second features of each three-dimensional point within the preset receptive field range corresponding to each first anchor point, which is used as the first feature of each first anchor point.

[0205] The first model / second model is trained using a preset training set, which includes multiple 3D sample image data and label information corresponding to each 3D sample image data.

[0206] Optionally, the feature extraction device described above may further include:

[0207] The third determining module is used to determine the receptive field corresponding to the semantic label of each first anchor point as the preset receptive field of the first anchor point before inputting the three-dimensional point cloud data into the second model corresponding to each specified semantic label.

[0208] Optionally, the parameters of the first / second model may differ for different semantic labels.

[0209] Optionally, the feature extraction device described above may further include:

[0210] The downsampling module is used to downsample multiple first anchor points to obtain multiple second anchor points;

[0211] The fourth determining module is used to determine the third feature of each second anchor point based on the first features of a preset number of first anchor points within a preset neighborhood of the second anchor point.

[0212] The fifth determination module is used to, when the number of second anchor points is greater than a preset number threshold, take the second anchor points as first anchor points, return to call the downsampling module to perform downsampling on multiple first anchor points to obtain multiple second anchor points, until the number of second anchor points is not greater than the preset number threshold, and determine the third feature of each second anchor point at the current time as the target feature of the target to be extracted in the 3D point cloud data.

[0213] Optionally, the fourth determining module mentioned above is specifically used to divide the first anchor points with the same semantic tag among multiple first anchor points into the same anchor point set;

[0214] For each second anchor point, a predetermined number of first anchor points that are closest to the second anchor point are determined from the set of anchor points to which the second anchor point is located, and these first anchor points are used as the target anchor points corresponding to the second anchor point.

[0215] The third feature of the second anchor point is determined based on the first feature of each target anchor point corresponding to the second anchor point.

[0216] Optionally, the feature extraction device described above may further include:

[0217] The calculation module is used to calculate the similarity between the target features and the preset features of each target object stored in the preset database;

[0218] The execution module is used to perform target operations that match similarity.

[0219] The apparatus provided in this application embodiment can, after acquiring the 3D point cloud data of the target to be extracted, determine the semantic label of each 3D point based on the position information of each 3D point in the 3D point cloud data. Then, for each specified semantic label of the target to be extracted, based on the 3D points of that semantic label in the 3D point cloud data, determine multiple first anchor points corresponding to each specified semantic label, and then extract the first feature of the first anchor point. By accurately and effectively determining the semantic label to which each 3D point belongs through the position information of each 3D point in the 3D point cloud data, it is easier to select anchor points for each specified semantic label in the later stages. This ensures that the anchor points selected include all specified semantic labels, improving the comprehensiveness of the anchor point selection for the target to be extracted. Furthermore, the features extracted based on these anchor points include the features corresponding to all specified semantic labels, improving the accuracy and comprehensiveness of feature extraction.

[0220] Based on the same inventive concept, and according to the feature extraction method provided in the above embodiments of this application, this application also provides an electronic device, such as... Figure 12 As shown, it includes a processor 1201, a communication interface 1202, a memory 1203, and a communication bus 1204. The processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204.

[0221] Memory 1203 is used to store computer programs;

[0222] When processor 1201 executes the program stored in memory 1203, it performs the following steps:

[0223] Acquire the 3D point cloud data of the target to be extracted; the 3D point cloud data contains multiple 3D points corresponding to the target to be extracted, as well as the position information of each 3D point;

[0224] Based on the location information of each 3D point, determine the semantic label of each 3D point in the 3D point cloud data;

[0225] For each specified semantic label of the target to be extracted, based on the 3D points in the 3D point cloud data whose semantic labels are specified semantic labels, determine multiple 3D points corresponding to the specified semantic labels as the first anchor points;

[0226] Extract the first feature of each first anchor point.

[0227] The electronic device provided in this application embodiment can, after acquiring the 3D point cloud data of the target to be extracted, determine the semantic label of each 3D point based on the position information of each 3D point in the 3D point cloud data. Then, for each specified semantic label of the target to be extracted, based on the 3D points of that semantic label in the 3D point cloud data, determine multiple first anchor points corresponding to each specified semantic label, and then extract the first feature of the first anchor point. By accurately and effectively determining the semantic label to which each 3D point belongs through the position information of each 3D point in the 3D point cloud data, it is easier to select anchor points for each specified semantic label in the later stages. This ensures that the anchor points selected include all specified semantic labels, improving the comprehensiveness of the anchor point selection for the target to be extracted. Furthermore, the features extracted based on these anchor points include the features corresponding to all specified semantic labels, improving the accuracy and comprehensiveness of feature extraction.

[0228] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0229] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0230] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0231] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0232] Based on the same inventive concept, and according to the feature extraction method provided in the above embodiments of this application, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above feature extraction methods.

[0233] Based on the same inventive concept, and according to the feature extraction method provided in the above embodiments of this application, this application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute any of the feature extraction methods in the above embodiments.

[0234] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0235] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0236] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments such as apparatuses, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0237] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A feature extraction method characterized by, The method comprises: acquiring three-dimensional point cloud data of a target to be extracted; the three-dimensional point cloud data comprises a plurality of three-dimensional points corresponding to the target to be extracted, and position information of each three-dimensional point; determining semantic labels of each three-dimensional point in the three-dimensional point cloud data according to the position information of each three-dimensional point; for each specified semantic label of the target to be extracted, determining a plurality of three-dimensional points corresponding to the specified semantic label as first anchor points based on three-dimensional points with the specified semantic label in the three-dimensional point cloud data; wherein the specified semantic label is a plurality of specified semantic labels preset for the target to be extracted; extracting first features of each first anchor point.

2. The method of claim 1, wherein, The step of determining, for each specified semantic label of the target to be extracted, a plurality of three-dimensional points corresponding to the specified semantic label as first anchor points based on three-dimensional points with the specified semantic label in the three-dimensional point cloud data comprises: for each specified semantic label of the target to be extracted, determining key three-dimensional points with the specified semantic label in the three-dimensional point cloud data as first anchor points corresponding to the specified semantic label by using a three-dimensional (3D) key point detection technology; or performing 3D modeling based on the three-dimensional point cloud data to obtain a 3D model, and determining vertices of the 3D model matched with each specified semantic label of the target to be extracted as first anchor points corresponding to the specified semantic label.

3. The method of claim 1, wherein, The step of determining, for each specified semantic label of the target to be extracted, a plurality of three-dimensional points corresponding to the specified semantic label as first anchor points based on three-dimensional points with the specified semantic label in the three-dimensional point cloud data comprises: performing point cloud segmentation to obtain point cloud regions corresponding to each semantic label according to the semantic labels of each three-dimensional point in the three-dimensional point cloud data by using a point cloud segmentation technology; for each point cloud region corresponding to each specified semantic label, obtaining a plurality of three-dimensional points from the point cloud region corresponding to the specified semantic label as first anchor points corresponding to the specified semantic label according to a preset sampling strategy.

4. The method of claim 1, wherein, The step of extracting first features of each first anchor point comprises: for each first anchor point, inputting the first anchor point into a first model corresponding to the semantic label of the first anchor point to obtain first features of the first anchor point output by a first feature extraction unit in the first model; or inputting the three-dimensional point cloud data into a second model corresponding to each specified semantic label, extracting second features of each three-dimensional point by using a second feature extraction unit in the second model, and performing feature fusion on the second features of each three-dimensional point within a preset receptive field range of each first anchor point corresponding to each specified semantic label by using a feature fusion unit in the second model as first features of each first anchor point; wherein the first model / second model is obtained by training a preset training set, and the preset training set comprises a plurality of three-dimensional sample image data and label information corresponding to each three-dimensional sample image data.

5. The method of claim 4, wherein, Before inputting the three-dimensional point cloud data into the second model corresponding to each specified semantic label, the method further comprises: For each first anchor point, a receptive field corresponding to the semantic label of the first anchor point is determined as a preset receptive field of the first anchor point according to a preset correspondence between receptive fields and semantic labels.

6. The method according to claim 4 or 5, characterized in that, The parameters of the first model / second model corresponding to different semantic labels are different.

7. The method of claim 1, wherein, The method further comprises: down-sampling the first anchor points to obtain second anchor points; For each second anchor point, third features of the second anchor point are determined according to first features of a preset number of first anchor points within a preset neighborhood range of the second anchor point. When the number of second anchor points is greater than a preset number threshold, the second anchor points are taken as the first anchor points, and the step of down-sampling the first anchor points to obtain second anchor points is executed again until the number of second anchor points is not greater than the preset number threshold, and the third features of the second anchor points at the current time are determined as target features of a target to be extracted in the three-dimensional point cloud data.

8. The method of claim 7, wherein, The step of determining, for each second anchor point, third features of the second anchor point according to first features of a preset number of first anchor points within a preset neighborhood range of the second anchor point comprises: first anchor points with the same semantic label are divided into the same anchor point set; For each second anchor point, a preset number of first anchor points closest to the second anchor point are determined from the anchor point set in which the second anchor point is located as target anchor points corresponding to the second anchor point; third features of the second anchor point are determined according to first features of each target anchor point corresponding to the second anchor point.

9. The method according to claim 7 or 8, characterized in that, The method further comprises: a similarity between the target features and preset features of each target object stored in a preset database is calculated; a target operation matching the similarity is executed.

10. A feature extraction apparatus characterized by comprising: The device further comprises: an acquisition module configured to acquire three-dimensional point cloud data of a target to be extracted; the three-dimensional point cloud data comprises a plurality of three-dimensional points corresponding to the target to be extracted and position information of each three-dimensional point; a first determination module configured to determine semantic labels of the three-dimensional points in the three-dimensional point cloud data according to the position information of each three-dimensional point; a second determination module configured to, for each specified semantic label of the target to be extracted, determine a plurality of three-dimensional points corresponding to the specified semantic label as first anchor points based on three-dimensional points with the specified semantic label in the three-dimensional point cloud data; the specified semantic label is a plurality of specified semantic labels preset for the target to be extracted; an extraction module configured to extract first features of each first anchor point.

Images