Data processing method and device, computer readable storage medium

Abstract

The application discloses a kind of data processing method, device, computer readable storage medium.Therein, the method includes: obtaining the point cloud data including multiple sub-point cloud data, multiple sub-point cloud data is based on the coordinate of predetermined point cloud test point and is segmented to point cloud data;Obtain the initial feature of first sub-point cloud data and the initial feature of multiple second sub-point cloud data, multiple second sub-point cloud data is the sub-point cloud data except first sub-point cloud data in multiple sub-point cloud data;Based on the initial feature of first sub-point cloud data and the initial feature of multiple second sub-point cloud data, obtain the correlation weight coefficient that the initial feature of multiple second sub-point cloud data is respectively relative to the initial feature of first sub-point cloud data, and then determine the feature correction value of first sub-point cloud data.The application solves the technical problem that global feature information cannot be obtained when extracting features from point cloud data in related technologies.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a data processing method and device and computer readable storage medium. BACKGROUND

[0002] Point cloud data of a target object can be obtained by scanning the target object by a laser radar, and a three-dimensional bounding box of the target object can be obtained according to the point cloud data of the target object.

[0003] In related technologies, feature learning is directly performed based on point cloud data, or grid division and feature coding are first performed on the point cloud data, and then 2D (Dimensionality) or 3D convolutional neural networks are used to extract features of the grid-divided point cloud data. Although these methods can obtain point cloud data features, these methods have the following defects: when extracting point cloud data features, the learned features are fixed in position, and the obtained point cloud data features lack global information. That is, in related technologies, when extracting features from point cloud data, there is a problem that global feature information cannot be obtained.

[0004] In view of the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] Embodiments of the present application provide a data processing method, device and computer readable storage medium to at least solve the technical problem that global feature information cannot be obtained when extracting features from point cloud data in related technologies.

[0006] According to an aspect of an embodiment of the present application, a data processing method is provided, comprising: obtaining point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, the plurality of sub-point cloud data being obtained by dividing the point cloud data based on coordinates of predetermined point cloud test points; obtaining an initial feature of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, the first sub-point cloud data being any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data being sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, obtaining correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data respectively; and based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients, determining a feature correction value of the first sub-point cloud data.

[0007] Optionally, the obtaining the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data comprises: performing feature extraction on the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; and performing linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

[0008] Optionally, the method further comprises: correcting the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data by using the feature correction value of the first sub-point cloud data to obtain a target feature of the first sub-point cloud data.

[0009] Optionally, the correcting the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data by using the feature correction value of the first sub-point cloud data to obtain a target feature of the first sub-point cloud data comprises: performing convolution and normalization processing on the feature correction value of the first sub-point cloud data to obtain a target feature correction value; and correcting the original feature of the first sub-point cloud data by using the target feature correction value to obtain the target feature of the first sub-point cloud data.

[0010] Optionally, the obtaining the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data comprises: respectively calculating similarities between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data; and performing normalization processing on the similarities between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data to obtain the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data.

[0011] Optionally, the determining the feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients comprises: obtaining relevant features of the plurality of second sub-point cloud data with respect to the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients; and obtaining the feature correction value of the first sub-point cloud data based on the relevant features of the plurality of second sub-point cloud data with respect to the first sub-point cloud data.

[0012] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a first obtaining module, configured to obtain point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, the plurality of sub-point cloud data being obtained by dividing the point cloud data based on coordinates of predetermined point cloud test points; a second obtaining module, configured to obtain initial features of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, the first sub-point cloud data being any one of the plurality of sub-point cloud data, the plurality of second sub-point cloud data being sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; a third obtaining module, configured to obtain correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial features of the first sub-point cloud data based on the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data; and a determining module, configured to determine a feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients.

[0013] Optionally, the second obtaining module comprises: a first obtaining sub-module, configured to perform feature extraction on the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; and a second obtaining sub-module, configured to perform linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

[0014] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, the storage medium comprising a stored program, wherein the program, when executed, controls a device where the storage medium is located to perform the data processing method described in any one of the preceding embodiments.

[0015] According to another aspect of the embodiments of the present application, a processor is also provided, the processor being configured to execute a program, wherein the program, when executed, performs the data processing method described in any one of the preceding embodiments.

[0016] In the embodiment of the present application, the point cloud data of the target object is acquired, wherein the point cloud data comprises a plurality of sub-point cloud data, the plurality of sub-point cloud data is obtained by segmenting the point cloud data based on the coordinates of the predetermined point cloud test point; the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data are acquired, the first sub-point cloud data is any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data is the sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; the correlation weight coefficient of the initial feature of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data is acquired based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data; and the feature correction value of the first sub-point cloud data is determined based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients. The feature correction value of the first sub-point cloud data is obtained according to the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data other than the first sub-point cloud data, that is, the feature correction value of the first sub-point cloud data is obtained by taking into account the global feature, so the feature correction value of the first sub-point cloud data represents the correlation information between the feature of the first sub-point cloud data and the global feature. Therefore, the embodiment of the present application solves the technical problem that the global feature information cannot be acquired when the feature of the point cloud data is extracted in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the present application. In the drawings:

[0018] Figure 1 is a flowchart of an optional data processing method according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of another optional data processing method according to an embodiment of the present application;

[0020] Figure 3 is a structural block diagram of an optional data processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the protection scope of the present application.

[0022] It should be noted that the terms "first", "second", and the like, in the description and in the claims of the present application as well as in the above-described drawings mean for distinguishing between like objects and do not necessarily indicate a specific order or sequence. Unless otherwise indicated herein, like terms and phrases include their gramma tical conjugations as well as plural too. Accordingly, features specified in one example embodiment are applicable to other example embodiments as well unless otherwise specified. It should be understood that the use of terminology such as "example", "exemplary", etc. are used to appropriately describe certain embodiments, but is not intended to imply that other embodiments are less exemplary, less capable, less useful, etc. Moreover, the term "comprising" and variations thereof, as used in the description and in the claims of the present application, are intended to cover the process, method, system, product, or apparatus including a series of steps or units without necessarily excluding additional steps or units. The term "consisting essentially of, as used in the description and in the claims of the present application, is intended to cover the process, method, system, product, or apparatus including a series of steps or units without necessarily excluding additional steps or units.

[0023] Embodiment 1

[0024] According to an embodiment of the present application, a method embodiment of data processing is provided, it should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0025] It should be noted that the terms "first", "second", and the like, in the description and in the claims of the present application as well as in the above-described drawings mean for distinguishing between like objects and do not necessarily indicate a specific order or sequence. Unless otherwise indicated herein, like terms and phrases include their gramma tical conjugations as well as plural too. Accordingly, features specified in one example embodiment are applicable to other example embodiments as well unless otherwise specified. It should be understood that the use of terminology such as "example", "exemplary", etc. are used to appropriately describe certain embodiments, but is not intended to imply that other embodiments are less exemplary, less capable, less useful, etc. Moreover, the term "comprising" and variations thereof, as used in the description and in the claims of the present application, are intended to cover the process, method, system, product, or apparatus including a series of steps or units without necessarily excluding additional steps or units. The term "consisting essentially of, as used in the description and in the claims of the present application, is intended to cover the process, method, system, product, or apparatus including a series of steps or units without necessarily excluding additional steps or units.

[0026] Reference will now be made to the example embodiments according to the present application, which will be described in greater detail below. These example embodiments may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein. It should be understood that these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0027] Terminology

[0028] Point cloud data, a data set of a plurality of sampling points in a test object, data of each sampling point in the data set comprising spatial coordinates. There are various methods for obtaining point cloud data, for example, a plurality of sampling points on a test object can be sampled by a laser radar, and then the point cloud data of the target object is obtained.

[0029] Receptive field, indicating the size of the region on the input image that each pixel on the feature map output by each layer of the convolutional neural network maps back to when processing an image.

[0030] BN (Batch Normalization), a normalization algorithm.

[0031] GN (Group Normalization), a normalization algorithm.

[0032] Figure 1 The data processing method according to an embodiment of the present application is shown in Figure 1 The method comprises the following steps:

[0033] Step S102, obtaining point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, and the plurality of sub-point cloud data is obtained by dividing the point cloud data based on the coordinates of predetermined point cloud test points.

[0034] In an optional embodiment, the point cloud data comprises three-dimensional spatial coordinates of a plurality of sampling points and reflectivity of each sampling point. There are various methods for dividing the point cloud data to obtain a plurality of sub-point cloud data.

[0035] Step S104, obtaining initial features of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, the first sub-point cloud data being any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data being sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data.

[0036] Step S106, based on the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, obtaining correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial features of the first sub-point cloud data.

[0037] Step S108, determining a feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients.

[0038] By the above steps, by acquiring the point cloud data of the target object, wherein the point cloud data includes a plurality of sub-point cloud data, the plurality of sub-point cloud data is obtained by dividing the point cloud data based on the coordinates of the predetermined point cloud test points; the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data are acquired, the first sub-point cloud data is any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data is the sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data are acquired; and based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients, the feature correction value of the first sub-point cloud data is determined. The feature correction value of the first sub-point cloud data is obtained according to the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data other than the first sub-point cloud data, that is, the feature correction value of the first sub-point cloud data is obtained by taking into account the global feature, so the feature correction value of the first sub-point cloud data represents the correlation information between the feature of the first sub-point cloud data and the global feature. Therefore, the embodiment of the present application solves the technical problem that the global feature information cannot be acquired when the feature of the point cloud data is extracted in the related art.

[0039] In some optional embodiments, the method for acquiring the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data can include the following steps: performing feature extraction on the plurality of sub-point cloud data to acquire the original features of the plurality of sub-point cloud data; and performing linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

[0040] In the optional embodiment, compared with the method of directly extracting features based on the data of the plurality of sampling points in the point cloud data, the data processing efficiency is improved by performing feature extraction on the plurality of sub-point cloud data. The initial feature is acquired based on the extracted original feature, which ensures that the feature information is not lost in the calculation process and improves the robustness of data processing.

[0041] In some optional embodiments, after determining the feature correction value of the first sub-point cloud data, the method further includes: correcting the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data by using the feature correction value of the first sub-point cloud data to obtain the target feature of the first sub-point cloud data.

[0042] In the optional embodiment, the feature correction value of the first sub-point cloud data represents the correlation information between the feature of the first sub-point cloud data and the global feature, the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data is corrected by the feature correction value of the first sub-point cloud data, and the target feature of the first sub-point cloud data is obtained, which is equivalent to incorporating the perception of global information in the feature extraction process of the first point cloud data, realizing the enhancement of the point cloud data feature, and solving the technical problem that the global feature information cannot be obtained in the related art when the point cloud data is extracted.

[0043] In some optional embodiments, the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data is corrected by the feature correction value of the first sub-point cloud data to obtain the target feature of the first sub-point cloud data, including: performing convolution and normalization processing on the feature correction value of the first sub-point cloud data to obtain a target feature correction value; and correcting the original feature of the first sub-point cloud data by the target feature correction value to obtain the target feature of the first sub-point cloud data. The target feature correction value is obtained by performing convolution and normalization processing on the feature correction value of the first sub-point cloud data, and the target feature of the first sub-point cloud data is obtained based on the target feature correction value, which is simple and fast in processing.

[0044] In some optional embodiments, based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data are obtained, including: calculating the similarity between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data respectively; and performing normalization processing on the similarity between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data to obtain the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data respectively. In an embodiment, the similarity between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data is obtained by respectively performing point multiplication calculation on the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data. The correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data are obtained based on the similarity between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data, which is simple and efficient.

[0045] In some optional embodiments, determining the feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients respectively comprises: obtaining the relevant features of the plurality of second sub-point cloud data with respect to the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients respectively; and obtaining the feature correction value of the first sub-point cloud data based on the relevant features of the plurality of second sub-point cloud data with respect to the first sub-point cloud data. In one embodiment, the initial feature of each second sub-point cloud data is multiplied by the corresponding correlation weight coefficient to obtain the relevant feature corresponding to each second sub-point cloud respectively, and the feature correction value of the first sub-point cloud data is obtained by summing each relevant feature.

[0046] It should be noted that in the foregoing optional embodiments, the first sub-point cloud data and the plurality of second sub-point cloud data collectively constitute the global point cloud data.

[0047] Based on the above embodiments and optional embodiments, a data processing method is provided.

[0048] The following will take the application field of the present optional embodiment, i.e., the automatic driving field, as an example, and take collecting point cloud data of a target object by a laser radar as an example for illustration. It should be noted that the present optional embodiment is applicable to all fields requiring feature extraction of point cloud data, and is not limited to the automatic driving field. There are various methods for collecting point cloud data, and the method of collecting point cloud data is not limited to laser radar collection.

[0049] The laser radar is widely used in the automatic driving field. By scanning a target object through a laser radar, three-dimensional space coordinate information of a plurality of sampling points on the target object and reflectivity information of the sampling points are obtained. The target detection task of the laser radar is to output a three-dimensional bounding box of a target object according to input laser point cloud data, and the three-dimensional bounding box contains center point coordinates, bounding box size, bounding box orientation angle and other information. Among them, the feature description and extraction of point cloud data can realize the target detection algorithm based on point cloud data, and the description and extraction of point cloud features of point cloud data have an important influence on point cloud data processing.

[0050] In the related art, feature learning can be performed directly based on point cloud data, or grid division and feature coding are performed on the point cloud data first, and then 2D (Dimensionality) or 3D convolutional neural network is used to extract the features after grid division. Although these methods can achieve feature extraction of point cloud data, because the extracted point cloud data features lack global information, after obtaining the features of the point cloud data, the global information needs to be aggregated by cross-layer application of a feature extractor. This method has the following problems: as the size of the receptive field increases, the amount of parameters to be processed increases, which leads to poor scalability of application; the learned features are fixed position features, so the obtained features lack global information; when using a cross-layer application feature extractor to aggregate global information, multiple layers of parameter optimization need to be coordinated for pattern recognition, which leads to high method complexity, processing difficulty, and poor performance of the obtained features. Therefore, when extracting features from the aforementioned point cloud data, global feature information of the point cloud data needs to be obtained to solve the aforementioned problems.

[0051] In addition, in the field of autonomous driving, point cloud data is usually collected by vehicle-mounted sensors, and based on this method, the following problems exist in obtaining point cloud data: the data of some sampling points is easily lost, and there is noise in the obtained point cloud data; the data amount of near objects and distant objects is unbalanced, specifically, there are more sampling points for near objects and fewer sampling points for distant objects. These problems will affect the point cloud data, and further affect the usability of point cloud data feature extraction. In order to ensure the performance of point cloud data feature extraction, global point cloud correlation needs to be extracted from the point cloud data, that is, when extracting features from the aforementioned point cloud data, global feature information of the point cloud data needs to be obtained, thereby improving the performance of point cloud feature extraction.

[0052] However, in the related art, there is a technical problem that global feature information cannot be obtained.

[0053] Therefore, in the embodiments of the present disclosure, a data processing method is provided, which obtains a plurality of sub-point cloud data by segmenting the point cloud data, and obtains global feature information of the point cloud data by calculating the correlation information between the features of each sub-point cloud data and the features of other sub-point cloud data. The technical problem that global feature information cannot be obtained when extracting features from the point cloud data in the related art is solved. After obtaining the global feature information of the point cloud data, the global information of the original features of the point cloud data is enhanced by superimposing the obtained global feature information on the original features of the point cloud data, and more robust feature extraction of point cloud data information is achieved.

[0054] Figure 2 is a flowchart of another optional data processing method according to an embodiment of the present disclosure. Referring toFigure 2 As shown, the method includes the following steps:

[0055] Step S202: Obtain the original features of the point cloud data nodes.

[0056] Point cloud data acquired by LiDAR includes four-dimensional features (x, y, z, j) of multiple sampling points, where x, y, and z represent the coordinates of the sampling point on the X, Y, and Z axes, respectively, and j represents the reflectivity of the sampling point. In 3D point cloud target detection, directly performing 3D convolution results in high computational cost and slow processing speed. To improve processing speed, positional encoding is performed on the point cloud data, specifically including the following steps: The point cloud is discretized into multiple evenly spaced grids on the XY plane according to a preset length along the X and Y axes. Points in the point cloud are assigned to different grids based on their coordinates. Grids with a non-zero number of internal sampling points are considered valid grids, and each valid grid is used as a node position in the global scene of the point cloud data. It should be understood that the point cloud data obtained by dividing the data according to coordinates is equivalent to the sub-point cloud data in the aforementioned embodiment. That is, the process of positional encoding of the point cloud data is equivalent to the process of cutting the point cloud data to obtain multiple sub-point cloud data in the aforementioned embodiment.

[0057] Feature expansion and extraction are performed on the point cloud data of all valid node locations globally to obtain the original feature S corresponding to each node location, S = {S1, S2, ..., S...} i …S n}, where S1, S2, S i S n These represent the original features of the 1st node position, the 2nd node position, the i-th node position, and the nth node position, respectively.

[0058] Step S204: Based on the original features of the point cloud data nodes, obtain the correlation weight coefficients between the node position features.

[0059] For all node positions, the feature S = {S1, S2, ..., S...} n First, a linear convolution is performed to obtain the features after linear convolution processing. These features are equivalent to the initial features in the previous embodiment. The obtained initial features are labeled as Q = {q1, q2, ..., q...} i …q n}, where q1, q2, q i q n These represent the initial features of the 1st node position, the 2nd node position, the i-th node position, and the nth node position, respectively.

[0060] For the i-th node position, i = 1, 2, ..., n, perform the following operations to obtain the correlation weight coefficients between the initial features of the i-th node position and the features of other node positions:

[0061] Perform a dot product calculation on the initial features at the i-th node position (equivalent to the initial features of the first sub-point cloud data in the aforementioned embodiment) and the initial features at the j-th node position (equivalent to the initial features of the second sub-point cloud data in the aforementioned embodiment) to obtain the dot product result of the initial features at the i-th node position and the initial features at the j-th node position (equivalent to the similarity between the initial features of the first sub-point cloud data and the initial features of the second sub-point cloud data in the aforementioned embodiment); where j = 1, 2…n, and j ≠ i.

[0062] The softmax function is used to normalize the dot product of the initial features at the i-th node position and the initial features at each j-th node position, thus obtaining the weight coefficient vector W corresponding to the i-th node position. i ={w i1 ,w i2 …w ij …w in}, weight coefficient vector W i This is equivalent to the correlation weight coefficients of the initial features of multiple second sub-point cloud data relative to the initial features of the first sub-point cloud data. Where w i1 w i2 w ij w in These represent the correlation weight coefficients between the initial feature of the i-th node position and the features of the 1st, 2nd, j-th, and nth node positions, respectively.

[0063] In the aforementioned method, the dot product of the initial features at the i-th node position and the initial features at all other node positions is normalized using the Softmax function. This not only obtains the normalized result but also highlights the weights of important elements, thus making the features more prominent.

[0064] Step S206: Obtain relevant information between node location features and global features.

[0065] For the weight coefficient vector W corresponding to the i-th node position i ={w i1 ,w i2 …w ij …w in}, respectively using w i1 Multiply by the initial feature at the first node position, and use w i2 Multiply by the initial feature at the second node position, and use w ijand the initial feature of the jth node position, and w in and the initial feature of the nth node position, to obtain the correlation feature R i between the initial feature of the ith node position and the initial feature of the jth node position. i1 i2 ij in , wherein r i1 , r i2 , r ij , and r in represent the correlation feature between the initial feature of the ith node position and the initial feature of the 1st node position, the correlation feature between the initial feature of the ith node position and the initial feature of the 2nd node position, the correlation feature between the initial feature of the ith node position and the initial feature of the jth node position, and the correlation feature between the initial feature of the ith node position and the initial feature of the nth node position, respectively.

[0066] Summing the correlation features between the initial feature of the ith node position and the initial features of all the other node positions, the correlation information a i between the initial feature of the ith node position and the global feature is obtained, wherein:

[0067]

[0068] , wherein r ij represents the correlation feature between the initial feature of the ith node position and the initial feature of the jth node position.

[0069] The correlation information a i between the initial feature of the ith node position and the global feature is equivalent to the correlation feature based on the plurality of second sub-point cloud data with respect to the first sub-point cloud data in the foregoing embodiment, to obtain the feature correction value of the first sub-point cloud data.

[0070] In step S208, the correlation information between the node position feature and the global feature and the original feature of the node are aggregated.

[0071] For the ith node position, the correlation information a i is obtained between the initial feature of the ith node position and the global feature. i The advanced linear convolution is performed, and the convolution result is normalized by using Group Normalization to obtain the normalized correlation information a′ i , which is equivalent to the target feature correction value of the first sub-point cloud data in the foregoing embodiment.

[0072] The a′ i is multiplied by the original feature S i of the ith node position.​​Superposition is performed to obtain the aggregation result feat of the i-th node position i wherein

[0073] feat i = a' i + s i

[0074] wherein the aggregation result feat of the i-th node position i represents the feature extraction result of the i-th node position based on global information perception, which is equivalent to the target feature of the first sub-point cloud data in the foregoing embodiment.

[0075] The convolution result is normalized by using Group Normalization, which reduces the influence of batch size on the processing result compared with the traditional Batch Normalization processing method, for example, the problem of high calculation error rate of BN when the batch size is small can be solved. Thus, the accuracy of the processing result is improved.

[0076] According to the optional embodiment, the position encoding and original feature extraction of the point cloud data are obtained, the correlation weight coefficient between the node position features is obtained based on the original features of the point cloud data nodes, the related information between the node position features and the global features is obtained, the related information between the node position features and the global features and the original features of the nodes are aggregated, and the features containing global information perception of the node positions are obtained. In actual application, the features containing global information perception are input into the corresponding machine model, and high-precision three-dimensional target monitoring can be realized.

[0077] In the optional real-time manner, the perception of global information is integrated into the feature extraction process of the point cloud data, and the enhancement of the point cloud data features is realized; the position encoding of the point cloud data is performed to improve the processing efficiency; the Group Normalization is used instead of the traditional Batch Normalization to normalize the data, and the influence of excessive dependence of the processing process on the batch size is reduced, and the accuracy of the processing result is improved. In addition, in the optional real-time manner, the original features of the point cloud data are not first reduced in dimension and then calculated, and the calculation of the original features of the point cloud data is performed, so that the feature information is not lost in the calculation process, and the point cloud data features based on global information perception with high robustness are extracted.

[0078] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0079] Embodiment 2

[0080] According to the embodiments of the present application, a device for implementing the above data processing method is also provided, Figure 3 is a structural block diagram of the data processing device according to Embodiment 1 of the present application. As shown in the figure, Figure 3 the data processing device includes a first acquisition module 302, a second acquisition module 304, a third acquisition module 306 and a determination module 308. The following will be described in detail.

[0081] The first acquisition module 302 is configured to acquire point cloud data of a target object, wherein the point cloud data includes a plurality of sub-point cloud data, and the plurality of sub-point cloud data is obtained by segmenting the point cloud data based on coordinates of predetermined point cloud test points. The second acquisition module 304 is connected to the first acquisition module 302 and is configured to acquire initial features of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, wherein the first sub-point cloud data is any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data is the sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data. The third acquisition module 306 is connected to the second acquisition module 304 and is configured to acquire correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data based on the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data. The determination module 308 is connected to the third acquisition module 306 and is configured to determine a feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients.

[0082] It should be noted that the first acquisition module 302, the second acquisition module 304, the third acquisition module 306 and the determination module 308 correspond to steps S202 to S208 in Embodiment 1, respectively, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1.

[0083] As an optional real-time example, the second acquisition module 304 comprises: a first acquisition submodule, configured to perform feature extraction on the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; and a second acquisition submodule, configured to perform linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

[0084] Embodiment 3

[0085] The embodiments of the present application can provide a computer readable storage medium. Optionally, in the present embodiment, the computer readable storage medium can be used to save the program code executed by the data processing method provided in Embodiment 1.

[0086] Optionally, in the present embodiment, the computer readable storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0087] Optionally, in the present embodiment, the computer readable storage medium is configured to store program code for performing the following steps: acquiring point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, and the plurality of sub-point cloud data is obtained by dividing the point cloud data based on coordinates of predetermined point cloud test points; acquiring an initial feature of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, the first sub-point cloud data being any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data being sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, acquiring correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data; and based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients, determining a feature correction value of the first sub-point cloud data.

[0088] Optionally, in the present embodiment, the computer readable storage medium is configured to store program code for performing the following steps: acquiring an initial feature of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, comprising: performing feature extraction on the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; and performing linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

[0089] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: modifying the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data by using the feature modification value of the first sub-point cloud data to obtain the target feature of the first sub-point cloud data.

[0090] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: modifying the original feature of the first sub-point cloud data in the original features of the plurality of sub-point cloud data by using the feature modification value of the first sub-point cloud data to obtain the target feature of the first sub-point cloud data, including: performing convolution and normalization processing on the feature modification value of the first sub-point cloud data to obtain a target feature modification value; and modifying the original feature of the first sub-point cloud data by using the target feature modification value to obtain the target feature of the first sub-point cloud data.

[0091] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, obtaining correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data, including: respectively calculating similarities between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data; and performing normalization processing on the similarities between the initial features of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data to obtain the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data.

[0092] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: based on the initial features of the plurality of second sub-point cloud data and the respective corresponding correlation weight coefficients, determining the feature modification value of the first sub-point cloud data, including: based on the initial features of the plurality of second sub-point cloud data and the respective corresponding correlation weight coefficients, obtaining relevant features of the plurality of second sub-point cloud data with respect to the first sub-point cloud data; and based on the relevant features of the plurality of second sub-point cloud data with respect to the first sub-point cloud data, obtaining the feature modification value of the first sub-point cloud data.

[0093] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing the related hardware of the terminal device through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0094] The embodiment of the present application also provides a computer program product comprising a computer program. The computer program is executed by a processor to implement the following method: obtaining point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, the plurality of sub-point cloud data being obtained by dividing the point cloud data based on coordinates of predetermined point cloud test points; obtaining an initial feature of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, the first sub-point cloud data being any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data being sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; obtaining correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial feature of the first sub-point cloud data based on the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data; and determining a feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the respective correlation weight coefficients.

[0095] Optionally, the program is executed to perform the following steps: obtaining the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data comprises: performing feature extraction on the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; and performing linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial feature of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

[0096] Optionally, the program is executed to perform the following steps: using the feature correction value of the first sub-point cloud data to correct original features of the first sub-point cloud data in the original features of the plurality of sub-point cloud data to obtain target features of the first sub-point cloud data.

[0097] Optionally, the program is executed to perform the following steps: using the feature correction value of the first sub-point cloud data to correct original features of the first sub-point cloud data in the original features of the plurality of sub-point cloud data to obtain target features of the first sub-point cloud data, comprising: performing convolution and normalization processing on the feature correction value of the first sub-point cloud data to obtain a target feature correction value; and using the target feature correction value to correct the original features of the first sub-point cloud data to obtain the target features of the first sub-point cloud data.

[0098] Optionally, the program runs to perform the following steps: based on the initial feature of the first sub-point cloud data and the initial feature of the plurality of second sub-point cloud data, obtaining the correlation weight coefficient of the initial feature of the plurality of second sub-point cloud data relative to the initial feature of the first sub-point cloud data respectively, comprising: calculating the similarity between the initial feature of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data respectively; normalizing the similarity between the initial feature of the plurality of second sub-point cloud data and the initial feature of the first sub-point cloud data to obtain the correlation weight coefficient of the initial feature of the plurality of second sub-point cloud data relative to the initial feature of the first sub-point cloud data respectively.

[0099] Optionally, the program runs to perform the following steps: based on the initial feature of the plurality of second sub-point cloud data and the corresponding correlation weight coefficient respectively, determining the feature correction value of the first sub-point cloud data, comprising: based on the initial feature of the plurality of second sub-point cloud data and the corresponding correlation weight coefficient respectively, obtaining the relevant feature of the plurality of second sub-point cloud data relative to the first sub-point cloud data respectively; based on the relevant feature of the plurality of second sub-point cloud data relative to the first sub-point cloud data respectively, obtaining the feature correction value of the first sub-point cloud data.

[0100] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0101] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0102] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on multiple units. Part or all of the units can be selected to achieve the purpose of the present embodiment scheme according to actual needs.

[0103] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or software functional unit.

[0104] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0105] The above is only the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method comprises the following steps: obtaining point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, and the plurality of sub-point cloud data is obtained by dividing the point cloud data based on coordinates of predetermined point cloud test points; obtaining initial features of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, wherein the first sub-point cloud data is any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data is the sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; based on the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, obtaining correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial features of the first sub-point cloud data respectively; based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients respectively, determining a feature correction value of the first sub-point cloud data, wherein the feature correction value of the first sub-point cloud data is used to represent the correlation information between the initial features of the first sub-point cloud data and global features of the point cloud data; wherein, based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients respectively, determining the feature correction value of the first sub-point cloud data comprises: multiplying the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients respectively to obtain a plurality of multiplication results; and summing the plurality of multiplication results to obtain the feature correction value of the first sub-point cloud data; based on the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data, obtaining correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial features of the first sub-point cloud data respectively, comprises: calculating the similarity between the initial features of the plurality of second sub-point cloud data and the initial features of the first sub-point cloud data respectively; and performing normalization processing on the similarity between the initial features of the plurality of second sub-point cloud data and the initial features of the first sub-point cloud data to obtain the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data with respect to the initial features of the first sub-point cloud data respectively.

2. The method of claim 1, wherein, The method comprises the following steps: performing feature extraction on the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; performing linear convolution processing on the original features of the plurality of sub-point cloud data to obtain the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data.

3. The method of claim 2, wherein, The method further comprises the following steps: using the feature correction value of the first sub-point cloud data to correct the original features of the first sub-point cloud data in the original features of the plurality of sub-point cloud data to obtain target features of the first sub-point cloud data.

4. The method of claim 3, wherein, The method further comprises the following steps: The feature correction value of the first sub-point cloud data is subjected to convolution and normalization processing to obtain a target feature correction value; The original feature of the first sub-point cloud data is corrected by using the target feature correction value to obtain the target feature of the first sub-point cloud data.

5. The method of claim 1, wherein, The feature correction value of the first sub-point cloud data is determined based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients, comprising: Based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients, the correlation features of the plurality of second sub-point cloud data relative to the first sub-point cloud data are obtained; Based on the correlation features of the plurality of second sub-point cloud data relative to the first sub-point cloud data, the feature correction value of the first sub-point cloud data is obtained.

6. A data processing apparatus, characterized by Comprising: The first acquisition module is used for acquiring point cloud data of a target object, wherein the point cloud data comprises a plurality of sub-point cloud data, and the plurality of sub-point cloud data is obtained by segmenting the point cloud data based on coordinates of predetermined point cloud test points; The second acquisition module is used for acquiring initial features of a first sub-point cloud data and initial features of a plurality of second sub-point cloud data, wherein the first sub-point cloud data is any one of the plurality of sub-point cloud data, and the plurality of second sub-point cloud data is the sub-point cloud data other than the first sub-point cloud data in the plurality of sub-point cloud data; The third acquisition module is used for acquiring correlation weight coefficients of initial features of the plurality of second sub-point cloud data relative to initial features of the first sub-point cloud data based on the initial features of the first sub-point cloud data and the initial features of the plurality of second sub-point cloud data; The determination module is used for determining a feature correction value of the first sub-point cloud data based on the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients, wherein the feature correction value of the first sub-point cloud data is used to represent the correlation information between the initial features of the first sub-point cloud data and the global features of the point cloud data; The determination module is further used to perform the following steps: multiplying the initial features of the plurality of second sub-point cloud data and the corresponding correlation weight coefficients respectively to obtain a plurality of multiplication results; and summing the plurality of multiplication results to obtain the feature correction value of the first sub-point cloud data; The third acquisition module is further used to perform the following steps: calculating the similarity between the initial features of the plurality of second sub-point cloud data and the initial features of the first sub-point cloud data respectively; and performing normalization processing on the similarity between the initial features of the plurality of second sub-point cloud data and the initial features of the first sub-point cloud data to obtain the correlation weight coefficients of the initial features of the plurality of second sub-point cloud data relative to the initial features of the first sub-point cloud data.

7. The apparatus of claim 6, wherein, The second acquisition module comprises: The first acquisition submodule is used for extracting features of the plurality of sub-point cloud data to obtain original features of the plurality of sub-point cloud data; A second acquisition submodule is configured to perform linear convolution processing on original features of the plurality of sub-point cloud data to obtain initial features of the first sub-point cloud data and initial features of the plurality of second sub-point cloud data.

8. A computer-readable storage medium, characterized in that, The storage medium comprises a stored program, wherein the program, when executed, controls a device in which the storage medium is located to perform the data processing method of any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the data processing method of any one of claims 1 to 5.

Images