Data processing method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the present application provide a data processing method and device, electronic equipment and readable storage medium, and relate to the technical field of computer. The method comprises: taking each point in point cloud data of a target object as a target point, and taking any point in the point cloud data except the target point as a first point; obtaining a first tensor and a second tensor corresponding to the target point, each first element in the first tensor being a feature of a corresponding first point, and each second element in the second tensor being a feature difference between the target point and the corresponding first point; for each second element, according to a size relationship between the second element and a first threshold, a target value combination of the corresponding first point, the target value combination being used to represent whether the first element corresponding to the first point needs to be updated; and for each first element, according to the target value combination of the first point corresponding to the first element and the feature of the target point, the first element is updated. The embodiments of the present application improve the efficiency of data processing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data processing method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] In 3D spatial networks such as Bird's Eye View (BEV) and Pointnet, index operators such as Scatter and Gather are often used to process point cloud data.

[0003] However, the inherent hardware limitations of the Neural Processing Unit (NPU) mean that when faced with indexing operations involving a large number of values, it cannot call up enough threads to speed up the process like the Central Processing Unit (CPU), thus resulting in low data processing efficiency. Summary of the Invention

[0004] This application provides a data processing method, apparatus, electronic device, and readable storage medium to solve the technical problem of low data processing efficiency.

[0005] According to a first aspect of the embodiments of this application, a data processing method is provided, the method comprising: taking each point in the point cloud data of a target object as a target point, and taking any point in the point cloud data other than the target point as a first point; Obtain the first tensor and the second tensor corresponding to the target point. Each first element in the first tensor is a feature of the corresponding first point, and each second element in the second tensor is the feature difference between the target point and the corresponding first point. For each second element, the target value combination of the first point corresponding to the second element is determined according to the size relationship between the second element and the first threshold. The target value combination is used to characterize whether the first element corresponding to the first point needs to be updated. For each first element, update the first element based on the target value combination of the first point corresponding to the first element and the characteristics of the target point to obtain the updated first tensor.

[0006] In one possible implementation, the target value combination includes a first target value and a second target value, wherein the second target value is 1 when the first target value is 0, and the second target value is 0 when the first target value is 1; For each first element, obtain the first product between the first element and the first target value, and obtain the second product between the feature of the target point and the second target value; For each first element, add the first product to the second product to obtain the updated first element.

[0007] In another possible implementation, for each second element, the first difference between the second element and the first threshold is obtained, and the first difference is inverted to obtain the second difference. For any target difference between the first difference and the second difference, determine whether the target difference is a positive number; If the target difference is determined to be positive, then the target difference is processed by at least one consecutive operation operator to obtain the operation result corresponding to the target difference. If the target difference is determined to be non-positive, then the target difference is set to zero to obtain the operation result corresponding to the target difference. If the target difference is the first target difference, the calculation result corresponding to the target difference is taken as the second target value; If the target difference is the second difference, the calculation result corresponding to the target difference is taken as the first target value.

[0008] In yet another possible implementation, when the second element is less than the first threshold, the first target value is 1 and the second target value is 0. When the second element is not less than the first threshold, the first target value is 0 and the second target value is 1.

[0009] In yet another possible implementation, the target difference is input into the first operator, which then rounds the target difference up to obtain the third value. The third value is input into the second operator, which converts the third value into an integer type to obtain the fourth value; The fourth value is input into the third operator, which restricts the fourth value to a preset first range to obtain the operation result corresponding to the target difference; the first range is 0 to 1.

[0010] In yet another possible implementation, the features of any point in the point cloud data are used to represent at least one of the following: Coordinate information; Color information; Reflection intensity information; Semantic information.

[0011] In another possible implementation, for any target difference between the first and second differences, a ReLU operation is used to determine whether the target difference is positive.

[0012] In yet another possible implementation, the aforementioned implementation is executed by the NPU.

[0013] According to a second aspect of the embodiments of this application, a data processing apparatus is provided, the apparatus comprising: The processing module is used to take each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point. The acquisition module is used to acquire the first tensor and the second tensor corresponding to the target point. Each first element in the first tensor is a feature of a corresponding first point, and each second element in the second tensor is the feature difference between the target point and the corresponding first point. The determination module is used to determine the target value combination of the first point corresponding to the second element for each second element, based on the size relationship between the second element and the first threshold. The target value combination is used to characterize whether the first element corresponding to the first point needs to be updated. The update module is used to update the first element for each first element based on the combination of the target values ​​of the first point corresponding to the first element and the characteristics of the target point, so as to obtain the updated first tensor.

[0014] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, wherein the processor executes the program to implement the steps of the method provided in the first aspect.

[0015] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method provided in the first aspect.

[0016] According to a fifth aspect of the present application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps implementing the method provided in the first aspect.

[0017] The beneficial effects of the technical solutions provided in this application are: By taking each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point, the first tensor and the second tensor corresponding to the target point are obtained. Since each first element in the first tensor is the feature of the corresponding first point, and each second element in the second tensor is the feature difference between the target point and the corresponding first point, the features of each first point are summarized through the first tensor, and the feature differences between the target point and each first point are summarized through the second tensor. This allows for batch processing of each element in the first tensor and the second tensor in subsequent data processing.

[0018] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the feature of the first point corresponding to each second element in the second tensor needs to be updated.

[0019] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0021] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application; Figure 2 A flowchart illustrating a method for updating the first element in a data processing method provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for determining a combination of target values ​​in a data processing method provided in this application embodiment; Figure 4 A flowchart illustrating a method for obtaining computation results in a data processing method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0023] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] The following explains the relevant technologies: When the amount of data to be processed is large, the use of index operations poses a huge challenge to the NPU DSA architecture. The dedicated computing architecture of NPU DSA can only handle index operations with extremely low efficiency, resulting in very low data processing efficiency.

[0026] To address at least one of the aforementioned technical problems or areas requiring improvement in related technologies, this application proposes a data processing method. This method uses each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point. It then obtains a first tensor and a second tensor corresponding to the target point. Since each first element in the first tensor represents a feature of the corresponding first point, and each second element in the second tensor represents a feature difference between the target point and the corresponding first point, the method achieves the summarization of the features of each first point through the first tensor and the summarization of the feature differences between the target point and each first point through the second tensor. This allows for batch processing of each element in the first and second tensors during subsequent data processing.

[0027] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the first point corresponding to each second element in the second tensor needs to be updated.

[0028] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing.

[0029] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0030] This application provides a data processing method, such as... Figure 1 As shown, the method includes: S101, take each point in the point cloud data of the target object as the target point, and take any point in the point cloud data other than the target point as the first point.

[0031] In this embodiment, point cloud data is obtained by sampling target objects in the physical world. In autonomous driving scenarios and general 3D scenarios, target objects can be ground, vegetation, buildings, motor vehicles, pedestrians, obstacles, etc.

[0032] In this embodiment, the point currently being processed in the point cloud data is taken as the target point, and any point in the point cloud data other than the target point is taken as the first point, thus realizing the classification of data in the point cloud and facilitating subsequent batch processing.

[0033] S102, obtain the first and second tensors corresponding to the target point.

[0034] In the embodiments of this application, each first element in the first tensor is a feature of a corresponding first point, and each second element in the second tensor is the feature difference between the target point and the corresponding first point.

[0035] In this embodiment of the application, the features of each first point other than the current target point are used as a first element to construct a first tensor. That is, each element in the first tensor corresponds to a first point, and each element represents the features of a first point.

[0036] In this embodiment of the application, for each first element in the first tensor, the feature difference between the current element and the feature of the target point is calculated to obtain the second element. The corresponding second element is obtained through the feature difference between each first element and the target point.

[0037] In one example, the point cloud data includes points A, B, C, and D. The current target point is point A. Therefore, the first points are points B, C, and D. The first tensor corresponding to the target point contains three first elements, that is, the first tensor corresponding to the target point includes the features of point B, the features of point C, and the features of point D. For each first element, the feature difference between the first element and the feature of the target point is calculated, thus obtaining the first feature difference between point B and the target point, the second feature difference between point C and the target point, and the third feature difference between point D and the target point as the second element. That is, the second tensor corresponding to the target point includes the first feature difference, the second feature difference, and the third feature difference.

[0038] It should be noted that, for the convenience of subsequent data processing, the logical order of the elements corresponding to the first point in the first tensor and the second tensor corresponding to the same target point is the same. For example, in the example above, the first element in both the first tensor and the second tensor corresponds to point B.

[0039] In this embodiment of the application, the feature of any point in the point cloud data is used to represent at least one of the following: Coordinate information; Color information; Reflection intensity information; Semantic information.

[0040] In the embodiments of this application, coordinate information refers to the position of the point in three-dimensional space; color information refers to the color-related information of the point on the target object; reflection intensity information refers to the intensity of the reflected laser for lidar, which can be used to distinguish objects of different materials; semantic information is used to describe the semantic attributes and contextual relationships of the point.

[0041] In the embodiments of this application, the feature difference may be the difference in coordinate position between the target point and the first point, the difference in color between the target point and the first point, the difference in material between the target point and the first point, or the difference in semantic information between the target point and the first point.

[0042] S103, for each second element, determine the target value combination of the first point corresponding to the second element based on the size relationship between the second element and the first threshold.

[0043] In this embodiment, since the second element represents the feature difference between the target point and the corresponding first point, the first threshold can be set according to the data processing requirements. Based on the relationship between the second element and the first threshold, the feature difference can be filtered to see if it meets the data processing requirements. For example, the second element represents the coordinate difference between the target point and the corresponding first point, i.e., the distance between them. If the current data processing needs to filter out the first point within the first radius of the target point, then the first threshold can be set to the first radius. Based on the relationship between the second element and the first threshold, the first point within the first radius of the target point can be filtered out.

[0044] In this embodiment, the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated. If the first point meets the data processing requirements based on the size relationship between the second element and the first threshold, then the target value combination characterizing that the first element corresponding to the first point does not need to be updated is obtained. If the first point does not meet the data processing requirements based on the size relationship between the second element and the first threshold, then the target value combination characterizing that the first element corresponding to the first point needs to be updated is obtained.

[0045] S104. For each first element, update the first element according to the target value combination of the first point corresponding to the first element and the characteristics of the target point to obtain the updated first tensor.

[0046] In this embodiment of the application, for each first element, the target value combination of the first point corresponding to the first element is obtained. If it is determined from the target value combination that the first element in the first tensor needs to be updated, the first element is replaced with the feature of the target point. If it is determined from the target value combination that the first element in the first tensor does not need to be updated, the original value of the first element is kept in the first tensor, that is, the first element is still the feature of the corresponding first point.

[0047] In this embodiment of the application, for each first element in the first tensor, it is determined whether the first element needs to be updated based on the target value combination, and each first element that needs to be updated is updated with the feature of the target point to obtain the updated first tensor.

[0048] In one example, the neighborhood points of each point in the point cloud data are filtered. Points whose distance to the target point is less than a first radius are considered neighborhood points of the target point. After determining the target point, each point other than the target point is designated as the first point. The feature of each first point is used as a first element to form the first tensor corresponding to the target point. The distance between each first point and the target point is used as a second element to form the second tensor corresponding to the target point. By constructing a conditional mask: second element (distance) - first threshold (first radius), the size relationship between the second element and the first threshold is compared. If the second element is smaller than the first threshold, it means that the first point is a neighborhood point, and the first element corresponding to the first point in the first tensor does not need to be updated. If the second element is larger than the first threshold, it means that the point is not a neighborhood point of the target point, and the first element corresponding to the first point needs to be deleted from the first tensor. In order to maintain the regularity of the tensor shape and facilitate the batch processing of the neighborhood points of the target point in subsequent batch processing, the deleted first points are replaced with the features of the target point, thereby keeping the tensor size consistent and facilitating subsequent batch processing.

[0049] The data processing method provided in this application takes each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point. It obtains a first tensor and a second tensor corresponding to the target point. Since each first element in the first tensor is a feature of the corresponding first point and each second element in the second tensor is a feature difference between the target point and the corresponding first point, it realizes the summarization of the features of each first point through the first tensor and the summarization of the feature differences between the target point and each first point through the second tensor. This allows each element in the first tensor and the second tensor to be processed in batches during subsequent data processing.

[0050] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the first point corresponding to each second element in the second tensor needs to be updated.

[0051] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing.

[0052] Based on the above embodiments, as an optional embodiment, the target value combination includes a first target value and a second target value. When the first target value is 0, the second target value is 1, and when the first target value is 1, the second target value is 0.

[0053] In this embodiment of the application, if it is determined based on the size relationship that the first element of the corresponding first point does not need to be updated, then the first target value is 0 and the second target value is 1; if it is determined based on the size relationship that the corresponding first point element needs to be updated, then the first target value is 1 and the second target value is 0. In this application embodiment, a method for updating a first element is provided, such as... Figure 2 As shown, the specific content is as follows: S201, for each first element, obtain the first product between the first element and the first target value, and obtain the second product between the feature of the target point and the second target value; S202, for each first element, add the first product to the second product to obtain the updated first element.

[0054] In S201 of this application embodiment, for each first element, the first target value and the second target value of the first point corresponding to the first element are obtained, the first product between the first element and the first target value is obtained, and the first product between the second target value and the feature point of the target is obtained.

[0055] In S202 of this application embodiment, since the second target value is 0 when the first target value is 1, the first product is the first element, the second product is 0, and the sum of the first product and the second product is still the first element. The first element in the first tensor is not updated. This indicates that when the first target value is 1 and the second target value is 0, there is no need to update the element of the corresponding first point. Since the second target value is 1 when the first target value is 0, the first product is 0, the second product is the feature of the target point, and the sum of the first product and the second product is the feature of the target point. The first element in the first tensor is updated to the feature of the target point. This indicates that when the first target value is 0 and the second target value is 1, the element of the corresponding first point needs to be updated.

[0056] In one example, if the first objective value of point A is 0 and the second objective value is 1, then the first element of point A in the first tensor needs to be updated. If the first objective value of point B is 1 and the second objective value is 0, then the first element of point B in the first tensor does not need to be updated. Since processing each element in batches when processing the first tensor improves data processing efficiency, assuming the feature of point A is A, the feature of point B is B, and the feature of the target point is C, then the updated first element of point A is: A. 0+C 1=C, the first element of the updated second point B is: B 1+C Setting 0 to B ensures that, based on batch processing, the first element that needs to be updated is updated.

[0057] In the above scheme, by setting the target value combination to two combinations—when the first target value is 1 and the second target value is 0, and when the first target value is 0 and the second target value is 1—and obtaining the first product of the first element and the first target value of the corresponding first point, and the second product of the feature of the target point and the corresponding second target value, and using the sum of the first and second products as the updated first element, this scheme achieves the following: when the target value combination representation does not require updating the first element of the corresponding first point (i.e., when the first target value is 1 and the second target value is 0), the updated first element obtained based on the sum of the products is the original first element; when the target value combination representation requires updating the first element of the corresponding first point (i.e., when the first target value is 0 and the second target value is 1), the updated first element obtained based on the sum of the products is the feature of the target value. This scheme enables updating first elements with different update requirements through the same operation process, avoiding the reduction of data processing efficiency through logical judgment and index operations.

[0058] Based on the above embodiments, as an optional embodiment, a method for determining a combination of target values ​​is provided, such as... Figure 3 As shown, the specific content is as follows; S301, For each second element, obtain the first difference between the second element and the first threshold, and perform a numerical inversion operation on the first difference to obtain the second difference; S302, for any target difference between the first difference and the second difference, determine whether the target difference is a positive number; S303, if the target difference is determined to be positive, the target difference is processed by at least one continuous operation operator to obtain the operation result corresponding to the target difference; if the target difference is determined to be non-positive, the target difference is set to zero to obtain the operation result corresponding to the target difference. S304, if the target difference is the first difference, the calculation result corresponding to the target difference is taken as the second target value; S305, if the target difference is the second difference, the calculation result corresponding to the target difference is taken as the first target value.

[0059] In S301 of this application embodiment, for each second element, a first difference between the second element and the first threshold is obtained. The purpose of obtaining the first difference is to determine whether the feature difference between the corresponding first point and the target point meets the data processing requirements. In order to facilitate batch processing in the subsequent update process of the first element, the first difference is inverted to obtain a second difference. The purpose of obtaining the second difference is to determine whether the feature difference between the first point and the target point does not meet the data processing requirements.

[0060] In one example, the data processing requirement is to filter out neighborhood points of the target point within the first radius. To facilitate subsequent calculations, the first threshold can be set to the square of the first radius. The second element represents the square of the distance between the target point and the corresponding first point. The first difference = the square of the distance - the square of the first radius. For example, if the first difference is -4, the second difference will be 4 after inverting the first difference.

[0061] In S302 of this application embodiment, when the target difference is a first difference, it is determined whether the first difference is positive; when the target difference is a second difference, it is determined whether the second difference is positive. By determining whether the target difference is positive, it can be determined whether the first element corresponding to the first point of the second element used to calculate the target difference needs to be updated.

[0062] In S303 of this application embodiment, when the target difference is a first difference, if the first difference is determined to be positive, it indicates that the feature difference between the current corresponding first point and the target point meets the data processing requirements. Then, the first difference is processed using at least one continuous operation operator to obtain the operation result corresponding to the first difference. If the first difference is determined to be non-positive, it indicates that the feature difference between the current corresponding first point and the target point does not meet the data processing requirements. The first difference is then set to zero to obtain the operation result corresponding to the second difference. Similarly, when the target difference is a second difference, if the second difference is determined to be positive, it indicates that the feature difference between the current corresponding first point and the target point does not meet the data processing requirements. The second difference is processed using at least one continuous operation operator to obtain the operation result corresponding to the second difference. If the second difference is determined to be non-positive, it indicates that the feature difference between the current corresponding first point and the target point meets the data processing requirements. The second difference is then set to zero to obtain the operation result corresponding to the second difference.

[0063] In S304 of this application embodiment, since the target value combination includes a first target value and a second target value, and when the first target value is 1 and the second target value is 0, it means that the first element does not need to be updated at present, and when the first target value is 0 and the second target value is 1, it means that the first element needs to be updated at present. Therefore, when the target difference is the first difference, the operation result corresponding to the target difference is used as the second target value. This ensures that when the first difference is positive, the operation result of the first difference (the second target value) is calculated to be 1 by multiple consecutive operation operators. Thus, when updating the element, the feature of the target point can be selected from the features of the first element and the target point through the second target value. It also ensures that when the first difference is negative, the operation result of the first difference (the second target value) is calculated to be 0 by setting the first difference to zero. Thus, when updating the element, the feature of the target point can be removed from the features of the first element and the target point through the second target value.

[0064] In S305 of this application embodiment, when the target difference is the second difference, the operation result corresponding to the target difference is used as the first target value. This ensures that when the second difference is positive, the operation result (first target value) of the second difference is calculated to 1 by multiple consecutive operation operators. Thus, when updating elements, the first element can be selected from the features of the first element and the target point using the first target value. That is, the original first element is retained during the update. It also ensures that when the second difference is negative, the operation result (first target value) of the second difference is calculated to 0 by setting it to zero. Thus, when updating elements, the first element can be removed from the features of the first element and the target point using the first target value.

[0065] In the above scheme, the same processing flow is applied to all second elements in the second tensor during the process of obtaining the first target value and the second target value. This not only enables batch processing of the second elements, but also enables the acquisition of the first target value and the second target value corresponding to each first point based on the size relationship between the second element and the first threshold. This achieves the classification of the first points that need to be updated by intensive computation.

[0066] Based on the above embodiments, as an optional embodiment, when the second element is less than the first threshold, the first target value is 1 and the second target value is 0; when the second element is not less than the first threshold, the first target value is 0 and the second target value is 1.

[0067] In this embodiment of the application, the first target value and the second target value are used together to select which feature to use as the updated first element from the first element of the corresponding first point and the feature of the target point.

[0068] In this embodiment of the application, when the second element is less than the first threshold, it indicates that the first point corresponding to the current second element meets the requirements, and the first element corresponding to the first point can be retained without updating. Therefore, the first target value is 1 and the second target value is 0 at this time, so that in the subsequent update process, the first element is selected as the updated first element by multiplying the first element with the first target value and multiplying the feature of the target point with the second target value.

[0069] In this embodiment of the application, when the second element is greater than the first threshold, it indicates that the first point corresponding to the current second element does not meet the requirements and the first element corresponding to the first point needs to be updated. Therefore, at this time, the first target value is 0 and the second target value is 1, so that in the subsequent update process, the feature of the target point is selected as the updated first element by multiplying the first element with the first target value and multiplying the feature of the target point with the second target value.

[0070] In the above scheme, if the second element is less than the first threshold, it means that the first point corresponding to the current second element meets the filtering requirements. In the subsequent update of the first tensor, the first element corresponding to this first point is retained. Therefore, the first target value is 1, and the second target value is 0. If the second element is greater than the first threshold, it means that the first point corresponding to the current second element does not meet the filtering requirements. In the subsequent update of the first tensor, the first element corresponding to this first point needs to be replaced with the feature of the target point. Therefore, the first target value is 0, and the second target value is 1. This method filters all elements in batches through calculation, eliminating the need for indexing operations and improving data processing efficiency.

[0071] Based on the above embodiments, as an optional embodiment, a method for obtaining the calculation result is provided, such as... Figure 4 As shown, the specific content is as follows: S401, Input the target difference into the first operation operator, and the first operation operator performs an up-rounding operation on the target difference to obtain the third value; S402, input the third value into the second operator, and the second operator converts the third value into an integer type to obtain the fourth value; S403, input the fourth value into the third operation operator, and the third operation operator restricts the fourth value to a preset first range to obtain the operation result corresponding to the target difference; In S401 of this application embodiment, the target difference is rounded up using a first operation operator to obtain a third value. The first operation operator can be ceil().

[0072] In S402 of this embodiment, the third value is converted to an integer type using a second operand to obtain the fourth value, that is, the floating-point number is converted to a 32-bit integer. The second operand can be int(). In S403 of this application embodiment, the first range is 0 to 1. That is, the value of the final operation result obtained by the third operation operator is between 0 and 1 (inclusive). When the fourth value is greater than or equal to 1, the operation result of the target difference is 1. Since the fourth value is a positive number obtained by rounding up, the fourth value must be greater than or equal to 1. Therefore, the operation result of the target difference is 1. The third operation operator can be clamp(0,1).

[0073] In the above scheme, the same continuous operation is performed on each positive target difference, which enables the processing of elements in the tensor one by one without the need to use index operations to extract the elements to be processed, thus improving the efficiency of data processing.

[0074] Based on the above embodiments, as an optional embodiment, for any target difference between the first difference and the second difference, a ReLU operation is used to determine whether the target difference is positive.

[0075] In this embodiment, the ReLU operation preserves the original value of the target difference that is positive and turns the target difference that is negative to 0. When processing massive tensor data (such as point clouds), ReLU can significantly save computing resources and time, improve the efficiency of data processing, and the nonlinear operation of ReLU achieves nonlinear mapping with extremely low computational cost, thereby achieving noise reduction of interference information and producing a filtering effect.

[0076] In this application embodiment, all the above-described application embodiments are executed by the NPU, since the NPU can perform highly parallel arithmetic. This application embodiment classifies the first element that needs to be updated through arithmetic operations, and achieves complex element selection through simple multiplication and addition, perfectly adapting to the hardware characteristics of the NPU.

[0077] In one example, a coded representation of a data processing method is provided, as follows: 1.tmask = sqrdists–radius2 2.greater_filter = tmask.relu().ceil().int().clamp(0,1) 3.smaller_filter = (-tmask).relu().ceil().int().clamp(0,1) 4. sqrdists_idx = sqrdists_idx smaller_filter + query_self greater_filter Where sqrdists represents the second element in the second tensor, radius2 represents the first threshold, tmask is the first difference, (-tmask) is the second difference, greater_filter is the second target value, smaller_filter is the first target value, sqrdists_idx is the first element in the first tensor, and query_self is the feature of the target point.

[0078] tmask = sqrdists–radius2 is used to represent obtaining the first difference between the second element and the first threshold.

[0079] `greater_filter = tmask.relu().ceil().int().clamp(0,1)` is used to indicate that when the first difference is determined to be greater than 0, the second target value is determined to be 1 through multiple consecutive operators `relu()`, `ceil()`, `int()` and `clamp(0,1)`, and when the first difference is determined to be less than 0, the second target value is determined to be 0 through `relu()`.

[0080] smaller_filter = (-tmask).relu().ceil().int().clamp(0,1) is used to indicate that when the second difference is determined to be greater than 0, the first target value is determined to be 1 by a series of consecutive operators relu(), ceil(), int() and clamp(0,1), and when the second difference is determined to be less than 0, the first target value is determined to be 0 by relu().

[0081] sqrdists_idx = sqrdists_idx smaller_filter + query_self The `greater_filter` parameter indicates that when the first target value is 1 and the second target value is 0, the updated first element is the original first element; when the first target value is 0 and the second target value is 1, the updated first element is the feature of the target point.

[0082] The data processing method provided in this application introduces a nonlinear factor through the ReLU operation to reconstruct the conditional mask into a filter containing zeros, so that multiplying the tensor by the filter produces an equivalent filtering effect. Furthermore, the ReLU operation, as a common activation function in neural networks, is very compatible with the NPU DSA architecture.

[0083] The data processing method provided in this application embodiment is architecture-friendly for NPU DSA architectures such as Zhouyi, and operations such as clamp, ceil, int, and relu can be deeply integrated into the DSA data flow, greatly improving the performance of data processing.

[0084] The data processing method provided in this application has a wide range of applications. It can be used to process models containing large amounts of index operators. For example, coordinate transformation (Voxel) in automotive-grade BEV models and coordinate transformation in various 3D and VR models can also be performed using the data processing method provided in this application.

[0085] The data processing method provided in this application uses the Zhouyi NPU X3P model as an example. With a production environment of 190,000 data points as a test, the original index scatter operation took 13ms, while the data processing method provided in this application, using the same data volume and the same hardware configuration, took only 0.39ms, achieving a 30-fold performance improvement.

[0086] This application provides a data processing apparatus, such as... Figure 5 As shown, the data processing device 50 may include: a processing module 501, an acquisition module 502, a determination module 503, and an update module 504.

[0087] Specifically, the processing module 501 is used to take each point in the point cloud data of the target object as the target point and take any point in the point cloud data other than the target point as the first point; The acquisition module 502 is used to acquire a first tensor and a second tensor corresponding to the target point. Each first element in the first tensor is a feature of a corresponding first point, and each second element in the second tensor is a feature difference between the target point and the corresponding first point. The determination module 503 is used to determine the target value combination of the first point corresponding to the second element for each second element based on the size relationship between the second element and the first threshold. The target value combination is used to characterize whether the first element corresponding to the first point needs to be updated. The update module 504 is used to update the first element for each first element based on the combination of the target values ​​of the first point corresponding to the first element and the characteristics of the target point, so as to obtain the updated first tensor.

[0088] The data processing apparatus provided in this application takes each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point, and obtains a first tensor and a second tensor corresponding to the target point. Since each first element in the first tensor is a feature of the corresponding first point and each second element in the second tensor is a feature difference between the target point and the corresponding first point, the apparatus achieves the summarization of the features of each first point through the first tensor and the summarization of the feature differences between the target point and each first point through the second tensor. This allows for batch processing of each element in the first tensor and the second tensor in subsequent data processing.

[0089] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the first point corresponding to each second element in the second tensor needs to be updated.

[0090] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing.

[0091] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0092] Furthermore, the target value combination includes a first target value and a second target value, wherein when the first target value is 0, the second target value is 1, and when the first target value is 1, the second target value is 0; For each first element, obtain the first product between the first element and the first target value, and obtain the second product between the feature of the target point and the second target value; For each first element, add the first product to the second product to obtain the updated first element.

[0093] In another possible implementation, for each second element, the first difference between the second element and the first threshold is obtained, and the first difference is inverted to obtain the second difference. For any target difference between the first difference and the second difference, determine whether the target difference is a positive number; If the target difference is determined to be positive, then the target difference is processed by at least one consecutive operation operator to obtain the operation result corresponding to the target difference. If the target difference is determined to be non-positive, then the target difference is set to zero to obtain the operation result corresponding to the target difference. If the target difference is the first target difference, the calculation result corresponding to the target difference is taken as the second target value; If the target difference is the second difference, the calculation result corresponding to the target difference is taken as the first target value.

[0094] In yet another possible implementation, when the second element is less than the first threshold, the first target value is 1 and the second target value is 0. When the second element is not less than the first threshold, the first target value is 0 and the second target value is 1.

[0095] In yet another possible implementation, the target difference is input into the first operator, which then rounds the target difference up to obtain the third value. The third value is input into the second operator, which converts the third value into an integer type to obtain the fourth value; The fourth value is input into the third operator, which restricts the fourth value to a preset first range to obtain the operation result corresponding to the target difference; the first range is 0 to 1.

[0096] In yet another possible implementation, the features of any point in the point cloud data are used to represent at least one of the following: Coordinate information; Color information; Reflection intensity information; Semantic information.

[0097] In another possible implementation, for any target difference between the first and second differences, a ReLU operation is used to determine whether the target difference is positive.

[0098] In yet another possible implementation, the aforementioned implementation is executed by the NPU.

[0099] This application provides an electronic device (computer device / equipment / system) including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a data processing method. Compared with related technologies, it can achieve the following: by taking each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point, a first tensor and a second tensor corresponding to the target point are obtained. Since each first element in the first tensor is a feature of the corresponding first point, and each second element in the second tensor is a feature difference between the target point and the corresponding first point, the features of each first point are summarized through the first tensor, and the feature differences between the target point and each first point are summarized through the second tensor. This allows each element in the first tensor and the second tensor to be processed in batches during subsequent data processing.

[0100] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the first point corresponding to each second element in the second tensor needs to be updated.

[0101] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing.

[0102] In one alternative embodiment, an electronic device is provided, such as Figure 6 As shown, Figure 6 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0103] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0104] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0105] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0106] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0107] The electronic device package may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0108] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve the following: by taking each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point, a first tensor and a second tensor corresponding to the target point are obtained. Since each first element in the first tensor is a feature of the corresponding first point, and each second element in the second tensor is a feature difference between the target point and the corresponding first point, the features of each first point are summarized through the first tensor, and the feature differences between the target point and each first point are summarized through the second tensor. This allows for batch processing of each element in the first and second tensors during subsequent data processing.

[0109] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the first point corresponding to each second element in the second tensor needs to be updated.

[0110] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing.

[0111] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0112] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve: By taking each point in the point cloud data of the target object as the target point and any point in the point cloud data other than the target point as the first point, the first tensor and the second tensor corresponding to the target point are obtained. Since each first element in the first tensor is the feature of the corresponding first point, and each second element in the second tensor is the feature difference between the target point and the corresponding first point, the features of each first point are summarized through the first tensor, and the feature differences between the target point and each first point are summarized through the second tensor. This allows for batch processing of each element in the first tensor and the second tensor in subsequent data processing.

[0113] For each second element, the system determines whether the first element of the corresponding first point in the first tensor needs to be updated based on the relationship between the second element and the first threshold. This confirms whether the first point corresponding to each second element in the second tensor needs to be updated.

[0114] For each first element, since the target value combination is used to characterize whether the first element corresponding to the first point needs to be updated, the first element is updated according to the target value combination of the first point corresponding to the first element and the characteristics of the target point. This realizes the update of each first element in the first tensor according to the update requirements, thereby obtaining the updated first tensor. This avoids the need to determine the first element that needs to be updated and update the corresponding first element by using a search operation, thus avoiding irregular data access and a large amount of serial processing caused by performing search operations, and improving the efficiency of data processing.

[0115] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the illustrations or text descriptions.

[0116] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0117] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A data processing method, characterized in that, include: Each point in the point cloud data of the target object is taken as the target point, and any point in the point cloud data other than the target point is taken as the first point; Obtain a first tensor and a second tensor corresponding to the target point, where each first element in the first tensor is a feature of a corresponding first point, and each second element in the second tensor is a feature difference between the target point and the corresponding first point; For each second element, a target value combination for the first point corresponding to the second element is determined based on the size relationship between the second element and the first threshold. The target value combination is used to characterize whether the first element corresponding to the first point needs to be updated. For each first element, the first element is updated based on the target value combination of the first point corresponding to the first element and the characteristics of the target point to obtain the updated first tensor.

2. The method according to claim 1, characterized in that, The target value combination includes a first target value and a second target value. When the first target value is 0, the second target value is 1, and when the first target value is 1, the second target value is 0. For each first element, updating the first element based on the target value combination of the first point corresponding to the first element and the characteristics of the target point includes: For each first element, obtain the first product between the first element and the first target value, and obtain the second product between the feature of the target point and the second target value; For each first element, add the first product to the second product to obtain the updated first element.

3. The method according to claim 2, characterized in that, For each second element, determining the target value combination of the first point corresponding to the second element based on the relationship between the second element and the first threshold includes: For each second element, obtain the first difference between the second element and the first threshold, and invert the first difference to obtain the second difference. For any target difference between the first difference and the second difference, determine whether the target difference is a positive number; If the target difference is determined to be positive, the target difference is processed by at least one consecutive operation operator to obtain the operation result corresponding to the target difference; if the target difference is determined to be non-positive, the target difference is set to zero to obtain the operation result corresponding to the target difference. If the target difference is the first difference, the calculation result corresponding to the target difference is taken as the second target value; If the target difference is the second difference, the calculation result corresponding to the target difference is taken as the first target value.

4. The method according to claim 2 or 3, characterized in that, When the second element is less than the first threshold, the first target value is 1 and the second target value is 0. When the second element is not less than the first threshold, the first target value is 0 and the second target value is 1.

5. The method according to claim 3, characterized in that, The step of processing the target difference using at least one consecutive operation operator to obtain the operation result corresponding to the target difference includes: The target difference is input into the first operation operator, which then rounds the target difference up to obtain the third value. The third value is input into the second operator, which converts the third value into an integer type to obtain the fourth value; The fourth value is input into the third operation operator, which restricts the fourth value to a preset first range to obtain the operation result corresponding to the target difference; the first range is 0 to 1.

6. The method according to claim 1, characterized in that, The feature of any point in the point cloud data is used to represent at least one of the following: Coordinate information; Color information; Reflection intensity information; Semantic information.

7. The method according to claim 3, characterized in that, The step of determining whether the target difference is a positive number for any one of the first difference and the second difference includes: For any target difference between the first difference and the second difference, a ReLU operation is used to determine whether the target difference is positive.

8. The method according to any one of claims 1-7, characterized in that, The method is executed by a neural network processor (NPU).

9. A data processing apparatus, characterized in that, include: The processing module is used to take each point in the point cloud data of the target object as the target point and take any point in the point cloud data other than the target point as the first point. The acquisition module is used to acquire a first tensor and a second tensor corresponding to the target point, wherein each first element in the first tensor is a feature of a corresponding first point, and each second element in the second tensor is a feature difference between the target point and the corresponding first point; The determination module is used to determine, for each second element, a target value combination of the first point corresponding to the second element based on the size relationship between the second element and the first threshold. The target value combination is used to characterize whether the first element corresponding to the first point needs to be updated. The update module is used to update the first element for each first element according to the target value combination of the first point corresponding to the first element and the characteristics of the target point, so as to obtain the updated first tensor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-8.

Images