Data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data processing method and device, electronic equipment and storage medium, the data processing method comprising: obtaining 3D point cloud data of a posture of a target user; inputting the 3D point cloud data into a trained recognition model, processing the 3D point cloud data by the recognition model, and outputting a label corresponding to the 3D point cloud data, the label indicating an intention corresponding to the posture; determining the intention corresponding to the posture according to a preset correspondence between labels and intentions.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] As the population ages, the number of elderly patients is increasing, and the observation and care of these patients consumes a significant amount of manpower. In some situations, call buttons are installed to provide timely care for elderly patients. However, in emergencies, such as when the elderly person is immobile, they may not be able to press the button themselves, leading to delays in handling sudden situations during medical care. Consequently, the reliability and timeliness of medical monitoring are both low. Summary of the Invention

[0003] The purpose of this application is to provide a data processing method, apparatus, electronic device, and storage medium that can promptly detect and handle sudden situations of patients, and has high reliability and timeliness in medical monitoring.

[0004] To solve the above-mentioned technical problems, the embodiments of this application are implemented as follows:

[0005] In a first aspect, embodiments of this application provide a data processing method, which includes: acquiring 3D point cloud data of the pose of a target user; inputting the 3D point cloud data into a trained recognition model, processing the 3D point cloud data through the recognition model, and outputting a label corresponding to the 3D point cloud data, wherein the label indicates the intention corresponding to the pose; and determining the intention corresponding to the pose based on a preset correspondence between the label and the intention.

[0006] Secondly, embodiments of this application provide a data processing apparatus, comprising: an acquisition module for acquiring 3D point cloud data of a target user's posture; a processing module for inputting the 3D point cloud data into a trained recognition model, processing the 3D point cloud data through the recognition model, and outputting a label corresponding to the 3D point cloud data, the label indicating the intention corresponding to the posture; and a determination module for determining the intention corresponding to the posture based on the correspondence between the label and the intention.

[0007] Thirdly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory to implement the data processing method steps mentioned in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data processing method steps mentioned in the first aspect.

[0009] As can be seen from the technical solutions provided in the embodiments of this application above, by acquiring 3D point cloud data of the target user's posture, inputting the 3D point cloud data into a trained recognition model, processing the 3D point cloud data through the recognition model, and outputting labels corresponding to the 3D point cloud data, the labels indicate the intention corresponding to the posture. Based on the preset correspondence between labels and intentions, the intention corresponding to the posture is determined. Thus, by inputting the 3D point cloud data of the target user's posture into the recognition model, the labels corresponding to the 3D point cloud data can be identified, thereby determining the user's intention. Therefore, when the user has a need, they only need to make the corresponding posture and the recognition model will identify the intention corresponding to the posture. Then, medical staff can promptly handle the situation according to the user's intention, which can promptly detect and handle the patient's emergencies. The reliability and timeliness of medical monitoring are both high. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating the data processing method provided in an embodiment of this application;

[0012] Figure 2 A schematic diagram of the module composition of a data processing apparatus provided in an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] The purpose of this application is to provide a data processing method, apparatus, electronic device, and storage medium that can promptly detect and handle sudden situations of patients, and has high reliability and timeliness in medical monitoring.

[0015] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0016] For example, such as Figure 1 As shown in the figure, this application provides a data processing method, which can be executed by a server. The data processing method may specifically include the following steps:

[0017] In step S101, 3D point cloud data of the target user's pose is acquired.

[0018] Specifically, user postures include, but are not limited to, head postures, hand gestures, and foot postures. Head postures include, but are not limited to, tilting the head back, shaking the head, lowering the head, leaning back, and tilting the head. Hand gestures include, but are not limited to, waving the hand, bending the fingers, raising the fingers, clapping the hands together, making a fist, and clapping. Foot postures include, stepping forward, stepping backward, and moving left and right. Of course, user postures can also be other postures, which are not limited in this embodiment. Different user postures correspond to different user intentions. Taking hand gestures as an example, clapping indicates that the user intends to eat, waving the hand indicates that the user intends to go to the toilet, and making a fist with one hand indicates that the user intends to feel unwell. The intentions corresponding to user postures can be set according to actual conditions, and are not limited in this embodiment. Furthermore, considering the decline in physiological functions of users (such as the elderly) and the occurrence of emergencies, an alarm is directly triggered when the user's posture is a fallen or prone position.

[0019] 3D point cloud data is data acquired using a 3D depth image camera. 3D point cloud data refers to 3D data in point cloud format. Point cloud means that the original set information is preserved in 3D space without any discretization.

[0020] In step S103, the 3D point cloud data is input into the trained recognition model, the 3D point cloud data is processed by the recognition model, and the label corresponding to the 3D point cloud data is output.

[0021] The label indicates the intent corresponding to the gesture.

[0022] Specifically, the recognition model can be implemented using algorithms such as self-attention mechanisms, support vector machines, and decision trees. This model identifies the user's intent corresponding to 3D point cloud data. The labels corresponding to the output 3D point cloud data represent the user's intent, which indicates the user's current need.

[0023] In one possible implementation, the recognition model used in this application embodiment is a self-attention model. Specifically, 3D point cloud data is input into the trained recognition model, and the 3D point cloud data is processed by the recognition model including: converting the three-dimensional coordinates of the 3D point cloud data into higher-order moments, multiplying the higher-order moments with the transformation matrix of the self-attention network for alignment to obtain transformed data, extracting local features and global features of the transformed data, performing inverse interpolation and skip connections on the local and global features to obtain relevant feature values, and after max pooling and full connection on the relevant feature values, identifying the label corresponding to the 3D point cloud data.

[0024] Specifically, a frame of 3D point cloud data is input into the trained recognition model, represented as an N*3 2D tensor, where N represents the number of points and 3 corresponds to the x, y, and z coordinates in 3D space. A polynomial function is added to the 3D coordinates so that the recognition model can consider the higher-order geometric moments of the point cloud, thereby improving classification accuracy. For example, the 3D coordinates are transformed from (x, y, z) to (x, y, z, x², y², z², xy, xz, yz). Then, the transformed higher-order moments are multiplied by the transformation matrix of the self-attention network for alignment, ensuring the invariance of specific control transformations. Specifically, the alignment operation treats the input higher-order moment point cloud data as an n*9*1 single-channel image, performs three convolutions and one pooling operation, compresses it to 1024 nodes, and then connects it to two fully connected layers. Local features refer to features within a specific region of the transformed data, while global features refer to features across the entire transformed data region. Local feature extraction can be performed using local aggregation, while multilayer perceptron extraction can be used for global feature extraction.

[0025] Furthermore, regarding inverse interpolation, it involves transferring the features of known data points to the target data points through interpolation. The calculation formula is as follows:

[0026]

[0027] Specifically, wi is the weight of the calculated data point, which is inversely proportional to the distance between the data points; the closer the distance, the greater the influence. f(j) is the unknown point value, fi(j) is the known point value, k indicates that k data points are selected from the known point set for interpolation calculation, and P represents the degree of influence of distance on the weight. For each data point x in P2, find the k closest data points x1, ... x2, ... xk in P1 in the original data point cloud coordinate space. Currently, upsampling is achieved by interpolating the features of more data points with fewer data points using reverse interpolation. At this time, the features x1, ... x2, ... xk are known. If we want to obtain the features of data point x as shown in the above formula, we actually sum the weighted features of data points x1, ... x2, ... xk to obtain the features of data point x. This weight is inversely related to the distance between x and x1, ... x2, ... xk, meaning that the farther the data point is, the smaller its contribution to the features of x. The same applies to other points in P2, thus realizing the upsampling and backpropagation of features. Skip linking involves concatenating global and local optimization variables as features, similar to the downsampling operation in U-net, to obtain relevant feature values.

[0028] After obtaining the relevant feature values, a max pooling operation and a fully connected layer are used to segment the recognition task using the cross-entropy loss function, and the intention corresponding to the segmented pose is then used for intent recognition.

[0029] In one possible implementation, extracting local features from the transformed data includes: downsampling the transformed data to obtain a target number of data points; aggregating the data points according to a preset search radius and the number of points in the sub-region to obtain aggregated data; and then extracting local features from the aggregated data after max pooling.

[0030] Specifically, downsampling is performed on the transformed data. This involves randomly selecting a point as the initial point, calculating the distance between each point in the unselected sampling point set and the known sampling points, adding the point with the largest distance to the selected sampling point set, updating the distances, and iterating this process until the target number of data points is obtained. Then, an aggregation operation is performed. A preset search radius R and number of sub-region points K are used to aggregate the target number of data points. Specifically, n points are selected as centers from the target number of data points, and a sphere with radius R is drawn. For each sphere, K points are selected based on their distance from smallest to largest. If the number of points is greater than K, the first K points are selected as the sub-region; otherwise, resampling is performed until the target size K is reached. After data aggregation, for each sub-region, the K aggregated data points are processed through a 1x1 convolutional layer, batch sample normalization, and a ReLU non-linear activation function, followed by max pooling to extract local features.

[0031] In one possible implementation, extracting global features involves using multiple multilayer perceptrons to extract global features from the transformed data.

[0032] Specifically, a perceptron is a linear classification model for binary classification. It uses multiple perceptrons to extract global features from each 3D point cloud data, and then uses a self-attention network (transformer-net) for feature alignment.

[0033] In step S105, the intention corresponding to the gesture is determined according to the preset correspondence between the label and the intention.

[0034] Specifically, the correspondence between labels and intents can be pre-defined. For example, the intent corresponding to label 1 is eating, the intent corresponding to label 2 is going to the toilet, and the intent corresponding to label 3 is taking medicine. After the above steps identify the labels corresponding to the 3D point cloud data through the recognition model, the intent corresponding to the label is determined, that is, the intent corresponding to the user's current posture is determined.

[0035] After determining the user's intent, medical staff will provide services to the user or promptly address the user's current needs.

[0036] The technical solution disclosed in this application involves acquiring 3D point cloud data of the target user's posture, inputting the 3D point cloud data into a trained recognition model, processing the 3D point cloud data, and outputting labels corresponding to the 3D point cloud data. The labels indicate the intention corresponding to the posture. Based on the preset correspondence between labels and intentions, the intention corresponding to the posture is determined. Thus, by inputting the 3D point cloud data of the target user's posture into the recognition model, the labels corresponding to the 3D point cloud data can be identified, thereby determining the user's intention. Therefore, when the user has a need, they only need to make the corresponding posture, and the recognition model will identify the intention corresponding to the posture. Then, medical staff can promptly handle the situation according to the user's intention, enabling timely detection and handling of sudden situations in patients. The reliability and timeliness of medical monitoring are both high.

[0037] Furthermore, to improve the recognition accuracy of the subsequent recognition model, data preprocessing and data augmentation operations can be performed. In one possible implementation, after acquiring the 3D point cloud data of the target user's pose, the method further includes: preprocessing the 3D point cloud data to obtain initial data; performing data augmentation on the initial data to obtain augmented data, whereby data augmentation includes at least one of the following: flipping; rotating; scaling; cropping; translating; adding noise; and inputting the 3D point cloud data into a trained recognition model. Processing the 3D point cloud data through the recognition model includes: inputting the augmented data into the trained recognition model and processing the augmented data through the recognition model.

[0038] Specifically, preprocessing of 3D point cloud data includes cleaning outliers, correcting erroneous data, data integration, and data modification. Data augmentation employs the KPConv algorithm to randomly add colors, the Point-BERT algorithm to use common point clouds, and random sampling of 1024 point cloud data points from the original point cloud data for data scaling, resampling, and higher-order data augmentation. It is worth noting that the process of the recognition model processing augmented data is similar to the process of the recognition model processing unaugmented 3D point cloud data, and will not be repeated here in this application's embodiments. Thus, by performing data augmentation operations before processing the data through the recognition model, the recognition accuracy of the subsequent recognition model is improved.

[0039] In one possible implementation, before inputting the 3D point cloud data into the trained recognition model, the method further includes: acquiring training samples, which include 3D point cloud data of different poses and labels corresponding to each pose, the labels indicating the intent corresponding to the pose; preprocessing the training samples and performing data augmentation on the preprocessed training samples to obtain augmented samples; and inputting the augmented samples into the recognition model to be trained to obtain a trained recognition model.

[0040] Specifically, preprocessing of 3D point cloud data includes cleaning outliers, correcting erroneous data, data integration, and data transformation. Data augmentation includes at least one of the following: flipping; rotating; scaling; cropping; translating; and adding noise.

[0041] In one possible implementation, training the enhanced samples by inputting them into the recognition model includes: converting the 3D coordinates of the training samples into higher-order moments; aligning the higher-order moments by multiplying them with the transformation matrix of the self-attention network to obtain transformed data; extracting local features and global features from the transformed data; performing back-interpolation and skip connections on the local and global features to obtain relevant feature values; performing max pooling and full connections on the relevant feature values, and segmenting the recognition task using the cross-entropy loss function; and performing intent recognition on the intent corresponding to the pose of the segmented task.

[0042] Specifically, training samples are input into the recognition model to be trained. These training samples include 3D point cloud data, represented as an N*3 2D tensor, where N represents the number of points and 3 corresponds to the x, y, and z coordinates in 3D space. A polynomial function is added to the 3D coordinates so that the recognition model can consider the higher-order geometric moments of the point cloud, thereby improving classification accuracy. For example, the 3D coordinates are transformed from (x, y, z) to (x, y, z, x², y², z², xy, xz, yz). Then, the transformed higher-order moments are multiplied by the transformation matrix of the self-attention network for alignment, ensuring the invariance of specific control transformations. Specifically, the alignment operation involves treating the input higher-order moment point cloud data as an n*9*1 single-channel image, performing three convolutions and one pooling operation, compressing it into 1024 nodes, and then connecting it to two fully connected layers. Local features refer to features within a specific region of the transformed data, while global features refer to features across the entire transformed data region. Local feature extraction can be performed using local aggregation, while multilayer perceptron extraction can be used for global feature extraction.

[0043] Furthermore, regarding inverse interpolation, it involves transferring the features of known training sample data points to the target training sample data points through interpolation. The calculation formula is as follows:

[0044]

[0045] Specifically, wi is the weight of the calculated data point, which is inversely proportional to the distance between the data points; the closer the distance, the greater the influence. f(j) is the unknown point value, fi(j) is the known point value, k indicates that k data points are selected from the known point set for interpolation calculation, and P represents the degree of influence of distance on the weight. For each data point x in P2, find the k closest data points x1, ... x2, ... xk in P1 in the original data point cloud coordinate space. Currently, upsampling is achieved by interpolating the features of more data points with fewer data points using reverse interpolation. At this time, the features x1, ... x2, ... xk are known. If we want to obtain the features of data point x as shown in the above formula, we actually sum the weighted features of data points x1, ... x2, ... xk to obtain the features of data point x. This weight is inversely related to the distance between x and x1, ... x2, ... xk, meaning that the farther the data point is, the smaller its contribution to the features of x. The same applies to other points in P2, thus realizing the upsampling and backpropagation of features. Skip linking involves concatenating global and local optimization variables as features, similar to the downsampling operation in U-net, to obtain relevant feature values.

[0046] After obtaining the relevant feature values, a max pooling operation and a fully connected layer are used to segment the recognition task using the cross-entropy loss function, and the intention corresponding to the segmented pose is then used for intent recognition.

[0047] In each training process, the network structure of the recognition model is adjusted, specifically by field scaling, which involves training the network with different query radii, such as 0.25r, 0.5r, 1.5r, and 2.5r. Then, models such as PointMLP and Point Transformer are used for auxiliary training.

[0048] Furthermore, to enhance the accuracy of reasoning, five consecutive frames of point cloud data can be collected as a category, and support vector machines, decision trees, and other auxiliary methods can be used for classification reasoning. After the corresponding reasoning is performed, the interface scheduler allocates the corresponding interface and issues the corresponding prompt information for the different intentions identified after segmentation. For example, if the intention is identified as going to the toilet, the interface scheduler schedules the interface with the intention of going to the toilet and issues a toilet prompt to remind relevant personnel to handle the matter in a timely manner.

[0049] Based on the same technical concept, and corresponding to the data processing method provided in the above embodiments, this application also provides a data processing apparatus. Figure 2 This application provides a schematic diagram of the module composition of a data processing device, which is used to perform... Figure 1 The described data processing methods, such as Figure 2 As shown, the data processing device 200 includes: an acquisition module 201, used to acquire 3D point cloud data of the pose of a target user; a processing module 202, used to input the 3D point cloud data into a trained recognition model, process the 3D point cloud data through the recognition model, and output labels corresponding to the 3D point cloud data, the labels indicating the intention corresponding to the pose; and a determination module 203, used to determine the intention corresponding to the pose based on the correspondence between the labels and the intention.

[0050] As can be seen from the technical solutions provided in the embodiments of this application above, by acquiring 3D point cloud data of the target user's posture, inputting the 3D point cloud data into a trained recognition model, processing the 3D point cloud data through the recognition model, and outputting labels corresponding to the 3D point cloud data, the labels indicate the intention corresponding to the posture. Based on the preset correspondence between labels and intentions, the intention corresponding to the posture is determined. Thus, by inputting the 3D point cloud data of the target user's posture into the recognition model, the labels corresponding to the 3D point cloud data can be identified, thereby determining the user's intention. Therefore, when the user has a need, they only need to make the corresponding posture and the recognition model will identify the intention corresponding to the posture. Then, medical staff can promptly handle the situation according to the user's intention, which can promptly detect and handle the patient's emergencies. The reliability and timeliness of medical monitoring are both high.

[0051] In one possible implementation, the processing module 202 is further configured to preprocess the 3D point cloud data to obtain initial data; perform data augmentation on the initial data to obtain augmented data, wherein the data augmentation includes at least one of the following: flipping; rotating; scaling and cropping; translating; adding noise; and inputting the augmented data into a trained recognition model, thereby processing the augmented data through the recognition model.

[0052] In one possible implementation, the processing module 202 is further configured to acquire training samples, which include 3D point cloud data of different poses and labels corresponding to each pose, the labels indicating the intent corresponding to the pose; preprocess the training samples and perform data augmentation on the preprocessed training samples to obtain augmented samples; input the augmented samples into the recognition model to be trained for training to obtain a trained recognition model.

[0053] In one possible implementation, the processing module 202 is further configured to convert the three-dimensional coordinates of the training samples into higher-order moments; align the higher-order moments by multiplying them with the transformation matrix of the self-attention network to obtain transformed data; extract local features and global features from the transformed data; perform back-interpolation and skip connections on the local and global features to obtain relevant feature values; perform max pooling and full connection on the relevant feature values, and segment the recognition task using the cross-entropy loss function; and perform intent recognition on the intent corresponding to the pose of the segmented task.

[0054] In one possible implementation, the processing module 202 is further configured to convert the three-dimensional coordinates of the 3D point cloud data into higher-order moments; align the higher-order moments by multiplying them with the transformation matrix of the self-attention network to obtain transformed data; extract local features and global features of the transformed data; perform inverse interpolation and skip connections on the local and global features to obtain relevant feature values; and identify the labels corresponding to the 3D point cloud data after performing max pooling and full connection on the relevant feature values.

[0055] In one possible implementation, the processing module 202 is further configured to downsample the transformed data to obtain a target number of data points; aggregate the data points according to a preset search radius and the number of points in the sub-region to obtain aggregated data; and extract local features by max pooling the aggregated data.

[0056] In one possible implementation, the processing module 202 is also used to extract global features from the transformed data using multiple multilayer perceptrons.

[0057] The data processing apparatus provided in this application embodiment can implement the various processes in the embodiments corresponding to the above data processing method, and has the same or similar beneficial effects. To avoid repetition, it will not be described again here.

[0058] It should be noted that the data processing apparatus and the data processing method provided in the embodiments of this application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned data processing method and has the same or similar beneficial effects. Repeated parts will not be described again.

[0059] Corresponding to the data processing method provided in the above embodiments, based on the same technical concept, this application also provides an electronic device for performing the above data processing method. Figure 3 To illustrate the structure of an electronic device according to various embodiments of this application, as shown in the following diagrams... Figure 3 As shown. Electronic devices can vary considerably due to differences in configuration or performance, and may include one or more processors 301 and memory 302. Memory 302 may store one or more application programs or data. Memory 302 may be temporary or persistent storage. The application programs stored in memory 302 may include one or more modules (not shown), and each module may include a series of computer-executable instructions for the electronic device.

[0060] Furthermore, the processor 301 may be configured to communicate with the memory 302 and execute a series of computer-executable instructions stored in the memory 302 on the electronic device. The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input / output interfaces 305, and one or more keyboards 306.

[0061] Specifically, in this embodiment, the electronic device includes a processor, a communication interface, a memory, and a communication bus; wherein, the processor, the communication interface, and the memory communicate with each other via the bus; the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory to achieve the above.Figure 1 The steps in the method embodiments are the same as those in the above method embodiments, and have the same beneficial effects. To avoid repetition, the embodiments of this application will not be described again here.

[0062] This embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the above-mentioned functions. Figure 1 The steps in the method embodiments are the same as those in the above method embodiments, and have the same beneficial effects. To avoid repetition, the embodiments of this application will not be described again here.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes​ The steps of the function specified in one or more boxes.

[0067] In a typical configuration, an electronic device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0068] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0069] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0071] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data processing method, characterized in that, The data processing method includes: Acquire 3D point cloud data of the target user's pose; The 3D point cloud data is input into a trained recognition model, which processes the 3D point cloud data and outputs a label corresponding to the 3D point cloud data. The label indicates the intention corresponding to the pose. The intention corresponding to the gesture is determined based on the preset correspondence between the tags and the intentions. The step of inputting the 3D point cloud data into a trained recognition model and processing the 3D point cloud data through the recognition model includes: The three-dimensional coordinates of the 3D point cloud data are converted into higher-order moments; The higher-order moments are multiplied by the transformation matrix of the self-attention network and aligned to obtain the transformed data; Extract local features from the transformed data and extract global features from the transformed data; The local features and the global features are subjected to inverse interpolation and skip connections to obtain relevant feature values; After performing max pooling and full connection on the relevant feature values, the labels corresponding to the 3D point cloud data are identified.

2. The data processing method according to claim 1, characterized in that, After acquiring the 3D point cloud data of the target user's pose, the method further includes: The 3D point cloud data is preprocessed to obtain initial data; The initial data is augmented to obtain augmented data, wherein the data augmentation includes at least one of the following: Flip; Rotate; Scaling; Cutting; Translation; Add noise; The step of inputting the 3D point cloud data into the trained recognition model and processing the 3D point cloud data through the recognition model includes: The augmented data is input into the trained recognition model, and the recognition model processes the augmented data.

3. The data processing method according to claim 1 or 2, characterized in that, Before inputting the 3D point cloud data into the trained recognition model, the method further includes: Acquire training samples, which include 3D point cloud data of different poses and labels corresponding to each pose, wherein the labels indicate the intention corresponding to the pose; The training samples are preprocessed, and the preprocessed training samples are then augmented to obtain augmented samples. The enhanced samples are input into the recognition model to be trained to obtain the trained recognition model.

4. The data processing method according to claim 3, characterized in that, The step of inputting the enhanced samples into the recognition model to be trained includes: The three-dimensional coordinates of the training samples are converted into higher-order moments; The higher-order moments are multiplied by the transformation matrix of the self-attention network and aligned to obtain the transformed data; Extract local features from the transformed data and extract global features from the transformed data; The local features and the global features are subjected to inverse interpolation and skip connections to obtain relevant feature values; Max pooling and fully connected processing are applied to the relevant feature values, and the cross-entropy loss function is used to segment the recognition task. Intent recognition is performed on the intent corresponding to the pose of the segmented task.

5. The data processing method according to claim 1, characterized in that, The extraction of local features from the transformed data includes: The transformed data is downsampled to obtain the target number of data points; The data points are aggregated according to the preset search radius and the number of points in the sub-region to obtain aggregated data; The local features are extracted after max pooling the aggregated data.

6. The data processing method according to claim 1, characterized in that, The extraction of global features from the transformed data includes: Global features are extracted from the transformed data using multiple multilayer perceptrons.

7. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire 3D point cloud data of the target user's pose. The processing module is used to input the 3D point cloud data into the trained recognition model, process the 3D point cloud data through the recognition model, and output the label corresponding to the 3D point cloud data, wherein the label indicates the intention corresponding to the pose. The determination module is used to determine the intent corresponding to the gesture based on the correspondence between the label and the intent; The processing module is also used to convert the three-dimensional coordinates of 3D point cloud data into higher-order moments; to align the higher-order moments by multiplying them with the transformation matrix of the self-attention network to obtain transformed data; to extract local features and global features of the transformed data; to perform inverse interpolation and skip connections on the local and global features to obtain relevant feature values; and to identify the labels corresponding to the 3D point cloud data after performing max pooling and full connection on the relevant feature values.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the data processing method steps as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the data processing method steps as described in any one of claims 1-6.

Images