Shielding part point cloud dynamic completion method and system

Abstract

The invention provides a shielded part point cloud dynamic completion method and system, and the method comprises the steps: collecting body surface point cloud data in a non-shielding state and an arm shielding state in a human body breathing process, and building a voxel model; preprocessing the voxel model, including farthest point sampling and global scaling, so as to improve a model training effect; dividing the point cloud data into four quadrant areas according to the geometric center of the point cloud data so as to simulate the arm shielding condition; dynamically complementing the point cloud of the arm occlusion area by using the trained network, inputting the point cloud data of the occlusion area, and outputting the complemented point cloud data by the network; and carrying out dimensionality reduction on the complemented continuous frame point cloud by utilizing principal component analysis (PCA), and extracting one-dimensional breathing motion characteristics to obtain predicted breathing characteristics. According to the method, the PF-Net network and the LSTM network are fused to capture the spatial features of different time points, and the spatial-temporal features are fused on the global level, so that accurate analysis and complementation of dynamic point cloud data are realized, the problem of misjudgment of respiratory frequency, amplitude and mode caused by data missing in neonatal respiratory monitoring is solved, and the accuracy of neonatal respiratory monitoring is improved. And the monitoring precision and the clinical application value are improved.

Description

Technical Field

[0001] This application relates to the field of medical data processing technology, and in particular to a method and system for dynamic point cloud completion of occluded areas. Background Technology

[0002] In recent years, the health of newborns and infants has received widespread attention, and neonatal respiratory monitoring, as a crucial aspect of ensuring their health, has become particularly important. Traditional monitoring methods often rely on wearable sensors, such as adhesive electrodes, but these methods can damage the delicate skin of newborns. Therefore, research into non-contact respiratory monitoring technology is especially important, as its advantages include improved patient comfort, reduced infection risk, and enhanced monitoring convenience, continuity, and accuracy.

[0003] Among numerous non-contact technologies, depth cameras are widely used for continuous monitoring of respiratory rate due to their cost-effectiveness, safety, and adaptability to ambient light. However, in neonatal monitoring scenarios, frequent movement and occlusion by bedding often result in missing point cloud data in the chest and abdominal areas, affecting monitoring accuracy. Therefore, effectively addressing the lack of point cloud data has become a key issue in improving monitoring performance. Point cloud completion technology has emerged to address this issue, aiming to predict and reconstruct data points lost due to occlusion or resolution limitations using complex algorithms, thereby obtaining a more complete 3D point cloud model to accurately capture respiratory motion characteristics.

[0004] Currently, some progress has been made in the classification, segmentation, and completion of static point clouds, but research on dynamic point cloud data processing is relatively limited. The characteristics of dynamic point cloud data make it difficult to directly apply traditional time series models (such as RNN, LSTM, and GRU). The proposed Point Resolution Network (PF-Net) is a deep learning model specifically designed for point cloud completion, possessing the ability to effectively predict missing regions, especially performing well when dealing with irregular datasets. However, the original design of PF-Net primarily focused on static point clouds, and its performance may be limited in applications involving dynamic point clouds, particularly in scenarios with continuous time series changes. Summary of the Invention

[0005] In view of this, the purpose of this application is to propose a method and system for dynamic point cloud completion of occluded areas, which can specifically solve existing problems. A dynamic point cloud completion network combining PF-Net and Long Short-Term Memory (LSTM) multi-feature fusion is proposed to solve the problem of misjudgment of respiratory rate, amplitude, and pattern caused by missing data in neonatal respiratory monitoring.

[0006] To achieve the above objectives, this application proposes a method for dynamic point cloud completion of occluded areas, including:

[0007] Step S1: Collect point cloud data of the human body surface in an unobstructed state and in an arm-obstructed state during human respiration, and establish a voxel model based on the point cloud data of the human body surface.

[0008] Step S2: Preprocess the voxel model, including farthest point sampling and global scaling;

[0009] Step S3: Calculate the geometric center of all point clouds, and divide the point cloud data into four quadrants based on the center to simulate the four regions occluded by the arm;

[0010] Step S4: Train the network using a normal, unoccluded point cloud dataset. Use the trained network to dynamically complete the point cloud data of the arm-occluded area. Input the point cloud data of the occluded area into the trained network, and the network outputs the completed point cloud data.

[0011] To achieve the above objectives, this application also proposes a dynamic point cloud completion system for occluded areas, comprising:

[0012] The model building module is used to collect surface point cloud data of the human body in an unobstructed state during breathing and surface point cloud data of the human body in an arm-obstructed state, and to build a voxel model based on the surface point cloud data.

[0013] The preprocessing module is used to preprocess the voxel model, including farthest point sampling and global scaling.

[0014] The region division module is used to calculate the geometric center of all point clouds and divide the point cloud data into four quadrant regions based on the center, simulating the four regions occluded by the arm.

[0015] The dynamic completion module is used to train the network using a normal, unoccluded point cloud dataset. The trained network is then used to dynamically complete the point cloud data of the arm-occluded area. The point cloud data of the occluded area is input into the trained network, and the network outputs the completed point cloud data.

[0016] In summary, the advantages of this application and the user experience it brings are as follows: It provides a dynamic point cloud completion method for respiratory motion monitoring, which captures spatial features at different time points by integrating PF-Net and LSTM networks, and integrates spatiotemporal features at the global level to achieve accurate analysis and completion of dynamic point cloud data. This solves the problem of misjudgment of respiratory frequency, amplitude and pattern caused by missing data in neonatal respiratory monitoring, and improves monitoring accuracy and clinical application value. Attached Figure Description

[0017] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0018] Figure 1 A flowchart illustrating a method for dynamically completing point clouds of occluded areas according to an embodiment of this application is shown.

[0019] Figure 2 A flowchart for collecting point cloud data of the chest and abdomen and establishing a voxel model.

[0020] Figure 3 A network framework diagram for completing dynamic point clouds.

[0021] Figure 4 This is a visualization of the point cloud predicted by the network and the actual point cloud.

[0022] Figure 5 A comparison of respiratory motion features extracted by PCA dimensionality reduction from predicted and actual point cloud values ​​in four regions.

[0023] Figure 6 This is a schematic diagram of the Pearson correlation coefficient between the actual and predicted respiratory motion characteristics.

[0024] Figure 7 The diagram illustrates the creation of voxel models of the four arm-occluded regions, the segmentation of the arm region point cloud, and the visualization effect of network completion.

[0025] Figure 8 A schematic diagram of respiratory motion features extracted using PCA dimensionality reduction from point clouds predicted by the network for four arm-occluded areas.

[0026] Figure 9 A diagram illustrating the configuration of a point cloud dynamic completion system for occluded areas according to an embodiment of this application is shown.

[0027] Figure 10 A schematic diagram of the structure of an electronic device provided in one embodiment of this application is shown.

[0028] Figure 11 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0029] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] This application proposes an improved model based on PF-Net, L-PF-Net, which incorporates a Long Short-Term Memory (LSTM) module to process temporal information. L-PF-Net not only inherits PF-Net's spatial feature extraction capabilities but also utilizes LSTM to capture spatial features at different time points, enabling efficient and accurate analysis and completion of dynamic point cloud data. Both theoretical and experimental results demonstrate that L-PF-Net significantly improves the accuracy and robustness of dynamic point cloud completion tasks, particularly suitable for point cloud completion under neonatal occlusion conditions, providing more reliable data support for respiratory monitoring.

[0032] In order to achieve the purpose of this application, such as Figure 1 As shown, in one embodiment of this application, a method for dynamic point cloud completion of occluded areas based on an improved PF-Net network is provided, including the following steps:

[0033] Step S1: Collect point cloud data of the human body surface in an unobstructed state and in an arm-obstructed state during human respiration, and establish a voxel model based on the point cloud data of the human body surface.

[0034] Step S2: Preprocess the point cloud data voxel model, including farthest point sampling and global scaling, to improve the effect of subsequent model training.

[0035] Step S3: Calculate the geometric center of all point clouds, and divide the point cloud data into four quadrant regions (upper left, upper right, lower left, and lower right) based on the center to simulate the four regions occluded by the arm.

[0036] Step S4: Train the network on a normal, unoccluded point cloud dataset, and then use the trained network to dynamically complete the point cloud data of the occluded area of ​​the arm. Input the point cloud data of the occluded area, and the network outputs the completed point cloud data.

[0037] Step S5: Use Principal Component Analysis (PCA) to reduce the dimension of the continuous frame point cloud at the completed position and extract one-dimensional respiratory motion features.

[0038] The specific method for dynamic point cloud completion of occluded areas based on an improved PF-Net network described in this application includes the following steps: In step S1, point cloud data of the human body surface in an unoccluded state and in an arm-occluded state during human respiration are collected. A voxel model is established based on the point cloud data. The voxel model of the point cloud data is preprocessed, including sampling the farthest point and globally scaling to unify the number and scale of the point cloud in the dataset, in order to improve the effect of subsequent model training. By calculating the geometric center of all point clouds, the point cloud data is divided into four quadrant regions (upper left, upper right, lower left, and lower right) according to the center, simulating the four regions of arm occlusion. The training set and test set are processed, and the network is used to train on the normal unoccluded point cloud dataset. The trained network is then used to dynamically complete the point cloud of the segmented arm-occluded region. The occluded region point cloud data is input, and the network outputs the completed point cloud data. Finally, principal component analysis (PCA) is used to reduce the dimensionality of the continuous frame point cloud at the completed position to extract one-dimensional respiratory motion features. Finally, the effect of the network's dynamic completion is evaluated.

[0039] In step S1, a calibration plate is used to unify the coordinates of two RGB-D depth cameras fixed on both sides of the incubator (for newborns) to the same coordinate system, and point cloud information of the chest and abdomen surface is collected. Noise points are removed using a statistical filtering algorithm. Then, coarse registration of the point cloud is performed. FPFH features are used to calculate matching corresponding points in adjacent frames of the point cloud. FPFH features are calculated by calculating the normal vector of each point and the normal vector of points in the neighborhood. Point pairs are obtained by matching based on the FPFH features, and RANSAC is used to remove incorrect matching points. Rigid body transformation is estimated from the matching point pairs, and the source point cloud is registered to the target point cloud. The ICP algorithm is then used for fine registration of the two sets of point cloud information. Background areas and redundant information are removed using RGB and boundary threshold segmentation. Moving least squares algorithm is used to smooth the point cloud data. The processed point cloud information is inserted into an Octomap octree map, and a voxel map is created by defining the occupied and idle states of the point cloud in space. The process of step S1 is as follows: Figure 2 As shown.

[0040] In step S2, the original point cloud is first downsampled to a uniform number using the IFPS (Farthest Point Sampling) algorithm to ensure the density and representativeness of the point cloud while avoiding computational efficiency issues caused by excessively large point clouds. Next, a global scaling operation is performed to unify the size of the point clouds, ensuring that different point clouds have a consistent scale when input into the model. This effectively improves the model's generalization ability, thereby enhancing the accuracy and robustness of subsequent completion processes. This series of preprocessing steps lays a more solid foundation for model training, enabling subsequent processing to more accurately recover the point cloud information of occluded areas.

[0041] In step S3, the geometric center of all point clouds is first calculated, and then the point cloud data is divided into four quadrants (upper left, upper right, lower left, and lower right) using the coordinates of this center point. Next, the number of points in each region is adjusted to reach the target value (512 points). Specifically, for regions with more points than the target value, excess points are removed from the region boundaries; for regions with fewer points than the target value, the nearest points are selected from other regions that do not meet the target size to supplement them, while ensuring that the selected points are not duplicates. Finally, the processing results return four adjusted and balanced point sets for subsequent analysis and model training.

[0042] In step S4, before training the deep network, the point cloud deep learning network PF-Net needs to be improved to build a network framework for dynamic point cloud prediction. First, the input point cloud is downsampled using IFPS to obtain three different resolution scales. For these three different scales, three independent combined multilayer perceptrons (CMLPs) are used to map them into three separate feature vectors. Each CMLP consists of six 2D convolutional layers, capable of encoding each point into multiple dimensions (64-128-256-512-1024). In most previous works, the feature extractor of the encoder is a multilayer perceptron (MLP), which maps each point to different dimensions and extracts the maximum value from the final K dimensions to form a global latent vector. However, it does not fully utilize low-level and mid-level features containing rich local information and is affected by the dimension K of the max-pooling layer. In the CMLP, we also use an MLP to encode each point into multiple dimensions [64-128-256-512-1024]. Unlike traditional MLPs, we perform max pooling on the outputs of the last four layers of the MLP to obtain multi-dimensional feature vectors. All multi-dimensional feature vectors are concatenated to form a combined latent vector F, which contains both low-level and high-level feature information. Subsequently, the latent vectors obtained from point clouds at three different scales are stacked to form a final latent feature map M of size 1920×3. Finally, the MLP is used to map and integrate the latent feature map into the final spatial feature vector V (dimension 1920). For temporal feature extraction, the spatial feature vectors of consecutive frames are used as input to a Long Short-Term Memory (LSTM) network. By adding a temporal dimension, the input shape is constructed as (batch_size, 5, 1920), where the time step is 5, the LSTM has 512 hidden layers, and the last time step is selected as the feature output. Finally, two fully connected layers (FC) are used to map the dimension back to 1920 to form the temporal features. Finally, the spatial feature vector is added to the processed temporal feature vector to generate an enhanced spatiotemporal feature vector, which is used for subsequent point cloud generation tasks.

[0043] The enhanced spatiotemporal feature vector is then decoded using a point pyramid decoder module. This module generates point clouds of the missing regions using information from multiple fully connected layers. First, the input spatiotemporal feature vector is passed through three fully connected layers: FC1, FC2, and FC3. Each feature layer is responsible for predicting point clouds at different resolutions. The principal center point Y... primary The prediction will start from the deepest FC3, which has a size of M1×3. Then, the quadratic center point Y will be predicted using FC2. secondary The relative coordinates of Y. primary Each point in the matrix is ​​used as the center point to generate Y. primary of There are several points. We use expansion and addition operations to implement this process. Therefore, Y secondary The size is M²×3. Y detail This is the final prediction of the dot pyramid encoder. Y detail The generation is similar to Y secondary Y detail Its size is M×3.

[0044] After obtaining the predicted point cloud completion result, we input the predicted point cloud and the actual real point cloud into the discriminator module. The accuracy of the generated completed point cloud is judged by the output of the discriminator. The discriminator is a classification network with a structure similar to a combined multilayer perceptron (CMLP). Specifically, the discriminator consists of cascaded multilayer perceptron (MLP) layers [64-64-128-256], and performs max pooling on the features of the last three layers to extract feature vectors. The sizes of these feature vectors are 64, 128, and 256, respectively. Subsequently, these feature vectors are concatenated to form a latent vector, which is further mapped through a fully connected layer [256, 128, 16, 1] and then predicted by a sigmoid classifier to distinguish the difference between the generated point cloud and the real point cloud. Figure 3 The diagram shown is a network framework diagram for dynamic point cloud completion.

[0045] In step S5, the voxel model represents the respiratory motion state of the body surface at a given moment. By traversing all voxel models across frames, a cuboid bounding box (minimum bounding box) is constructed, accommodating the voxel model of the largest frame. For each voxel model, voxel blocks are extracted from the bounding box using the same traversal order, forming a one-dimensional column vector. This one-dimensional column vector is then reduced to a low-dimensional feature that can characterize the respiratory motion of the body surface using PCA (Principal Component Analysis). This low-dimensional data can describe the changing properties of the voxel model. This method allows point cloud data to fully encompass the three-dimensional changes in the human body during respiration.

[0046] Figure 4This is a visualization of the point cloud predicted by the network and the actual point cloud. From... Figure 4 The comparison between the real point cloud and the completed point cloud shows that the method of this application performs point cloud completion very well at the four corners, obtaining clearer and more detailed point cloud information.

[0047] Figure 5 A comparison image showing the respiratory motion features extracted using PCA dimensionality reduction from predicted and actual point cloud values ​​in four regions. From... Figure 5 As can be seen, the method in this application predicts that the actual points and the completed points in the point cloud are almost identical, showing a very high degree of consistency.

[0048] Figure 6 This is a schematic diagram of the Pearson correlation coefficient between the actual and predicted respiratory movement characteristics. From... Figure 6 As can be seen, in the method of this application, the Pearson correlation coefficients of the respiratory motion characteristics of the true and predicted values ​​are all above 0.9, indicating a high degree of correlation.

[0049] Figure 7 This document presents a diagram illustrating the creation of voxel models of the four arm-occluded regions, the segmentation of the arm region point cloud, and the visualization effect achieved using network completion. From... Figure 7 As can be seen, this application achieves accurate analysis and completion of dynamic point cloud data, solves the problem of misjudgment of respiratory rate, amplitude and pattern caused by data loss in neonatal respiratory monitoring, and improves monitoring accuracy and clinical application value.

[0050] Figure 8 A schematic diagram illustrating respiratory motion features extracted using PCA dimensionality reduction from point clouds predicted by a network for four arm-occluded regions. Low-dimensional data can describe the changing properties of voxel models. This method allows point cloud data to fully encompass the three-dimensional changes in the human body during respiration.

[0051] The application provides a point cloud dynamic completion system for occluded areas. This system is used to execute the point cloud dynamic completion method for occluded areas described in the above embodiments, such as... Figure 9 As shown, the system includes:

[0052] The model building module 901 is used to collect surface point cloud data of the human body in an unobstructed state and surface point cloud data of the human body in an arm-obstructed state during breathing, and to build a voxel model based on the surface point cloud data.

[0053] Preprocessing module 902 is used to preprocess the voxel model, including farthest point sampling and global scaling;

[0054] The region division module 903 is used to calculate the geometric center of all point clouds and divide the point cloud data into four quadrant regions based on the center, simulating the four regions occluded by the arm.

[0055] The dynamic completion module 904 is used to train the network using a normal, unoccluded point cloud dataset. The trained network is then used to dynamically complete the point cloud data of the arm-occluded area. The point cloud data of the occluded area is input into the trained network, and the network outputs the completed point cloud data.

[0056] The point cloud dynamic completion system for occluded parts provided in the above embodiments of this application and the point cloud dynamic completion method for occluded parts provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0057] This application also provides an electronic device corresponding to the dynamic point cloud completion method for occluded areas provided in the foregoing embodiments, for executing the dynamic point cloud completion method for occluded areas. This application does not limit the scope of the embodiments.

[0058] Please refer to Figure 10 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 10 As shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected via the bus 202. The memory 201 stores a computer program that can run on the processor 200. When the processor 200 runs the computer program, it executes the point cloud dynamic completion method for occluded parts provided in any of the foregoing embodiments of this application.

[0059] The memory 201 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 203 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0060] Bus 202 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs. After receiving an execution instruction, the processor 200 executes the program. The dynamic point cloud completion method for occluded areas disclosed in any of the foregoing embodiments of this application can be applied to the processor 200, or implemented by the processor 200.

[0061] The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 200 or by instructions in software form. The processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 201. The processor 200 reads the information in memory 201 and, in conjunction with its hardware, completes the steps of the above method.

[0062] The electronic device provided in this application embodiment and the point cloud dynamic completion method for occluded parts provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0063] This application also provides a computer-readable storage medium corresponding to the point cloud dynamic completion method for occluded areas provided in the foregoing embodiments. Please refer to... Figure 11 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the point cloud dynamic completion method for occluded parts provided in any of the foregoing embodiments.

[0064] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0065] The computer-readable storage medium provided in the above embodiments of this application and the method for dynamic completion of point clouds of occluded parts provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0066] It should be noted that:

[0067] The algorithms and displays provided herein are not inherently related to any pre-existing computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this application is not directed to any pre-existing programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of pre-existing languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0068] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0069] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0070] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0071] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0072] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation system according to the embodiments of this application. This application can also be implemented as a device or system program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0073] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0074] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for dynamic completion of point clouds in occluded areas, characterized in that, include: Step S1: Collect point cloud data of the human body surface in an unobstructed state and in an arm-obstructed state during human respiration, and establish a voxel model based on the point cloud data of the human body surface. Step S2: Preprocess the voxel model, including farthest point sampling and global scaling; Step S3: Calculate the geometric center of all point clouds, and divide the point cloud data into four quadrants based on the center to simulate the four regions occluded by the arm; Step S4: Train the network using a normal, unoccluded point cloud dataset. Use the trained network to dynamically complete the point cloud data of the arm-occluded area. Input the point cloud data of the occluded area into the trained network, and the network outputs the completed point cloud data.

2. The method according to claim 1, characterized in that, The method further includes: Step S5: Use principal component analysis to perform dimensionality reduction on the continuous frame point cloud of the completed position to extract one-dimensional respiratory motion features.

3. The method according to claim 1, characterized in that, Step S1 includes: The coordinates of the two RGB-D depth cameras were unified to the same coordinate system using a calibration board, and point cloud information of the chest and abdomen surface was collected. Use statistical filtering algorithms to remove noise points; By coarsely registering the point cloud, the matching corresponding points in the point clouds of two adjacent frames are calculated using FPFH features. The FPFH features are obtained by calculating the normal vector of each point and the normal vector of the points in the neighborhood. Point pairs are obtained by matching based on the FPFH features, and RANSAC is used to remove erroneous matching points. The rigid body transformation is estimated by matching point pairs, the source point cloud is registered to the target point cloud, and the ICP algorithm is used to finely register the two sets of point cloud information. RGB and boundary threshold segmentation are used to remove background areas and redundant information; The point cloud data is smoothed using the moving least squares algorithm; The processed point cloud information is inserted into the Octomap octree map, and a voxel map is created by defining the occupied and idle states of the point cloud in space.

4. The method according to claim 1, characterized in that, Step S2 includes: The farthest point sampling algorithm is used to downsample the original point cloud to a uniform number. The size of the point cloud is standardized through a global scaling operation.

5. The method according to claim 1, characterized in that, Step S3 includes: Based on the geometric center of all points, the point cloud data is divided into four quadrant regions using the coordinates of the points at the center. The number of points in each region is adjusted to reach the set target value. For regions with more points than the target value, the excess points are removed from the region boundary. For regions with fewer points than the target value, the nearest points are selected from other regions that do not meet the target size to supplement the points, while ensuring that the selected points are not duplicates. Obtain four adjusted and balanced point sets.

6. The method according to claim 1, characterized in that, Step S4 includes: The input point cloud is downsampled using IFPS to obtain point clouds at three different resolution scales. For the point clouds at the three different resolution scales, three independent combined multilayer perceptrons are used to map them into three separate feature vectors. Max pooling is performed on the outputs of the last four layers of the multilayer perceptrons to obtain multidimensional feature vectors representing features at a predetermined resolution. The three separate feature vectors are stacked to form a latent feature map. The latent feature map is then integrated into a spatial feature vector using MLP. The continuous frame spatial feature vectors are used as input to the Long Short-Term Memory (LSTM) network. By adding a time dimension, the input shape is constructed, and the last time step is selected as the feature output. Two fully connected layers are used to map the dimensions to form a time feature vector. The spatial feature vector is added to the temporal feature vector to generate an enhanced spatiotemporal feature vector. The enhanced spatiotemporal feature vector is then decoded using the PPD module, and point clouds of the missing regions are generated using information from multiple fully connected layers.

7. The method according to claim 2, characterized in that, Step S5 includes: The surface information of each frame is modeled as a voxel model, which represents the respiratory motion state of the surface at a certain moment. Iterate through all the voxel models in all frames and construct a cuboid bounding box, i.e., the minimum bounding box, such that the bounding box can contain the largest voxel model in a single frame. For each voxel model, voxel blocks are extracted from the bounding box using the same traversal order to form a one-dimensional column vector; The PCA algorithm is used to reduce the dimensionality of this one-dimensional column vector to a low-dimensional feature that can characterize the respiratory motion features of the body surface.

8. A point cloud dynamic completion system for occluded areas, characterized in that, include: The model building module is used to collect surface point cloud data of the human body in an unobstructed state and surface point cloud data of the human body in an occluded state during breathing, and to build a voxel model based on the surface point cloud data. The preprocessing module is used to preprocess the voxel model, including farthest point sampling and global scaling. The region division module is used to calculate the geometric center of all point clouds and divide the point cloud data into four quadrant regions based on the center, simulating the four regions occluded by the arm. The dynamic completion module is used to train the network using a normal, unoccluded point cloud dataset. The trained network is then used to dynamically complete the point cloud data of the arm-occluded area. The point cloud data of the occluded area is input into the trained network, and the network outputs the completed point cloud data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.

Images