Modeling method, system and equipment for gypsum removal by robot and storage medium

Abstract

The invention relates to the technical field of robots, and particularly provides a modeling method, system and device for removing gypsum by a robot and a storage medium, and the method comprises the steps: obtaining structured light scanning point cloud data of a target gypsum surface; synchronously acquiring reflection spectrum data of the gypsum surface to near-infrared light; constructing an initial three-dimensional voxel model based on the structured light scanning point cloud; on the basis of the reflection spectrum data, through a pre-trained spectrum-density regression model, gypsum material density attributes of all voxels in the three-dimensional voxel model are predicted and marked; detecting the adhesion strength of the gypsum and the skin by utilizing high-frequency ultrasonic waves, and marking an area of which the adhesion strength exceeds a set threshold value in the three-dimensional voxel model as a high-risk area; and utilizing a pre-trained convolutional neural network to identify boundary positions of gypsum and skin based on the three-dimensional voxel model marked with the gypsum material density attribute and the adhesion strength, and fusing boundary information into the three-dimensional voxel model. And the safety, efficiency and comfort of gypsum removal are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robots, and particularly relates to a modeling method, system, device and storage medium for robot gypsum removal. BACKGROUND

[0002] Gypsum fixation in orthopedics is a conventional means for treating diseases such as fractures and dislocations, but the traditional gypsum removal process highly depends on manual operation of an electric gypsum saw by a doctor, and has significant pain points. First, the operation has high risk, the doctor needs to control the depth of the saw blade by experience, and it is easy to injure the skin of the patient, especially for children, the elderly or people with sensory disorders. Second, the noise and vibration generated by the traditional tool can cause strong discomfort and fear of the patient. In addition, the removal efficiency is low. At present, the existing technologies mainly focus on local improvement of the gypsum material itself or manual tools, and still belong to the blank in the field of automatic and intelligent removal equipment. With the development of artificial intelligence and robot technology, how to design a system capable of automatically, accurately and safely planning and executing a removal path has become a technical problem to be solved in the field. SUMMARY

[0003] In view of the above deficiencies of the prior art, the application provides a modeling method, system, device and storage medium for robot gypsum removal to solve the above technical problems.

[0004] In a first aspect, the application provides a modeling method for robot gypsum removal, comprising: obtaining structural light scanning point cloud and near-infrared spectrum data of target gypsum; constructing an initial three-dimensional voxel model based on the structural light scanning point cloud; based on the reflection spectrum data, predicting and labeling the gypsum material density attribute of each voxel in the three-dimensional voxel model through a pre-trained spectrum-density regression model; using high-frequency ultrasonic waves to detect the adhesion strength of gypsum and skin, and labeling the area with adhesion strength exceeding a set threshold as a high-risk area in the three-dimensional voxel model; using a pre-trained convolutional neural network to identify the boundary position of gypsum and skin based on the three-dimensional voxel model labeled with the gypsum material density attribute and the adhesion strength, and fusing the boundary information into the three-dimensional voxel model.

[0005] In an optional embodiment, obtaining structural light scanning point cloud and near-infrared spectrum data of target gypsum comprises: obtaining three-dimensional point cloud data of the surface of the gypsum through structural light scanning; obtaining density distribution data of the gypsum material through near-infrared spectrum scanning; spatiotemporally registering the three-dimensional point cloud data and the density distribution data to form a fusion data source.

[0006] In an optional implementation, the three-dimensional point cloud data and density distribution data are spatiotemporally registered, including: The structured light scanning and near-infrared spectral scanning are synchronously triggered and acquired by a sensor group integrated inside a housing; The optical path of the sensor group is pre-calibrated using a built-in calibration plate to obtain a fixed transformation matrix between the coordinate system of the structured light camera and the coordinate system of the near-infrared spectrometer. The fixed transformation matrix is ​​used to map each near-infrared spectral data point to the corresponding three-dimensional coordinates in the structured light point cloud.

[0007] In an optional implementation, based on the structured light scanning point cloud and near-infrared spectral data, an initial three-dimensional voxel model is constructed, and the material properties of the plaster are labeled, including: The iterative nearest point algorithm is used to register the structured light scanning point cloud with the spectral point cloud data in three-dimensional coordinates, so that the two are precisely aligned in space. Based on the spatial extent of the registered point cloud, a regular three-dimensional voxel mesh is generated; For each voxel occupied by plaster, the registered spectral point cloud data is searched in its neighborhood, the spectral information is fused by the inverse distance weighted average algorithm, and then input into a pre-trained density prediction model to obtain the material density value of the voxel. Based on the material density value, label each voxel with a density level.

[0008] In an optional implementation, high-frequency ultrasound is used to detect the adhesion strength between plaster and skin, and areas where the adhesion strength exceeds a set threshold are marked as high-risk areas in the three-dimensional voxel model, including: The ultrasonic echo at the interface between plaster and skin is acquired by a high-frequency ultrasonic probe, and the original electrical signal after the echo is converted is filtered and noise-reduced to obtain the ultrasonic detection signal. Cross-correlation calculations were performed on two consecutive frames of ultrasonic detection signals to obtain the micro-displacement data Δx of the interface tissue. Based on the micro-displacement data, the strain field distribution ε in this region is calculated using the least squares strain estimation algorithm; Based on the pre-calibrated Young's modulus E of gypsum, and using the simplified model based on Hooke's law, the strain field ε is converted into an adhesion force distribution map. The areas with adhesion force values ​​greater than 0.3 N / cm² in the adhesion force distribution map are marked as high-risk areas in the three-dimensional voxel model, and a safety buffer zone with an extension of 1.5 times the original safety distance is automatically generated for the high-risk areas.

[0009] In one alternative implementation, the convolutional neural network is an improved 3D U-Net architecture; The network enhances feature extraction of boundaries through skip connections with an attention mechanism and is trained using a loss function that combines Dice Loss and Focal Loss to accurately segment plaster, skin and bony prominences simultaneously. The output of the network is used to identify the boundary between plaster and skin and the coordinates of bony prominences, and to mark the identified bony prominences as absolute no-go zones in the three-dimensional voxel model, while generating a safety buffer of a preset width around them. The boundary and restricted area information is fused into the three-dimensional voxel model as attribute labels.

[0010] In an optional implementation, it further includes: An ArgMax operation is performed on the multi-channel probability map output by the convolutional neural network to generate a single-channel 3D classification label map, in which each voxel is assigned a classification label representing plaster, skin, or bony prominence. Based on the three-dimensional classification label map, the Marching Cubes algorithm is used to extract the interface between two types of voxels with the label value of plaster and the label value of skin, and generate a three-dimensional triangular mesh model representing the plaster-skin boundary. Extract all voxels with the label value of bony prominence from the three-dimensional classification label map, perform three-dimensional connected component analysis to distinguish different bony prominence regions, and perform a morphological expansion operation with a preset width using spherical structural elements on each independent bony prominence region. In the three-dimensional voxel model, uniformly label the voxels covered by the original bony prominence region and the expanded region as absolute no-go zones. The geometric data of the three-dimensional triangular mesh model and the voxel index information of the absolute restricted area are fused into the three-dimensional voxel model as attribute data.

[0011] Secondly, the present invention provides a modeling system for robot plaster removal, comprising: The data acquisition module is used to acquire structured light scanning point cloud data of the target gypsum surface; and simultaneously acquire the reflectance spectrum data of the gypsum surface to near-infrared light. The model building module is used to build an initial three-dimensional voxel model based on the structured light scanning point cloud; The first labeling module is used to predict and label the gypsum material density properties of each voxel in the three-dimensional voxel model based on the reflectance spectral data and through a pre-trained spectral-density regression model. The second marking module is used to detect the adhesion strength between plaster and skin using high-frequency ultrasound, and to mark areas where the adhesion strength exceeds a set threshold as high-risk areas in the three-dimensional voxel model. A boundary generation module is used to identify the boundary position between plaster and skin based on a three-dimensional voxel model labeled with plaster material density properties and adhesion strength using a pre-trained convolutional neural network, and to fuse the boundary information into the three-dimensional voxel model.

[0012] Thirdly, a device is provided, comprising: The memory is used to store the modeling program for the robot to remove the plaster cast; A processor is configured to implement the steps of the modeling method for removing plaster casts as provided in the first aspect when executing the modeling program for removing plaster casts by the robot.

[0013] Fourthly, a computer-readable storage medium is provided, on which a modeling program for robot plaster removal is stored, wherein when the modeling program for robot plaster removal is executed by a processor, the steps of the modeling method for robot plaster removal provided in the first aspect are implemented.

[0014] The beneficial effects of this invention are as follows: the modeling method, system, equipment, and storage medium for robotic plaster cast removal provided by this invention significantly improve the safety, efficiency, and comfort of plaster cast removal through the integration of multimodal sensing and artificial intelligence algorithms. Its beneficial effects are: First, the path planning using reinforcement learning can accurately avoid high-risk areas and physiologically restricted areas, fundamentally avoiding the risk of accidental skin injury and greatly ensuring operational safety. Second, the automated removal process reduces the operation time from tens of minutes in traditional manual methods to within a few minutes, and effectively suppresses noise and dust, greatly reducing the patient's psychological fear and discomfort. Finally, this intelligent method reduces reliance on the experience of senior physicians, optimizes the allocation of medical resources, and achieves standardized, traceable, and precise medical operations. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0017] Figure 2 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0021] The modeling method for robot plaster removal provided in this embodiment of the invention is executed by a computer device, and correspondingly, the modeling system for robot plaster removal runs in the computer device.

[0022] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The executing entity can be a modeling system for removing plaster casts using a robot. Depending on the specific requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0023] like Figure 1 As shown, the method includes: S1. Acquire structured light scanning point cloud and near-infrared spectral data of the target gypsum; S2. Construct an initial three-dimensional voxel model based on the structured light scanning point cloud; S3. Based on the reflectance spectral data, predict and label the gypsum material density properties of each voxel in the three-dimensional voxel model using a pre-trained spectral-density regression model; S4. Use high-frequency ultrasound to detect the adhesion strength between plaster and skin, and mark areas where the adhesion strength exceeds a set threshold as high-risk areas in the three-dimensional voxel model; S5. Using a pre-trained convolutional neural network, the boundary position between plaster and skin is identified based on a three-dimensional voxel model labeled with the density properties of plaster material and the adhesion strength, and the boundary information is fused into the three-dimensional voxel model.

[0024] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0025] The data acquisition and fusion described in step S1 is achieved through a highly integrated multimodal sensing system. This system integrates a structured light projector, an industrial camera, and a near-infrared spectrometer within its housing, forming a collaborative sensor group. The structured light projector uses an 850nm wavelength laser light source with a scanning accuracy better than 0.05mm; the near-infrared spectrometer operates in the 900-1700nm band and is used to capture the density distribution characteristics of gypsum material.

[0026] During implementation, system calibration is performed first: a built-in high-precision checkerboard calibration board is used to calibrate the intrinsic and extrinsic parameters of the structured light camera and near-infrared spectrometer, respectively, and a fixed transformation matrix T between their coordinate systems is calculated using a point cloud registration algorithm. Subsequently, during data acquisition, the main controller synchronously triggers structured light scanning and spectral acquisition to ensure data temporal consistency. After acquisition, the transformation matrix T is used to convert each spectral data point into three-dimensional coordinates in the structured light camera coordinate system based on its pixel coordinates, thereby achieving precise spatial alignment between spectral data and three-dimensional point cloud data. This generates a fused data source that simultaneously contains geometric information and material properties, providing a data foundation for the subsequent construction of a three-dimensional voxel model.

[0027] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0028] S201. The iterative nearest point algorithm is used to register the structured light scanning point cloud with the spectral point cloud data in three-dimensional coordinates, so that the two are precisely aligned in space.

[0029] Although a preliminary coordinate system was established through hardware calibration during the data acquisition phase, an Iterative Closest Point (ICP) algorithm was employed to automatically and precisely register the high-precision geometric point cloud (denoted as P_geo) obtained by structured light scanning with the spectral point cloud (denoted as P_spec) mapped to the same coordinate system, in order to eliminate any potential minor errors. Specifically, the algorithm iteratively searches for the nearest neighbor in P_geo for each point in P_spec, and calculates the optimal rigid body transformation (rotation matrix R and translation vector T) using singular value decomposition (SVD) to minimize the overall mean square error (MSE) between the two point clouds. The iteration stops when the error is less than a set threshold (e.g., 0.05 mm) or when the maximum number of iterations is reached, ultimately outputting a precisely aligned fused point cloud dataset.

[0030] S202. Generate a regular three-dimensional voxel mesh based on the spatial range of the registered point cloud.

[0031] Using the spatial bounding box of the fused point cloud obtained in the previous step as a reference, a regular three-dimensional voxel mesh is created. The voxel size is set according to the system accuracy requirements; in this embodiment, it is preferably 0.1mm × 0.1mm × 0.1mm. Each voxel is uniquely identified by its three-dimensional index, and its attribute values ​​are initialized to empty.

[0032] S203. For each voxel occupied by plaster, search for registered spectral point cloud data in its neighborhood, fuse the spectral information using an inverse distance weighted average algorithm, and input it into a pre-trained density prediction model to obtain the material density value of the voxel.

[0033] For each voxel in the voxel grid, if it is located inside the plaster model defined by P_geo, an interpolation operation is performed. A spherical search domain with a radius of 0.2 mm is established around the voxel, and all spectral data points from P_spec within the domain are obtained. The spectral information of these points is fused using the inverse distance weighting (IDW) method, with points closer to the center of the voxel having a higher weight (the weight is calculated using the formula w_i = 1 / d_i^2). The fused spectral feature vector is then input into a pre-trained density prediction model. In this embodiment, the model uses a support vector regression (SVR) machine, which has been trained based on a large amount of plaster sample spectral data with known density values, and can output the plaster material density value (unit: g / cm³) corresponding to the current voxel with high accuracy.

[0034] S204. Label each voxel with a density level based on the material density value.

[0035] Based on pre-determined clinical thresholds through experiments, each voxel with a density value is labeled with a classification tag. The specific rules are as follows: if the density value is below the threshold T_low (e.g., 1.2 g / cm³), it is labeled as a "low-density area"; if it is above the threshold T_high (e.g., 1.8 g / cm³), it is labeled as a "high-density area"; and values ​​in between are labeled as "normal-density areas." At this point, a three-dimensional voxel model possessing both geometric information and physical material properties, suitable for intelligent path planning, is complete.

[0036] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0037] S301. A continuous frame radio frequency signal of the plaster-skin interface is acquired by a high-frequency ultrasound probe, and the ultrasound detection signal is filtered and noise-reduced.

[0038] A high-frequency linear ultrasonic probe (recommended center frequency 10-15MHz) integrated into the end effector of a robotic arm is positioned close to the plaster surface and moves along a preset path. The probe emits ultrasonic waves at a rate of thousands of frames per second and receives the returned radio frequency (RF) signals. Since the raw ultrasonic detection signal is susceptible to electronic noise and acoustic artifacts, it requires immediate filtering. This embodiment employs wavelet denoising as the preferred method, using Symlets wavelet basis functions to decompose the signal and filtering out high-frequency noise through soft thresholding while retaining the true signal components reflecting tissue micro-displacement, providing a high signal-to-noise ratio data foundation for subsequent analysis.

[0039] S302. Perform cross-correlation calculation on two consecutive frames of ultrasonic detection signals to obtain the micro-displacement data Δx of the interface tissue.

[0040] The principle of elastography is to infer the mechanical properties of tissue by calculating the deformation of the tissue under slight pressure. The system extracts two consecutive frames (time interval Δt is typically 0.1-1 ms) of preprocessed ultrasound detection signals. For the data segments corresponding to the same scan line in these two frames, a normalized cross-correlation algorithm is used for point-by-point calculation. This algorithm uses a sliding time window (typically 1-2 wavelengths long, with a step size of half a wavelength) to find the time shift Δτ corresponding to the maximum correlation coefficient between the two signal segments. This time shift Δτ is multiplied by the propagation speed c of the ultrasound in the tissue to obtain the axial micro-displacement Δx at that point (Δx = c * Δτ / 2). By traversing all scan lines and all depth points, the two-dimensional micro-displacement field Δx(x,z) of the entire region of interest is finally obtained.

[0041] S303. Based on the micro-displacement data, the strain field distribution ε in the region is calculated using the least squares strain estimation algorithm.

[0042] The micro-displacement field itself still contains noise, and direct differential calculation of strain will amplify the error. Therefore, this embodiment uses the least squares strain estimation method to obtain a smooth and reliable strain distribution ε. The basic principle of this method is as follows: within a small spatial window (e.g., 5 points in the axial depth direction and 3 scan lines in the transverse direction), it is assumed that the micro-displacement Δx changes linearly with the depth z, i.e., Δx(z) = a * z + b. By fitting all Δx data points within this window using the least squares method, the slope a is obtained. This slope a is the axial strain εzz at the center point of the window. By sliding this window along the depth and transverse directions, traversing the entire micro-displacement field, a high-precision two-dimensional strain field distribution map ε(x,z) is finally generated.

[0043] S304. Based on the pre-calibrated Young's modulus E of gypsum, the strain field ε is converted into an adhesion force distribution map according to the simplified model based on Hooke's law.

[0044] Strain ε reflects the relative deformation of the tissue and needs to be converted into absolute adhesion force. According to a simplified model of Hooke's Law, within the elastic range, stress σ is proportional to strain ε (σ = E * ε), and the adhesion force F at the plaster-skin interface can be approximated by this stress. Therefore, adhesion force F = E * ε * A, where A is the unit area. To simplify calculations and provide a visual representation, this embodiment outputs an adhesion force distribution map in N / cm². The key is the Young's modulus E of the plaster, which is obtained through pre-calibration: in a laboratory environment, standard compression tests are performed on plaster samples from the same batch using a materials testing machine, and their stress-strain curves are measured. The slope of the linear segment is the Young's modulus E (typical range 1-5 GPa). The system stores this calibration value E as a known constant in the database for real-time calculation.

[0045] S305. The areas with adhesion force values ​​greater than 0.3 N / cm² in the adhesion force distribution map are marked as high-risk areas in the three-dimensional voxel model, and a safety buffer zone with an extension of 1.5 times the original safety distance is automatically generated for the high-risk areas.

[0046] Each pixel value in the calculated adhesion force distribution map is compared with a clinical safety threshold (set to 0.3 N / cm² in this embodiment, verified through extensive experiments). Any area with an adhesion force value exceeding this threshold is identified as a "high-risk area." The coordinates of these areas are mapped onto the previously constructed 3D voxel model and assigned a specific high-risk label. To further ensure safety, the system automatically generates a safety buffer zone around the contour of each high-risk area. The expansion distance of this buffer zone is calculated based on the original system's baseline safety distance (e.g., 2 mm), with an expansion factor of 1.5 times, meaning the final safety distance requirement for this area is increased to 3 mm. In subsequent path planning algorithms, the robotic arm's end-effector cutting tool must not only avoid high-risk areas but also strictly avoid this expanded safety buffer zone, thereby achieving ultimate protection for the patient's skin. All annotations and buffer zone information are highlighted in a prominent color (e.g., red) in the system's 3D visualization interface.

[0047] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0048] First, the network input data is prepared. The system uses the three-dimensional voxel mesh generated in the previous steps as a base, and splices its geometric information (depth values ​​from structured light scanning) and physical property information (reflectance data from near-infrared spectroscopy and adhesion strength between plaster and skin detected by high-frequency ultrasound) along the channel dimension to form a multi-channel 3D data block, which serves as the network input tensor. The size of this tensor is [D, H, W, C], where D, H, and W represent depth, height, and width, respectively, and C is the number of channels (C≥2 in this embodiment).

[0049] The network employs a specially modified 3D U-Net architecture. Its encoder consists of four downsampling stages, each containing two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function, followed by a max-pooling layer with a stride of 2 for downsampling. The decoder then performs upsampling through transposed convolutions, progressively restoring spatial resolution. The core improvement of this design lies in the introduction of an attention gate mechanism in the skip connection between the encoder and decoder. This mechanism automatically learns and focuses on key regions such as plaster-skin boundaries and bony prominences, suppressing feature propagation from irrelevant background regions, thereby significantly improving the accuracy and clarity of boundary segmentation.

[0050] To address class imbalance (e.g., the extremely small pixel proportion of bony prominences) and boundary optimization issues during training, this embodiment designs a composite loss function. This function is a weighted sum of Dice Loss and Focal Loss (L_total = α * L_dice + β * L_focal). Dice Loss directly optimizes the overlap of segmented regions, particularly beneficial for accurate boundary delineation; Focal Loss, through a modulation factor, increases attention to hard-to-classify samples (such as pixels in bony prominences), mitigating the negative impact of class imbalance. The network uses the AdamW optimizer for pre-training and fine-tuning on a dataset containing a large amount of clinical CT / MRI and simultaneous optical scanning data.

[0051] During forward inference, the input multi-channel voxel data is processed by the network to output a probability map with three channels, corresponding to the pixel-level classification probabilities of "plaster," "skin," and "bone protrusion." The final segmentation label map is obtained through ArgMax operations. The system then performs post-processing on this output: First, the contour at the intersection of the "skin" and "plaster" probabilities is extracted as a high-precision plaster-skin boundary for subsequent safety distance calculations. More importantly, the system automatically identifies all connected regions classified as "bone protrusion" and marks them as "absolute no-go zones" in the 3D voxel model. To ensure absolute safety, the system also automatically generates a 3D safety buffer zone of a preset width (usually set to 2.0 mm) around each absolute no-go zone. Finally, all this generated information—boundary lines, no-go zones, and buffer zones—is seamlessly integrated into the main 3D voxel model as a new attribute label layer, providing crucial and absolutely reliable safety constraint information for the reinforcement learning path planning module.

[0052] In some embodiments, the modeling system for robot-assisted plaster removal may include multiple functional modules composed of computer program segments. The computer programs for each segment of the modeling system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) The robot has the function of modeling the removal of plaster casts.

[0053] In this embodiment, the modeling system for removing plaster casts by a robot can be divided into multiple functional modules based on the functions it performs, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0054] The data acquisition module acquires structured light scanning point cloud data of the target gypsum surface; and simultaneously acquires the reflectance spectrum data of the gypsum surface to near-infrared light. The model building module constructs an initial three-dimensional voxel model based on the structured light scanning point cloud; The first labeling module is used to predict and label the gypsum material density properties of each voxel in the three-dimensional voxel model based on the reflectance spectral data and through a pre-trained spectral-density regression model. The second marking module is used to detect the adhesion strength between plaster and skin using high-frequency ultrasound, and to mark areas where the adhesion strength exceeds a set threshold as high-risk areas in the three-dimensional voxel model. A boundary generation module is used to identify the boundary position between plaster and skin based on a three-dimensional voxel model labeled with plaster material density properties and adhesion strength using a pre-trained convolutional neural network, and to fuse the boundary information into the three-dimensional voxel model.

[0055] Figure 3 The modeling method for removing plaster casts using a robot, provided in the embodiments of this application, can be applied to devices. Those skilled in the art will understand that the device structures involved in the embodiments of this invention do not constitute a limitation on the device. A device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0056] The device 300 may include a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] The memory 320 can be used to store execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the device 300 is able to perform some or all of the steps in the above method embodiments.

[0058] The processor 310 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0059] The communication unit 330 is used to establish a communication channel, enabling the storage device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0060] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0061] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code. It includes several instructions to cause a computer device (which may be a personal computer, a server, or a second device, network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0062] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0063] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0064] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0066] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A modeling method for removing plaster casts using a robot, characterized in that, include: Acquire structured light scanning point cloud data of the target gypsum surface; Simultaneously acquire the reflectance spectrum data of the plaster surface to near-infrared light; An initial three-dimensional voxel model is constructed based on the structured light scanning point cloud; Based on the reflectance spectral data, the gypsum material density properties of each voxel in the three-dimensional voxel model are predicted and labeled using a pre-trained spectral-density regression model. High-frequency ultrasound was used to detect the adhesion strength between plaster and skin, and areas where the adhesion strength exceeded a set threshold were marked as high-risk areas in the three-dimensional voxel model. A pre-trained convolutional neural network is used to identify the boundary position between plaster and skin based on a three-dimensional voxel model labeled with the density properties of plaster material and the adhesion strength, and the boundary information is fused into the three-dimensional voxel model.

2. The method according to claim 1, characterized in that, Obtain structured light scanning point cloud and near-infrared spectral data of the target plaster, including: Three-dimensional point cloud data of the gypsum surface were obtained by structured light scanning; Density distribution data of gypsum material were obtained by near-infrared spectroscopy scanning; The three-dimensional point cloud data and density distribution data are spatiotemporally registered to form a fused data source.

3. The method according to claim 2, characterized in that, The spatiotemporal registration of the three-dimensional point cloud data and density distribution data includes: The structured light scanning and near-infrared spectral scanning are synchronously triggered and acquired by a sensor group integrated inside a housing; The optical path of the sensor group is pre-calibrated using a built-in calibration plate to obtain a fixed transformation matrix between the coordinate system of the structured light camera and the coordinate system of the near-infrared spectrometer. The fixed transformation matrix is ​​used to map each near-infrared spectral data point to the corresponding three-dimensional coordinates in the structured light point cloud.

4. The method according to claim 3, characterized in that, Based on the structured light scanning point cloud and near-infrared spectral data, an initial three-dimensional voxel model was constructed, and the material properties of the plaster were labeled, including: The iterative nearest point algorithm is used to register the structured light scanning point cloud with the spectral point cloud data in three-dimensional coordinates, so that the two are precisely aligned in space. Based on the spatial extent of the registered point cloud, a regular three-dimensional voxel mesh is generated; For each voxel occupied by plaster, the registered spectral point cloud data is searched in its neighborhood, the spectral information is fused by the inverse distance weighted average algorithm, and then input into a pre-trained density prediction model to obtain the material density value of the voxel. Based on the material density value, label each voxel with a density level.

5. The method according to claim 1, characterized in that, High-frequency ultrasound was used to detect the adhesion strength between plaster and skin. Areas with adhesion strength exceeding a set threshold were marked as high-risk areas in the three-dimensional voxel model, including: The ultrasonic echo at the interface between plaster and skin is acquired by a high-frequency ultrasonic probe, and the original electrical signal after the echo is converted is filtered and noise-reduced to obtain the ultrasonic detection signal. Cross-correlation calculations were performed on two consecutive frames of ultrasonic detection signals to obtain the micro-displacement data Δx of the interface tissue. Based on the micro-displacement data, the strain field distribution ε in this region is calculated using the least squares strain estimation algorithm; Based on the pre-calibrated Young's modulus E of gypsum, and using the simplified model based on Hooke's law, the strain field ε is converted into an adhesion force distribution map. The areas with adhesion force values ​​greater than 0.3 N / cm² in the adhesion force distribution map are marked as high-risk areas in the three-dimensional voxel model, and a safety buffer zone with an extension of 1.5 times the original safety distance is automatically generated for the high-risk areas.

6. The method according to claim 1, characterized in that, The convolutional neural network is an improved 3D U-Net architecture; The network enhances feature extraction of boundaries through skip connections with an attention mechanism and is trained using a loss function that combines DiceLoss and Focal Loss to accurately segment plaster, skin and bony prominences simultaneously. The output of the network is used to identify the boundary between plaster and skin and the coordinates of bony prominences, and to mark the identified bony prominences as absolute no-go zones in the three-dimensional voxel model, while generating a safety buffer of a preset width around them. The boundary and restricted area information is fused into the three-dimensional voxel model as attribute labels.

7. The method according to claim 6, characterized in that, The method further includes: An ArgMax operation is performed on the multi-channel probability map output by the convolutional neural network to generate a single-channel 3D classification label map, in which each voxel is assigned a classification label representing plaster, skin, or bony prominence. Based on the three-dimensional classification label map, the Marching Cubes algorithm is used to extract the interface between two types of voxels with the label value of plaster and the label value of skin, and generate a three-dimensional triangular mesh model representing the plaster-skin boundary. Extract all voxels with the label value of bony prominence from the three-dimensional classification label map, perform three-dimensional connected component analysis to distinguish different bony prominence regions, and perform a morphological expansion operation with a preset width using spherical structural elements on each independent bony prominence region. In the three-dimensional voxel model, uniformly label the voxels covered by the original bony prominence region and the expanded region as absolute no-go zones. The geometric data of the three-dimensional triangular mesh model and the voxel index information of the absolute restricted area are fused into the three-dimensional voxel model as attribute data.

8. A modeling system for robot-assisted plaster removal, characterized in that, include: The data acquisition module acquires structured light scanning point cloud data of the target gypsum surface; Simultaneously acquire the reflectance spectrum data of the plaster surface to near-infrared light; The model building module constructs an initial three-dimensional voxel model based on the structured light scanning point cloud; The first labeling module is used to predict and label the gypsum material density properties of each voxel in the three-dimensional voxel model based on the reflectance spectral data and through a pre-trained spectral-density regression model. The second marking module is used to detect the adhesion strength between plaster and skin using high-frequency ultrasound, and to mark areas where the adhesion strength exceeds a set threshold as high-risk areas in the three-dimensional voxel model. A boundary generation module is used to identify the boundary position between plaster and skin based on a three-dimensional voxel model labeled with plaster material density properties and adhesion strength using a pre-trained convolutional neural network, and to fuse the boundary information into the three-dimensional voxel model.

9. A modeling device for removing plaster casts by a robot, characterized in that, include: The memory is used to store the modeling program for the robot to remove the plaster cast; A processor, configured to implement the steps of the modeling method for removing plaster casts by a robot as described in any one of claims 1-7 when executing the modeling program for removing plaster casts by a robot.

10. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores a modeling program for robot plaster removal, which, when executed by a processor, implements the steps of the modeling method for robot plaster removal as described in any one of claims 1-7.

Images