168 results about "3d segmentation" patented technology

The invention belongs to the technical field of three-dimensional scene understanding, and particularly relates to a training method and device for a three-dimensional open vocabulary semantic segmentation model. The training method comprises the steps of obtaining multi-view RGB-D images of a target area, performing multi-stage reasoning on each image through a visual language model, generating a target vocabulary list, prompting a two-dimensional segmentation model to establish a pixel-level text label, performing depth mapping on the images to generate a first point cloud, and generating a second point cloud; mapping the text tag to the first point cloud to generate a point-by-point text tag; pre-training a neural network model with a sparse encoder-decoder structure by taking the point-by-point text label as a supervision signal, and generating a three-dimensional segmentation model on the first point cloud; and for the second point cloud of the complete scene of the target area, matching point feature embedding and text embedding with the highest similarity in the shared vision-language feature space, generating a credible point-text tag pair, and finely adjusting the three-dimensional segmentation model based on the credible point-text tag pair.

The invention relates to the technical field of medical image processing, in particular to a lung tumor CT image 3D segmentation method and system based on multi-modal image fusion. The method comprises the following steps: acquiring a lung tumor CT image; determining a first texture feature based on the lung tumor CT image; identifying a lung tumor boundary by using the first texture feature; detecting a nodule protrusion area in the boundary of the lung tumor; obtaining tissue infiltration data from the nodule protrusion area; determining a second texture feature according to the tissue infiltration data; determining a tumor heterogeneity feature according to the first texture feature and the second texture feature; evaluating the potential malignancy degree by utilizing tumor heterogeneity characteristics; and dividing a tumor risk area of the lung tumor CT image based on the potential malignancy degree. According to the invention, accurate heterogeneity identification and risk region division of the lung tumor CT image are realized based on a medical image processing technology, and the accuracy of lung tumor 3D segmentation is improved.

The invention discloses an object three-dimensional reconstruction method, device and system based on deep learning. The reconstruction method comprises the following steps: acquiring a multi-view color image of an object through a controllable image acquisition device; reconstructing a sparse three-dimensional point cloud by using a motion recovery structure method and obtaining a camera pose; initializing parameters of the three-dimensional Gaussian sputtering model based on the sparse point cloud and performing training optimization; a target object semantic segmentation data set is constructed, and a low-rank adaptive technology is adopted to finely segment all models; generating prompts through an open vocabulary detection model at each view angle, obtaining an accurate segmentation mask, and optimizing a three-dimensional segmentation weight by adopting a joint loss function fusing color consistency loss and edge perception loss; and finally outputting the color three-dimensional point cloud of the target object. According to the method, the original image is segmented, so that the influence of the quality of the rendered image is avoided; the segmentation precision of the model in a specific scene is improved through field adaptive fine tuning; and the accuracy of the segmentation boundary is ensured by adopting a double-loss joint optimization mechanism.

The invention relates to the technical field of concrete structure damage detection, in particular to a post-fire RC structure burst damage three-dimensional segmentation method, which comprises the following steps: firstly, determining the type and three-dimensional characteristics of post-fire RC structure member surface concrete burst damage, and constructing a corresponding three-dimensional point cloud data set for network training and verification; and then based on a KP-FCNN network structure, a KPConv layer is improved and optimized, so that the detection and segmentation precision is improved, the model size is reduced, the reasoning time is remarkably shortened, the optimal segmentation precision of 82.3% is achieved under the working conditions of different damage degrees, automatic damage segmentation of the burst damage of the concrete structure after the fire disaster is realized, and the automatic damage segmentation of the burst damage of the concrete structure after the fire disaster is realized. And technical support is provided for subsequent damage three-dimensional quantification and deployment to an unmanned aerial vehicle system.

The invention discloses a 3D segmentation method and system based on bidirectional fusion, and the method comprises the steps: obtaining the image information and camera information of a fixed scene; then, the image information and the camera information pass through a three-dimensional image segmentation model, and a preliminary segmentation result of a three-dimensional space is obtained; and finally, based on the preliminary segmentation result, carrying out bidirectional fusion on two adjacent frames of point cloud images to obtain an optimized segmentation result, then obtaining a geometric information-based segmentation result of the scene, and carrying out bidirectional fusion on the point cloud image based on the geometric information segmentation result and the point cloud image based on the optimized segmentation result to obtain a final segmentation result. According to the method, the complexity of the outdoor scene and the difference of the sparse degrees of the point clouds are fully considered, the problem that segmentation masks of front and back frames of a long-distance object in an outdoor environment are inconsistent is solved, the segmentation consistency of an outdoor target is enhanced, the applicability to a large-range and complex scene is improved, and the method is suitable for large-scale and complex scenes. And the feasibility of the SAM3D-based three-dimensional segmentation method in practical engineering application is improved.

A head mounted display (HMD) device is provided. The HMD includes a display, a gaze sensor configured to detect a gaze direction of a user of the HMD device, a camera, a position sensor configured to detect a position of the HMD device, communication circuitry, memory storing one or more computer programs, and one or more processors communicatively coupled to the display, the gaze sensor, the camera, the position sensor, the communication circuitry, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the HMD device to, based on the position of the HMD device, acquire three-dimensional (3D) segmentation information for a space corresponding to the position of the HMD device, acquire gaze point-based interest information corresponding to a profile of the user, the gaze point-based interest information including interest information generated based on gaze points of a plurality of users, display, on the display, a first visual effect for a plurality of regions in the space based on the 3D segmentation information and the interest information, and display, on the display, a second visual effect for a plurality of objects in a first region among the plurality of regions based on the 3D segmentation information and the interest information when the position of the HMD device corresponds to the first region.

A computer device performs feature extraction on two-dimensional medical images included in a three-dimensional medical image, to obtain image features corresponding to the two-dimensional medical images. The three-dimensional medical image are obtained by continuously scanning a target tissue structure. The computer device determines offsets of the two-dimensional medical images in a target direction based on the image features. The computer device performs feature alignment on the image features based on the offsets, to obtain aligned image features. The computer device performs three-dimensional segmentation on the three-dimensional medical image based on the aligned image features, to obtain three-dimensional layer distribution of the target tissue structure in the three-dimensional medical image.

The invention discloses a multi-axis RWKV-UNet + + multi-mode MRI (Magnetic Resonance Imaging) brain tumor segmentation method, and belongs to the technical field of medical image processing. According to the invention, multi-modal MRI three-dimensional body data is input and preprocessed, and fusion features are output through a modal fusion module; the fusion features are input into an encoder containing multi-axis RWKV sequence modeling, and long-range dependence is extracted; after the output of the encoder is processed by the bottleneck layer, the global Token aggregator converges the global context and reinjects the global context; the enhanced features are input into a UNet + + nested topology decoder, the jump features are fused with the up-sampling features after being subjected to jump RWKV semantic alignment, and finally a three-dimensional segmentation probability graph is generated through mapping. The method is mainly used for accurate three-dimensional segmentation of the multi-mode MRI brain tumor, and provides support for clinical brain tumor diagnosis and treatment.

The invention discloses a brain tumor MRI image segmentation method based on a soft clustering KAN network, and belongs to the technical field of medical information processing, and the method comprises the steps: carrying out the voxel alignment and intensity normalization of a multi-modal three-dimensional brain tumor MRI image, and constructing a unified input sample; layer-by-layer downsampling and multi-scale feature extraction are realized through a KAN residual encoder formed by alternately cascading double-layer KAN attention subnets and three-dimensional maximum pooling layers; three-dimensional position coding is added to the highest-layer features, and bottleneck features containing global structure priori are obtained through soft K-means clustering bottleneck layer modeling; fusing the jump connection feature and the up-sampling feature by using an attention gating mechanism to complete decoding; and through soft clustering regularization and a multi-scale depth supervision constraint optimization model, finally through fusion of a main segmentation branch and a refined branch, outputting a multi-subarea three-dimensional segmentation result of the whole tumor, the tumor core and the enhanced tumor.

The invention discloses a CBCT (Cone Beam Computed Tomography) image distance determination method and device for maxillary sinus and maxillary posterior teeth, a medium and a product, and relates to the field of image processing, the method comprises the following steps: acquiring CBCT images of maxillary sinus and maxillary posterior teeth; according to the CBCT images of the maxillary sinus and the maxillary posterior teeth, performing three-dimensional segmentation of the maxillary sinus and the maxillary posterior teeth by adopting a structure segmentation model; the structure segmentation model is constructed based on a deep learning network; according to a three-dimensional segmentation result, generating a surface mesh model based on a marching cube algorithm; determining point cloud data based on a point cloud generation algorithm according to the surface mesh model; according to the point cloud data, the nearest distance between the maxillary sinus and the maxillary posterior teeth and the corresponding nearest apical point of the maxillary posterior teeth are determined; and determining the position relationship between the maxillary sinus and the maxillary posterior teeth according to the nearest distance, and classifying the position relationship. The distance between the maxillary sinus and the maxillary posterior tooth can be efficiently and accurately measured, and the position relation can be judged.

The invention discloses a three-dimensional scene segmentation method based on 3D Gaussian Splitting, the core idea is to make full use of the relation between different views in the step-by-step segmentation process, which relates to two complementary aspects: 1, the target is to enhance the algorithm for aggregating a 2D segmentation mask into 3D segmentation, especially to improve the precision and integrity in a scene with a small number of lenses; this function of a small number of lenses is crucial to providing cues for new views because the cues depend on segmentation predictions from preceding views. Second, by retaining the reliable cues used in the previous view, the new view can avoid cueing inconsistent targets. On the basis, a simple and effective lifting algorithm based on Gaussian center perspective is provided, the completeness of three-dimensional segmentation is guaranteed, boundary distinguishing is enhanced, and accurate three-dimensional segmentation is achieved. Besides, a prompt propagation strategy is introduced, and the reliable prompt from the adjacent view is effectively utilized by restoring the 2D prompt into the scene, so that accurate segmentation is realized.

The invention relates to a three-dimensional object segmentation method and system based on a neural radiation field, and the method comprises the steps: training a basic neural radiation field model through a multi-view original RGB image, and obtaining a pre-trained neural radiation field model; fine tuning is performed on the pre-trained neural radiation field model according to a two-dimensional segmentation mask corresponding to the original RGB image, a three-dimensional segmentation neural radiation field model is obtained, and the two-dimensional segmentation mask comprises a first pixel value representing a foreground object and a second pixel value representing a background; segmenting an image to be segmented through the three-dimensional segmentation neural radiation field model to generate segmented three-dimensional point clouds; and clustering the three-dimensional point cloud to obtain a three-dimensional segmentation result of the to-be-segmented image. According to the method, high-precision three-dimensional segmentation is realized through a two-stage fine tuning strategy, a neural radiation field model architecture does not need to be changed, a three-dimensional segmentation process is simplified, and segmentation precision and counting accuracy are improved.

The invention discloses a radiotherapy path planning method based on surface profile monitoring, particularly relates to the field of medical image data processing, and is used for solving the problems of low reusability and low intelligent degree of existing medical radiotherapy path planning. The method comprises the following steps: constructing a distribution prediction model of radiotherapy incidence points by collecting marked target region characteristic parameters and incidence point distribution in a historical case, completing three-dimensional segmentation of a target region and an organ structure in a CT image of a current patient, and establishing a unified voxel coordinate system; constructing an incidence point array covering the surface of the target region according to a prediction result, generating a candidate radiotherapy path set, and performing risk organ recognition on each path; further combining the spatial distance between the path trajectory and the risk organ contour surface to calculate the comprehensive risk score of the candidate path; and finally, performing dose simulation on the candidate paths according to a scoring sequence, outputting a radiotherapy path planning result meeting dose and safety requirements, and improving the precision and clinical applicability of path selection.

The invention discloses a pulmonary nodule growth prediction method based on the combination of LSTM and GAN, and relates to the technical field of image analysis and processing, and the method comprises the following steps: collecting multi-time-sequence chest CT data, and carrying out the marking and auditing to form a data set; preprocessing the marked CT data to obtain a standardized pulmonary nodule ROI (Region of Interest); segmenting a pulmonary nodule region by using the three-dimensional image segmentation network, and generating a three-dimensional segmentation mask; inputting the segmentation masks of the multiple time nodes into an LSTM network, and extracting a time sequence feature vector representing nodule dynamic evolution; based on the time sequence feature vector and the random noise, the generator synthesizes the predicted pulmonary nodule image at the future moment, and the discriminator jointly optimizes model parameters through multiple loss functions; and inputting the three-dimensional segmentation masks at the current moment and the historical moment, and generating a predicted pulmonary nodule image at the future moment. The method can effectively improve the situation that the prior art is insufficient in utilization of time sequence information and lacks high-quality generation and inference ability.

Unsupervised volumetric 3D animation (UVA) of non-rigid deformable objects without annotation learns the 3D structure and dynamics of the object only from a single view red / green / blue (RGB) video and decomposes the single view RGB video into semantically meaningful portions that can be tracked and animated. Using a 3D automatic decoder framework, the UVA model learns 3D geometry and partial decomposition of underlying objects from still or video images in a fully unsupervised manner via a microperspective n-point (PnP) algorithm pairing with a keypoint estimator. This allows the UVA model to perform 3D segmentation, 3D keypoint estimation, novel view synthesis, and animation. The UVA model may obtain animatable 3D objects from a single or several images. The UVA method also characterizes a space in which all objects are represented in their normative, animated ready form. Applications include creating a shot from an image or video of a social media application.

The invention discloses a cerebral hemorrhage heterogeneity quantitative evaluation system based on cyclic cross-attention clustering, and relates to the field of cerebral hemorrhage evaluation, and the system comprises an image segmentation module which is used for carrying out the three-dimensional segmentation of a brain plain-scan CT image through an nnU-NetV2 model, and outputting the binary masks of a hematoma region and an edema region; the feature extraction module is used for extracting high-dimensional pixel features from the mask region and generating an initial feature embedding matrix; the cyclic cross-attention clustering module is used for dynamically updating a clustering center through a multi-scale cyclic EM cross-attention mechanism and executing clustering analysis on the feature embedding matrix; the heterogeneity scoring module is used for calculating information entropy and generating hematoma and edema heterogeneity scores; and the prognosis prediction module is used for predicting the hematoma expansion risk and the neural function outcome through a machine learning model. According to the scheme, objective quantification of cerebral hemorrhage heterogeneity can be realized, and the prediction accuracy of the hematoma expansion risk and the neural function outcome is improved.

Provided are an augmented reality device for providing an augmented reality service for controlling an object in a real world space, and an operating method for the same. The method may include recognizing a first plane comprising a wall and a second plane comprising a floor from a spatial image based on a photograph of the real world space; generating a three-dimensional (3D) model of the real world space by extending the wall and extending the floor along the respective planes and performing 3D in-painting on an area of the extended wall and the extended floor that is hidden by an obstructive object; segmenting an object selected by a user input from the spatial image based on two-dimensional (2D) segmentation; and segmenting the object on the spatial image from the real world space based on a 3D model or 3D position information of the object using 3D segmentation.

Systems and methods for segmenting 3D images are provided. In an embodiment, the system includes a neural network having a low-resolution module trained to infer a complete low-resolution segmentation from an input low-resolution 3D image and to generate corresponding low-resolution feature maps; and a high-resolution module trained to infer a complete high-resolution segmentation from an input high-resolution 3D image and the feature maps from the low-resolution module. Methods for training the neural network and measuring a volume of an object using the 3D segmentations are also described.

Systems and methods are provided for determining fabrics of a geological sample using three-dimensional segmentation. An example method can include receiving three-dimensional (3D) image of a geological sample, adjusting an initial size of the 3D image of the geological sample, and partitioning the resized 3D image of the geological sample into cubes. The example method can include, for each cube, generating orthogonal planes based on a center of mass of each cube and extracting, for the orthogonal planes associated with each cube, one or more features to represent texture of the geological sample. The example method can further include grouping the cubes into one or more clusters based on the one or more features and constructing a volume of the resized 3D image of the geological sample based on the one or more clusters for a texture analysis of the geological sample.

Segmenting method, for carrying out 3D segmentation of a sample, the sample comprising at least one biological object, the sample developing over time, such that at least one biological object divides or changes shape or position over time, the method comprising:at various times, acquiring a stack of images (P(t)) of the sample;segmenting images, such as to obtain masks corresponding to each biological object;implementing a segmentation algorithm that is said to be prompted, such as to use masks obtained, for an object, in an image, to define masks, for the same object, in another image.

This invention discloses a radiotherapy pathway planning method based on surface contour monitoring, specifically relating to the field of medical image data processing. It addresses the issues of low reusability and intelligence in existing medical radiotherapy pathway planning methods. By collecting labeled target area feature parameters and incident point distribution from historical cases, a distribution prediction model for radiotherapy incident points is constructed. Three-dimensional segmentation of the target area and organ structures is performed in the current patient's CT images, establishing a unified voxel coordinate system. Based on the prediction results, an array of incident points covering the target area surface is constructed, generating a candidate radiotherapy pathway set, and risk organ identification is performed for each pathway. Furthermore, the spatial distance between the pathway trajectory and the risk organ contour surface is combined to calculate a comprehensive risk score for the candidate pathways. Finally, dose simulation is performed on the candidate pathways according to the score order, outputting radiotherapy pathway planning results that meet dose and safety requirements, thus improving the accuracy and clinical applicability of pathway selection.

The invention provides an intelligent detection and segmentation method and device for livestock meat. The method comprises the following steps: acquiring an original image of the livestock meat, marking segmentation points of the original image to form a segmented image, forming a data pair by the original image and the segmented image, and forming a training data set by the data pair; constructing a segmentation model, and training the segmentation model by using the training data set; obtaining a target image of a target livestock meat product, and obtaining a two-dimensional key point of the target livestock meat product through the segmentation model according to the target image; generating a three-dimensional segmentation point based on the two-dimensional key point; and controlling a segmentation robot to segment the target livestock meat product according to the three-dimensional segmentation point. The invention further provides electronic equipment and a computer readable storage medium. According to the invention, the robot is used for replacing manual work to carry out refined segmentation on the pork carcass middle section rib rows, the labor cost is reduced, and the intelligent and automatic level of refined segmentation in the meat industry is effectively improved.

This invention discloses an adaptive edge-aware 3D medical image segmentation method, with the following specific steps: S1, constructing an adaptive edge-aware network, which includes an encoder and a decoder, with a skip connection between the encoder and decoder; S2, acquiring and processing a 3D medical image; S3, inputting the preprocessed image from step S2 into the encoder of the adaptive edge-aware network through a patch partitioning layer, then into the decoder through residual blocks and adaptive weight matching blocks. The decoder output and the original input image are skip-connected through adaptive weight matching blocks, and finally, the image segmentation result is output through residual blocks and Fourier convolution. This invention exhibits stronger robustness and boundary accuracy in multi-organ 3D segmentation tasks, providing an efficient and scalable solution for medical image segmentation.

The invention discloses a citrus X-ray image rapid reconstruction and defect segmentation method based on sparse point cloud and 3DGS, and relates to image processing, and the method comprises the following steps: S1, obtaining X-ray image data in a citrus under a sparse view angle; s2, performing three-dimensional reconstruction on the X-ray image data by adopting a three-dimensional Gaussian point cloud reconstruction method based on adaptive density control to obtain a reconstructed three-dimensional body; s3, performing automatic defect segmentation on the reconstructed three-dimensional body through the three-dimensional segmentation model to obtain two-dimensional defect mask slices and three-dimensional defect voxel data; and S4, according to the two-dimensional defect mask slices and the three-dimensional defect voxel data, performing comprehensive evaluation on the internal defects of the citrus, and outputting an internal quality report of the citrus. According to the method, full-process automatic processing from sparse view angle X-ray image data acquisition to three-dimensional volume reconstruction to automatic defect segmentation and type identification is realized, and the purpose of efficiently, accurately and losslessly identifying the internal defects of the citrus is achieved.

The invention relates to an airway path planning method and device based on a throat CT image and computer equipment. The method comprises the following steps: acquiring a throat CT image; the throat CT image is preprocessed, the preprocessing includes unifying thickness parameters of the throat CT image, and data expansion processing is carried out on a target layer, including the glottis part, in the throat CT image with the consistent thickness parameters in a copying mode; inputting the preprocessed throat CT image into the trained three-dimensional segmentation model for three-dimensional segmentation to obtain a segmented airway three-dimensional structure; obtaining a path constraint condition according to the obstacle differentiation weights of different areas in the divided airway three-dimensional structure; layering treatment is carried out on the divided airway three-dimensional structure, and a multi-layer airway cross section is obtained; obtaining a center point sequence based on the center point coordinates of the cross sections of the multiple layers of airways; and obtaining an initial airway path corresponding to the throat CT image based on the target point in the airway, the center point sequence and the path constraint condition.

The invention discloses a 3D image reconstruction model training method and device and a 3D image reconstruction method and device.According to the method, before model training, the one-to-one correspondence relation between 2D anchor points on a 2D Xray image and 3D anchor points on a target 3D segmented image is established, a bone structure on the 2D Xray image of a complex and large structure is split into simple segmented structures, and then the 3D image is reconstructed. And the 2D Xray image and the target 3D segmented image can be split for training. And meanwhile, different bone joints are decoupled, so that the video memory capacity required when the neural network model is trained is reduced, and the convergence speed is higher. In addition, the 2D anchor points and the 3D anchor points uniquely determine the relationship between the 2D anchor points and the 3D anchor points, so that data amplification is easier, and a final algorithm can be used for training to obtain a 3D image reconstruction model with higher precision.

The invention belongs to the field of intelligent medical treatment, and particularly relates to a bladder tumor image segmentation method and device, a medium and a program product. The method comprises the following steps: S101, acquiring an image of a bladder tumor patient; s102, inputting the image into a 2D U-Net for tumor segmentation to obtain a 2D segmentation result; s103, traversing the shape of the tumor based on the 2D segmentation result, inputting the image into 3D U-Net for tumor segmentation to obtain a 3D segmentation result if the shape of the current tumor is regular during traversing, and taking the segmentation result of the current tumor in the 3D segmentation result; if the tumor shape is irregular, the segmentation result of the current tumor in the 2D segmentation result is obtained, and after traversal is completed, the segmentation results of the multiple tumors are output after merging. Through the hybrid 2D-3D method, the segmentation performance can be optimized by using the complementary advantages of the two methods, and a better segmentation result is obtained.

The application discloses a double-branch Raio gating graph aggregation multi-modal meningioma segmentation method, belongs to the cross technical field of computer vision and medical image processing, is used for meningioma segmentation, and comprises the following steps: preparing a data set, constructing a deep neural network model, and performing neural network training; inputting the data set into the trained deep neural network model; first inputting a double-branch multi-modal input module to output a multi-modal fusion feature tensor; then inputting the multi-modal fusion feature tensor into a hierarchical Swin Transformer encoder to output three-dimensional segmentation masks of enhanced tumors, tumor cores and whole tumor regions of meningioma. Through double-branch multi-modal input fusion, hierarchical Swin Transformer coding, edge perception modulation and Laplace gating graph aggregation, high-precision three-dimensional segmentation of enhanced tumors, tumor cores and whole tumor regions of meningioma is realized.

The invention discloses a three-dimensional target positioning method and system based on two-dimensional segmentation and three-dimensional reconstruction and related equipment, and belongs to the technical field of three-dimensional localization, and the method comprises the steps: carrying out the key target segmentation of a visual perception video based on a two-dimensional image segmentation model, obtaining a video frame, and carrying out the RGB label labeling of the video; inputting the marked video frame into the three-dimensional reconstruction model to obtain a three-dimensional point cloud model with a mask; screening point clouds on the basis of label numerical values; analyzing the screened point clouds by using a clustering algorithm to generate a clustering point set; and extracting three-dimensional space center points of different clustering point sets as three-dimensional positions of the segmented target, and obtaining a spatial position of the target relative to the shooting point. According to the method, the robot can be better supported to complete identification, three-dimensional segmentation and positioning of a key target by using the instantly acquired two-dimensional optical image information, and the robot is supported to complete tasks such as navigation and operation by using related information.

This invention discloses a method for esophageal cancer tumor target region segmentation based on cross-modal feature fusion of PET / CT images. The method uses a Transformer-fused Attention Progressive Semantic Nested Network (TransAttPSNN) as the 3D segmentation model for esophageal cancer tumor target regions. The TransAttPSNN network has an AttPSNN backbone structure and includes two segmentation networks: one for PET streams and the other for CT streams. Transformer cross-modal adaptive feature fusion modules are embedded in the different feature levels of the two segmentation networks. Compared with existing technologies, this invention effectively improves the segmentation accuracy of esophageal cancer tumor target regions and achieves better segmentation performance.