Navigation positioning method and device based on personalized digital human anatomical organ reconstruction

Abstract

The invention provides a navigation positioning method and device based on personalized digital human anatomical organ reconstruction, and the method comprises the steps: directly obtaining human body features from a human body image and / or a human body video stream through a deep learning network, and outputting target parameters of a skin multi-person linear model, and rapid and objective parameterization of the human body model based on the appearance data is realized. The first position of the target vertex in the first human body internal anatomical model can be quickly positioned based on the target parameter in combination with the first human body internal anatomical model, and the position is mapped to the second position of the human body image through spatial transformation of the virtual camera parameter, so that the target navigation positioning position is determined. In the process, the model parameters do not need to be manually and subjectively modified, the in-vivo organ structure is efficiently predicted according to the human body appearance image and / or the human body video stream, and the technical problems that in personalized human body three-dimensional anatomical structure reconstruction, model parameter adjustment depends on manpower, and prediction of the in-vivo structure based on appearance data is lacked are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer-aided surgical navigation, and in particular to a navigation positioning method and device based on personalized digital human anatomical organ reconstruction. BACKGROUND

[0002] In the medical field, traditional navigation technology mainly relies on organ reconstruction and registration positioning based on patient medical images. This kind of method usually needs to use CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) and other medical imaging means to obtain three-dimensional anatomical information of the patient to realize accurate navigation. However, these methods have several limitations. First, they are highly dependent on expensive and space-limited medical imaging equipment, which limits their application in resource-limited or dynamic scenarios, such as emergency medical scenarios for accidents and battlefields. Second, the reconstruction and registration process after image acquisition is usually complex, requiring a large amount of time and computing resources, and the registration accuracy may be unstable in actual clinical environments due to interference factors. In addition, this kind of method usually requires detailed imaging and processing before surgery, which further increases the surgery preparation time and overall medical cost.

[0003] In addition to surgical navigation positioning, non-invasive diagnosis and treatment methods represented by ultrasound also have urgent needs for accurate positioning. However, the current positioning accuracy and reliability still largely depend on the clinical experience and operation skills of the operator, and it is difficult to guarantee the positioning quality in beginners or automatic navigation systems. At the same time, due to the significant differences in human anatomical structure among individuals in the population, existing ultrasound positioning methods based on empirical rules are difficult to meet the needs of large-scale, standardized or automated clinical scenarios, such as automatic scanning tasks of ultrasound robots for large-scale health check-ups.

[0004] In recent years, although some research has tried to introduce deep learning methods to assist human positioning, most of these methods rely on specific data sets or supervised training for specific targets, lack the ability to model the whole body range and diverse anatomical structures, and are difficult to realize a positioning system with strong generalization. In addition, in actual clinical applications, due to the complexity of the data itself and the high dynamic nature of the medical scene, existing methods still face many challenges in positioning accuracy and robustness, and it is difficult to meet the core needs of intelligent medical care for high-precision, strong generalization positioning performance.

[0005] In summary, the individual organ reconstruction and navigation in the current medical field still mainly relies on the acquisition and processing of medical images (such as CT / MRI). Although such methods can provide more accurate anatomical information, the reconstruction process is generally complex and time-consuming, which is difficult to meet the urgent needs of real-time and automatic processing in clinical practice. The existing individual anatomical structure reconstruction technology is mostly based on the direct modeling of static medical images, which lacks the adaptability to cross-scene changes and the flexibility to respond to dynamic changes in patient posture, and is difficult to cover a variety of clinical use scenarios. At the same time, the general digital human modeling method mostly focuses on the surface reconstruction of the external morphology of the human body, and usually cannot effectively express the personalized anatomical structure and organ level information in the body, so its applicability in medical navigation, intervention planning and other applications is extremely limited. At present, there is still a lack of an effective technical path that can directly infer and reconstruct the internal organ structure based on individual external RGB or RGB-D visual information, and further realize navigation positioning and intervention guidance. This key technical bottleneck seriously restricts the further popularization and application of personalized digital human technology in medical navigation and even intelligent medical systems. SUMMARY

[0006] The application provides a navigation positioning method and device based on personalized digital human anatomical organ reconstruction, to solve the defect that the reconstruction method in the prior art cannot directly predict the internal organ structure according to the appearance RGB or RGBD information of a person, limiting its application in actual medical scenarios.

[0007] The application provides a navigation positioning method based on personalized digital human anatomical organ reconstruction, comprising the following steps: Obtaining the human body features of a human body image and / or a human body video stream, and inputting the human body image and / or the human body video stream into a deep learning network to obtain the target parameters of a skin multi-person linear model output by the deep learning network; the target parameters include shape parameters, posture parameters and virtual camera parameters; the shape parameters are used to represent the body shape features of the human body; the posture parameters are used to represent the posture features of the human body; and the virtual camera parameters are used to represent the intrinsic and extrinsic parameters of a virtual camera in the image digital space; Obtaining the position index of an organ to be positioned, obtaining the target vertex corresponding to the position index in a first human body internal anatomical model, and determining the first position of the target vertex in the first human body internal anatomical model based on the target parameters; the first human body internal anatomical model is a human body internal anatomical model with the same parameterization as the skin multi-person linear model; Based on the spatial transformation of the virtual camera parameters, the first position is transformed to a second position of the human body image, and the target navigation positioning position is determined based on the second position.

[0008] The application provides a navigation positioning method based on personalized digital human dissection organ reconstruction. Obtaining a shape space mapping relationship between skin information in the second human internal dissection model and skin information in the skin multi-person linear model; Mapping internal organs of the second human internal dissection model to the skin multi-person linear model based on shape space consistency of the statistical shape model and the shape space mapping relationship, to obtain the first human internal dissection model.

[0009] The application provides a navigation positioning method based on personalized digital human dissection organ reconstruction. Segmenting human structures and performing three-dimensional reconstruction on N sets of medical image data to obtain N sets of reconstructed models of human skin and internal multi-organ dissection; Obtaining an average statistical shape and a high-dimensional shape space of the human structure model from the N sets of reconstructed models according to shape statistics technology; the average statistical shape represents an average human shape of samples covered by the N sets of medical image data; and the high-dimensional shape space represents group difference human shape features of the samples covered by the N sets of medical image data; Determining the second human internal dissection model based on the average statistical model and the high-dimensional shape space.

[0010] The application provides a navigation positioning method based on personalized digital human dissection organ reconstruction. In the case that the organ to be positioned is not of a preset type, obtaining a reference dissection positioning of the organ to be positioned in the first human internal dissection model; Determining the target navigation positioning position based on the first human internal dissection model, the reference dissection positioning, and an offset between the organ to be positioned and the reference dissection positioning.

[0011] The application provides a navigation positioning method based on personalized digital human dissection organ reconstruction. The method comprises the following steps: Inputting first human features corresponding to the RGB human image and second human features corresponding to the human depth image into the feature extraction encoder to obtain human features; inputting the human body features into the target parameter decoder to obtain target parameters output by the target parameter decoder; The target parameters include human body pose decoding, the shape parameters, and the virtual camera parameters; the human body pose decoding includes human body joint point positions of a digital human and the pose parameters, and the pose parameters are represented by a rotation matrix, an angle, or a quaternion.

[0012] According to the navigation positioning method for reconstructing an anatomical organ based on a personalized digital human provided in the application, after the human body features are input into the target parameter decoder to obtain target parameters output by the target parameter decoder, the method further includes: inputting the pose parameters, the shape parameters, and the virtual camera parameters into the digital human mapping module to obtain the first human body internal anatomical model and the skinned multi-person linear model reconstructed by the deep learning network based on the input image.

[0013] According to the navigation positioning method for reconstructing an anatomical organ based on a personalized digital human provided in the application, determining a target navigation positioning position based on the second position includes: Based on the intrinsic and extrinsic parameters of the real camera hardware, in combination with the depth information of the human body image and / or the depth information of the human body video stream, the second position is transformed to a real world space position based on the real camera hardware through spatial transformation to obtain a target navigation positioning position.

[0014] The application further provides a navigation positioning device for reconstructing an anatomical organ based on a personalized digital human, which includes the following units: The acquisition unit is configured to acquire human body features of a human body image and / or a human body video stream, and input the human body image and / or the human body video stream into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters are used to represent body shape features of a human body; the pose parameters are used to represent posture features of a human body; and the virtual camera parameters are used to represent intrinsic and extrinsic parameters of a virtual camera in a digital image space. The first position determining unit is configured to acquire a position index of an organ to be positioned, acquire a target vertex corresponding to the position index in a first human body internal anatomical model, and determine a first position of the target vertex in the first human body internal anatomical model based on the target parameters; the first human body internal anatomical model is a human body internal anatomical model that is parameterized with the skinned multi-person linear model. determine a target navigation positioning position based on the first position and the second position.

[0015] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the navigation positioning method based on the reconstruction of the anatomical organ of the personalized digital human according to any one of the above.

[0016] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the navigation positioning method based on the reconstruction of the anatomical organ of the personalized digital human according to any one of the above.

[0017] The application further provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the navigation positioning method based on the reconstruction of the anatomical organ of the personalized digital human according to any one of the above.

[0018] The application provides the navigation positioning method and device based on the reconstruction of the anatomical organ of the personalized digital human, which directly obtains the human features from the human image and / or human video stream through the deep learning network, outputs the target parameters of the skinning multi-human linear model, including the shape parameters, the pose parameters and the virtual camera parameters, and realizes the rapid and objective parameterization of the human model based on the appearance data. Moreover, the shape parameters accurately depict the human body shape, the pose parameters accurately capture the human posture, and the virtual camera parameters accurately describe the internal and external parameters of the virtual camera in the image digital space. Based on the target parameters, the first position of the target vertex in the first human internal anatomical model can be quickly positioned, and the position is mapped to the second position of the human image through the spatial transformation of the virtual camera parameters, so that the target navigation positioning position is determined. This process does not need manual subjective modification of the model parameters, directly utilizes the deep learning technology, and efficiently predicts the internal organ structure according to the human appearance, the human image and / or the human video stream, effectively solves the technical problems that the model parameter adjustment in the reconstruction of the personalized human three-dimensional anatomical structure depends on manual work and lacks the prediction of the internal structure based on the appearance data, and greatly improves the efficiency and accuracy of the reconstruction of the human three-dimensional anatomical structure, thereby providing more reliable technical support for the medical navigation positioning and other fields. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0020] Figure 1 is one of the flowcharts of the navigation positioning method based on the personalized digital human anatomical organ reconstruction provided by the present application.

[0021] Figure 2 is another flowchart of the navigation positioning method based on the personalized digital human anatomical organ reconstruction provided by the present application.

[0022] Figure 3 is a schematic diagram of the target parameter outputted by the deep learning network based on the skinning multi-person linear model provided by the present application.

[0023] Figure 4 is a flowchart of the first human internal anatomical model construction provided by the present application.

[0024] Figure 5 is a structural schematic diagram of the navigation positioning device based on the personalized digital human anatomical organ reconstruction provided by the present application.

[0025] Figure 6 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0027] The terms "first", "second", etc. in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind.

[0028] In the related art, a digital human parameterized model: through statistical shape model technology, such as PCA (Principal Component Analysis), or deep learning features, such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and Transformer extraction method, a large number of human shape change rules are mapped to a high-dimensional and small amount of parameter space (for example, only 10 shape parameters can almost express all human shapes), so that different body shapes and postures can be represented by a small amount of parameters, the solution space of three-dimensional human reconstruction is reduced, and the deep ambiguity problem is alleviated. In this field, SMPL (Skinned Multi-Person Linear model) is one of the commonly used representative models.

[0029] Figure 1 is one of the flowcharts of the navigation positioning method based on the personalized digital human anatomical organ reconstruction provided by the application, Figure 2 is the second flowchart of the navigation positioning method based on the personalized digital human anatomical organ reconstruction provided by the application, as shown in Figure 1 、 Figure 2 The method comprises steps 110, 120 and 130.

[0030] Step 110: obtaining human features of a human body image and / or a human body video stream, and inputting the human body image and / or the human body video stream into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters comprise shape parameters, posture parameters and virtual camera parameters; the shape parameters are used to represent body shape features of the human body; the posture parameters are used to represent posture features of the human body; and the virtual camera parameters are used to represent internal and external parameters of a virtual camera in a digital space of the image.

[0031] Specifically, at present, parameterized 3D human body surface models represented by SMPL are widely studied, and a considerable number of deep learning model algorithms can obtain personalized parameters of the SMPL parameterized model: shape parameters and posture parameters according to the ordinary RGB / RGBD appearance of a person.

[0032] The navigation and positioning method based on personalized digital human anatomical organ reconstruction in this embodiment of the invention has real-time performance: (1) The method has uniqueness and exclusivity: the algorithm realizes the same parametric shape change of the human internal anatomical organ model (taking the skeleton as an example) and the human skin parameterization model (taking SMPL as an example), that is, after determining the skin parameters of the target person to be simulated, the shape of the skin and bones of the person will be directly obtained. Thus, the same parametric makes the method of this embodiment of the invention have real-time bone model shape acquisition. (2) The idea of ​​this embodiment of the invention is to use a regression method based on deep learning to directly regress the shape parameters of human skin from the human appearance RGB(D), rather than manual adjustment by the human. In this step, there are many existing deep learning network frameworks that can support the implementation of this step. However, the idea of ​​this embodiment of the invention is precisely to use the widely used and continuously improving SMPL deep learning algorithm to correct more accurate human body surface parameters, and the work of this embodiment of the invention is to make the human internal organ model also conform to the SMPL parameterization, the specific method is as follows: First, human features are obtained from human images and / or human video streams. This can be done by obtaining human features from human images and human features from human video streams, or by obtaining human features from human images and human features from human video streams. This embodiment of the invention does not specifically limit this.

[0033] Here, the human body image can be pre-captured by an image acquisition device, captured in real time, or downloaded or scanned from the Internet. The human body video stream can be pre-captured and stored human body video or a real-time captured human body video stream; this embodiment of the invention does not specifically limit this. The human body video stream includes a sequence of human body image frames, which contains multiple frames, each originating from the human body video stream. The multiple frames are arranged in chronological order within the human body video stream, thus forming the human body image frame sequence.

[0034] After obtaining human images and / or human video streams, the human images and / or human video streams can be input into a deep learning network to obtain the target parameters of the skinned multi-person linear model output by the deep learning network. The target parameters include three types of parameters: shape parameters, pose parameters, and virtual camera parameters. Shape parameters are used to characterize the body shape features of the human body; pose parameters are used to characterize the posture features of the human body; and virtual camera parameters are used to characterize the intrinsic and extrinsic parameters of the virtual camera in the digital space of the image.

[0035] The SMPL model represents human body deformation using vertex displacements, defined by 10-dimensional shape parameters. and 72-dimensional attitude parameters The program controls the deformation of a standard human skin mesh with 6890 vertices and 23 joints to generate human figures of different shapes and poses, as well as camera parameters. Camera parameters are used to ensure that the reconstructed digital human skin matches the human image on the RGB / RGBD image.

[0036] Figure 3 This is a schematic diagram illustrating the target parameters of a skinned multi-person linear model based on a deep learning network, as provided by the present invention. Figure 3 As shown, the human body image includes RGB human body images and / or human body depth images (RGBD human body images), and the deep learning network includes a feature extraction encoder, a target parameter decoder, and a digital human mapping module. The first human body feature F corresponding to the RGB human body image can be... RGB And the second human feature F corresponding to the human depth image Depth The input is fed into the feature extraction encoder to obtain human body features. Here, the feature extraction encoder can be a CNN model, a deep neural network (DNN), or a combination of CNN and DNN, etc., and this embodiment of the invention does not specifically limit it.

[0037] Next, the first human feature F is effectively fused using the multimodal fusion module MA based on the soft attention mechanism, according to the occlusion and lighting characteristics of the image. RGB Second human body characteristics F Depth This process obtains human body features, improving the network's robustness under extreme conditions. Then, an objective parameter decoder and a fully connected layer are used to regress the precise locations of 24 3D key points of the human body from these features. (3*24), and the twist rotation parameters (1*24) and the shape parameter β(10) of SMPL. Combined with 3D keypoints Twist rotation parameters Key points of SMPL initial attitude Using inverse kinematics, the SMPL attitude parameters θ(3*24) are obtained. The formula for the attitude parameters is: Applying θ and β to the SMPL model allows for the reconstruction of a human skin mesh. The parameter dimension representations given above are for reference only; there are many other options for representing the pose parameter θ, such as rotation matrices, angles, quaternions, etc.

[0038] The human body features are input into the target parameter decoder to obtain the target parameters output by the target parameter decoder. The target parameters include human pose decoding, shape parameters, and virtual camera parameters. Human pose decoding includes the human body joint position and pose parameters of the digital human. The pose parameters are represented by rotation matrices, angles, and quaternions.

[0039] Then, the pose parameters, shape parameters, and virtual camera parameters can be input into the digital human mapping module to obtain the first human internal anatomical model and the skinned multi-person linear model reconstructed from the input images of the deep learning network. Here, the first human internal anatomical model is a statistical model of the human skin and organ model.

[0040] Step 120: Obtain the location index of the organ to be located, obtain the target vertex corresponding to the location index in the first human internal anatomy model, and determine the first position of the target vertex in the first human internal anatomy model based on the target parameters; the first human internal anatomy model is a human internal anatomy model with the same parameterization as the skinned multi-person linear model.

[0041] Specifically, the location index of the organ to be located can be obtained, the target vertex corresponding to the location index can be obtained in the first human internal anatomical model, and the first position P1 of the target vertex in the first human internal anatomical model can be determined based on the target parameters.

[0042] Here, the organ to be located can be the liver, spleen, or bones, etc., and the embodiments of the present invention do not specifically limit it.

[0043] If the navigation target is the liver, the required vertex index on the personalized organ of the liver is used directly. This is because the first human internal anatomy model is a human internal anatomy model with the same parameterization as the skinned multi-person linear model (SMPL model). That is, both the SMPL model and the first human internal anatomy model constructed in this embodiment of the invention will have a unified model index order. For example, the human chin is always vertex number x in the first human internal anatomy model and the skinned multi-person linear model, and specific anatomical points of the liver (such as the vertex of the right lobe of the liver and the entrance of the portal vein) also have fixed index numbers in the first human internal anatomy model and the skinned multi-person linear model. After determining the target vertex, the first position of the target vertex in the first human internal anatomy model can be determined based on the target parameters. Here, the first position refers to the model space position of the target vertex in the first human internal anatomy model.

[0044] Understandably, this unified location index avoids re-annotating organ key points each time, allowing for cross-model matching of the same anatomical structures directly through the location index.

[0045] Step 130: Based on the spatial transformation of the virtual camera parameters, transform the first position to the second position of the human body image, and determine the target navigation and positioning position based on the second position.

[0046] Specifically, after obtaining the first position P1, the first position P1 can be transformed to the second position P2 of the human body image based on the spatial transformation of the virtual camera parameters, and the target navigation and positioning position P3 can be determined based on the second position P2. Here, the second position P2 is the pixel coordinate position in the human body image or human body video stream.

[0047] After obtaining the second position, based on the intrinsic and extrinsic parameters of the real camera hardware, combined with the depth information of the human body image and / or the depth information of the human body video stream, the second position can be transformed to a real-world spatial position based on the real camera hardware through spatial transformation, thus obtaining the target navigation and positioning position. The target navigation and positioning position refers to the precise three-dimensional coordinates determined in the real world (such as an operating room or rehabilitation training environment) after mapping the digital human model (first human internal anatomical model) to the real world. This coordinate is used to guide medical operations, such as the positioning of surgical instruments.

[0048] Option 1: If the navigation target (organ to be located) is the liver, the required vertex indexes on the personalized liver organ model can be directly used (because SMPL and our constructed first human internal anatomy model will have a unified model index order) to obtain the 3D vertex coordinates of the skeleton in the virtual camera coordinate system. Projecting based on the virtual camera parameter α, the position of this point in the original RGB / RGBD image can be obtained. If a depth map is input, the three-dimensional coordinates based on the real camera space can be obtained. Since the position of the camera hardware in space is known, the location of a real, specific human organ anatomical point, anatomical path, and anatomical region can be obtained through coordinate transformation.

[0049] Option 2: For organs that lack statistical modeling or cannot be statistically constructed, a personalized skeletal model is used as a reference to find anatomical location. That is, if a navigation target is near a certain rib, we use the rib's location plus a reasonable offset to replace this navigation target.

[0050] Because clinical practice recognizes a strong correlation between the location of bones and organs—for example, when scanning the heart with ultrasound, the scan is generally performed between the fourth and fifth ribs—the location of anatomical points on the bones can be determined by (vertex index of the personalized first human internal anatomy model -> position in the original RGB / RGBD image -> 3D coordinates based on real camera space).

[0051] The method provided in this invention directly acquires human features from human images and / or human video streams using a deep learning network, outputting target parameters for a skinned multi-person linear model, including shape parameters, pose parameters, and virtual camera parameters. This achieves rapid and objective parameterization of the human model based on appearance data. Furthermore, the shape parameters accurately depict the human body shape, the pose parameters precisely capture the human posture, and the virtual camera parameters accurately describe the intrinsic and extrinsic parameters of the virtual camera in the digital image space. Based on the target parameters and combined with a first internal anatomical model of the human body, the first position of the target vertex in the first internal anatomical model can be quickly located. Through spatial transformation of the virtual camera parameters, this position is mapped to a second position in the human image, thereby determining the target navigation and positioning location. This process eliminates the need for manual modification of model parameters, directly utilizing deep learning technology to efficiently predict internal organ structures based on human appearance images and / or human video streams. It effectively solves the technical problems of relying on manual adjustment of model parameters and lacking the ability to predict internal structures based on appearance data in personalized three-dimensional human anatomical reconstruction, greatly improving the efficiency and accuracy of three-dimensional human anatomical reconstruction and providing more reliable technical support for fields such as medical navigation and positioning.

[0052] Based on the above embodiments, the steps for determining the second human internal anatomical model include: Step 210: Perform human structure segmentation and three-dimensional reconstruction on N medical image data to obtain N sets of initial human skin and internal organ anatomy reconstruction models. Step 220: Based on shape statistics techniques, obtain the average statistical shape and high-dimensional shape space of the human body structure model from N sets of reconstructed models; the average statistical shape represents the average human body shape of the samples covered by N medical image data; the high-dimensional shape space represents the group-differential human body shape characteristics of the samples covered by N medical image data. Step 230: Based on the average statistical model and the high-dimensional shape space, determine the second human internal anatomical model.

[0053] Specifically, N medical image data sets refer to the medical image datasets used to construct internal anatomical models of the human body. These medical image data sets can be large-scale whole-body medical image datasets such as CT, MRI, and DXA (Dual-energy X-ray absorptiometry), thus ensuring the validity of subsequent statistical analysis. For each medical image data set, human structure segmentation and 3D reconstruction are required to obtain N sets of initial reconstructed models of human skin and internal multi-organ anatomy.

[0054] Human body structure segmentation refers to distinguishing human body structures (such as bones, liver, lungs, heart, etc.) from the image background in medical image data. Human body structure segmentation can employ image processing and computer vision techniques, such as thresholding, region growing, edge detection, and deep learning-based image semantic segmentation methods, which are not specifically limited in this embodiment of the invention. The purpose of segmentation is to accurately extract the three-dimensional geometric shape information of each organ.

[0055] 3D reconstruction refers to the conversion of segmented human body structural data into a 3D model. 3D reconstruction can employ either surface rendering or volume rendering techniques. Surface rendering methods extract surface information from the segmentation results to construct a surface mesh model composed of triangles or other polygons. Through 3D reconstruction, we can obtain N sets of initial reconstructed models of human skin and internal organ anatomy. These reconstructed models include the 3D geometric shape information of the human skin and internal organs.

[0056] After obtaining N sets of initial reconstructed models of human skin and internal organ anatomy, statistical shape modeling (SSM) can be used to extract the average statistical shape and high-dimensional shape space from these reconstructed models. Statistical shape modeling is a mathematical method used to describe the shape variations and variability of objects. Its basic idea is to align a set of shape samples, calculate their average shape, and analyze their variation patterns around the average shape.

[0057] The Mean Shape represents the average human body shape of the samples covered by the N medical image datasets. It is obtained by calculating the average coordinates of each corresponding vertex after point-to-point registration of the N reconstructed models. Point-to-point registration means aligning the N reconstructed models to the same coordinate system, so that their corresponding vertices have a semantic correspondence.

[0058] The high-dimensional shape space represents the population-discrepancy human shape features of the samples covered by N medical image datasets. It is obtained by performing principal component analysis (PCA) on the vertex coordinates of the registered N sets of reconstructed models. PCA can reduce the high-dimensional shape data to a low-dimensional shape space and extract the main shape variation patterns. These main shape variation patterns correspond to the principal components of PCA and can be used to describe the shape differences between different individuals.

[0059] After obtaining the average statistical shape and high-dimensional shape space, a second internal anatomical model of the human body can be determined based on these two factors. This second internal anatomical model serves as a reference model for subsequent organ localization.

[0060] The method provided in this invention performs human structure segmentation and 3D reconstruction on N medical image data to obtain N initial reconstructed models of human skin and internal organ anatomy. Then, based on shape statistics techniques, the average statistical shape and high-dimensional shape space of the human structure model are obtained from the N reconstructed models. The average statistical shape represents the average human shape of the samples covered by the N medical image data; the high-dimensional shape space represents the group-specific human shape characteristics of the samples covered by the N medical image data. Finally, based on the average statistical model and the high-dimensional shape space, a second internal human anatomical model is determined. This second internal human anatomical model includes both the general characteristics of the internal anatomical structure (average statistical shape) and reflects the shape differences between individuals (high-dimensional shape space), providing a reliable foundation for subsequent personalized organ localization.

[0061] Based on the above embodiments, the steps for constructing the first human internal anatomical model include: Step 310: Obtain the shape space mapping relationship between the skin information in the second human internal anatomical model and the skin information in the skinned multi-person linear model; Step 320: Based on the shape space consistency of the statistical shape model and the shape space mapping relationship, the internal organs of the second human internal anatomy model are mapped to the skinned multi-person linear model to obtain the first human internal anatomy model.

[0062] Specifically, when constructing the SMPL parametric human skin model, due to the limitations of the 3D human body surface data collected at the time (human dataset 2), obtaining the true segmentation information of the organs of these individuals was almost impossible. Therefore, this embodiment of the invention proposes an innovative method, which utilizes another dataset (human dataset 1) from which human organ segmentation information can be obtained. This dataset contains a large number of whole-body medical images (such as CT, MRI, and DXA images), and through segmentation processing, obtains human skin and internal organ models corresponding to dataset 2. When constructing the statistical model of the human skin and organ models in dataset 1, one can choose to construct a model of skin and a single organ, or a model of skin and multiple internal organs.

[0063] Subsequently, statistical shape modeling techniques (such as PCA principal component analysis) or deep learning feature extraction methods (such as CNN, RNN, Transformer) are used to map the high-dimensional three-dimensional human body and organ data of dataset 2 to a high-dimensional parameter space. During this process, it is ensured that the skin and organs of dataset 2 are homoparameterized.

[0064] One of the core innovations of this invention lies in using the skin of dataset 2 as a medium to construct a shape space mapping relationship between the skin shape space of dataset 2 (skin information in the second human internal anatomical model) and the skin shape space of dataset 2 (skin information in the skinned multi-person linear model). Based on this core shape space mapping relationship, the high-dimensional parameters of dataset 2 are directly mapped to the high-dimensional parameters of the skin and organs in dataset 1. Thus, a parameter association between the private human dataset 1 and dataset 2 is successfully established.

[0065] Therefore, embodiments of the present invention do not require further exploration of how to implement more accurate deep learning regression methods for SMPL human parameters. Simply using publicly available, mature methods to obtain accurate SMPL parameters is sufficient to achieve accurate personalized organ inference.

[0066] For example, the skin vertex coordinates of the second human internal anatomical model can be extracted first, and these coordinates can be projected into its corresponding shape space. Then, the skin vertex coordinates of the skinned multi-person linear model can be extracted, and these coordinates can also be projected into its corresponding shape space. Finally, linear regression or other machine learning methods can be used to establish a shape space mapping relationship between the skin information in the second human internal anatomical model and the skin information in the skinned multi-person linear model.

[0067] After obtaining the shape space mapping relationship between the skin information in the second human internal anatomical model and the skin information in the skinned multi-person linear model, the internal organs of the second human internal anatomical model can be mapped to the skinned multi-person linear model, thereby obtaining the first human internal anatomical model.

[0068] Here, the shape space consistency of statistical shape models can be utilized. Shape space consistency refers to the fact that objects belonging to the same category should have a certain similarity in shape space. Therefore, the vertex coordinates of the internal organs of the second human internal anatomy model can be projected into their corresponding shape space. Then, these coordinates can be transformed into the shape space of the skinned multi-person linear model using shape space mapping relationships. Finally, the transformed coordinates can be used as the vertex coordinates of the corresponding internal organs in the skinned multi-person linear model.

[0069] This completes the mapping of internal organs from the second human internal anatomy model to the skinned multi-person linear model. The mapped model is called the first human internal anatomy model. Since the first human internal anatomy model and the skinned multi-person linear model have the same parameterization method and vertex correspondence, the parameters of the skinned multi-person linear model can be directly used to control the shape and posture of the first human internal anatomy model.

[0070] The method provided in this invention transfers statistically significant internal organ information from a second human internal anatomical model to a skinned multi-person linear model, resulting in a first human internal anatomical model. This allows us to control the shape and orientation of the first human internal anatomical model using the parameters of the skinned multi-person linear model, thereby achieving personalized internal organ reconstruction. Furthermore, since the first human internal anatomical model and the skinned multi-person linear model share the same parameterization method and vertex correspondence, the target parameters of the skinned multi-person linear model output by the deep learning network can be directly used to locate the positions of internal organs, thus achieving fast and objective parameterization based on appearance data.

[0071] Based on the above embodiments, the method further includes: Step 410: If the organ to be located is not of a preset type, obtain the reference anatomical location of the organ to be located in the first human internal anatomical model. Step 420: Determine the target navigation positioning position based on the first human internal anatomical model, the reference anatomical location, and the offset between the organ to be located and the reference anatomical location.

[0072] Specifically, in this embodiment of the invention, several preset types of organs to be located are pre-defined. For example, bones, liver, spleen, and kidneys can be preset as preset types. For these preset types of organs, they can be directly located using a first human internal anatomical model. However, if the organ to be located does not belong to a preset type, for example, if the organ to be located is an organ that does not have or cannot be statistically constructed, then a new method is required for location.

[0073] In this step, it is first determined whether the organ to be located belongs to a preset type. If the organ to be located does not belong to a preset type, then the reference anatomical location of the organ to be located in the first human internal anatomical model is obtained. The reference anatomical location refers to the position of an easily located anatomical structure in the first human internal anatomical model that has a clear anatomical relationship with the organ to be located. For example, if the organ to be located is the gallbladder, the liver can be selected as the reference anatomical location; if the organ to be located is the pancreas, the duodenum can be selected as the reference anatomical location. This embodiment of the invention does not specifically limit this.

[0074] After obtaining the reference anatomical location, the target navigation location can be determined based on the first human internal anatomical model, the reference anatomical location, and the offset between the organ to be located and the reference anatomical location. The offset refers to the positional difference of the organ to be located relative to the reference anatomical location.

[0075] The method provided in this invention, when the organ to be located is not of a preset type, obtains the reference anatomical location of the organ in a first human internal anatomical model, and then determines the target navigation location based on the first human internal anatomical model, the reference anatomical location, and the offset between the organ to be located and the reference anatomical location. This achieves the determination of the target navigation location based on the reference anatomical location and the offset when the organ to be located is not of a preset type. This method can expand the application scope of this invention, enabling it to be applied to the location of more types of organs. Furthermore, because this method utilizes the anatomical relationships between organs, it can improve the accuracy and robustness of the location.

[0076] In summary, the method provided by this invention has the following advantages: (1) Real-time performance: Deep learning algorithms predict human body parameters quickly. Data set 1 is associated with the human body parameterization model SMPL to achieve the parameterization of the shape of the internal anatomical organs of the human body with the same parameters as SMPL. The parameterization model reconstructs the human body quickly (if there are parameters, the shape of the model can be obtained directly. The implementation method is essentially the addition of vector data).

[0077] (2) Comprehensiveness: It realizes the reconstruction of three-dimensional meshes from images to complete human skin, bones and other internal organs.

[0078] (3) Accuracy: The skin grid (digital human) is used as an intermediate representation, and a large amount of human bone and other organ and skin data are used to obtain prior knowledge.

[0079] (4) Rich application scenarios: It can be used in medical-related scenarios such as anatomy teaching and assisting intelligent robots in anatomical structure positioning and navigation.

[0080] The input of this invention is an RGB image, an RGBD image, or a real-time video stream, and the output includes information such as a three-dimensional human skeletal mesh, organ locations, and carotid artery scanning paths.

[0081] In terms of automated scanning, this invention utilizes anatomical reconstruction technology to accurately locate organ positions (such as the carotid artery), thereby enabling the robot to automatically plan the scanning path based on anatomical information, achieving efficient and accurate scanning of the specified organ.

[0082] In the field of anatomy teaching, this invention offers significant advantages. When a camera is pointed at a human body to capture an image, the system can reconstruct the internal anatomical structures in real time and accurately overlay the reconstructed anatomical information onto the current video frame. This function provides an intuitive and real-time display of anatomical structures for medical teaching, clinical practice, and other scenarios, helping to improve teaching effectiveness and clinical diagnostic efficiency.

[0083] Based on any of the above embodiments Figure 4This is a schematic diagram of the process for constructing the first human internal anatomical model provided by the present invention, as shown below. Figure 4 As shown, firstly, human body structures are segmented and reconstructed in three dimensions using medical image datasets (such as CT scan data) to obtain statistical models of the skin and one or more internal organs. This statistical model is denoted as M1, which is the second internal anatomical model of the human body. At the same time, based on the SMPL model dataset containing only the human body surface, a corresponding parametric skin model (skinned multi-person linear model) is constructed, denoted as M2.

[0084] Subsequently, using the shape space mapping relationship between the skin information in M1 (the second human internal anatomy model) and the skin information in M2 (SMPL skin model), as well as the shape space consistency of the statistical shape model, the internal organs of the second human internal anatomy model are mapped to the skinned multi-person linear model to obtain the first human internal anatomy model M1-X.

[0085] The first human internal anatomy model M1-X has the same parameterization format as M2, thus ensuring that the two models can share parameters and achieve accurate real-time dynamic changes.

[0086] Ultimately, based on the above model construction process and parameter association, high-precision, real-time personalized digital human anatomical organ reconstruction and navigation positioning were achieved, facilitating real-time rendering, interaction, and operation on application terminals (such as the Windows platform).

[0087] Furthermore, by utilizing image and graphics technologies and game engines (such as Unity), the M2 and M1-X models were constructed within a virtual environment. This allows us to package the predicted 3D human skin, bones, arterial meshes, and other tissues or organs onto various edge platforms, including head-mounted displays, glasses-free 3D displays, tablets, and 2D display computers, through the game engine. This enables users to see personalized human anatomical organs displayed on RGB / RGBD images of the human body in a real-time, intuitive, and dynamic manner. This not only enhances the visual experience but also allows users to gain a deeper understanding of human anatomy through interactive operations, thus playing a significant role in multiple fields such as medical education, surgical planning, and patient communication.

[0088] The navigation and positioning device based on personalized digital human anatomical organ reconstruction provided by the present invention will be described below. The navigation and positioning device based on personalized digital human anatomical organ reconstruction described below can be referred to in correspondence with the navigation and positioning method based on personalized digital human anatomical organ reconstruction described above.

[0089] Based on any of the above embodiments, the present invention provides a navigation and positioning device based on personalized digital human anatomical organ reconstruction. Figure 5 This is a schematic diagram of the navigation and positioning device based on personalized digital human anatomical organ reconstruction provided by the present invention, as shown below.Figure 5 As shown, the method includes: The acquisition unit 510 is used to acquire human features from human images and / or human video streams, and input the human images and / or human video streams into a deep learning network to obtain the target parameters of the skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters are used to characterize the body shape features of the human body; the pose parameters are used to characterize the posture features of the human body; the virtual camera parameters are used to characterize the intrinsic and extrinsic parameters of the virtual camera in the image digital space; The first position unit 520 is used to obtain the position index of the organ to be located, obtain the target vertex corresponding to the position index in the first human internal anatomy model, and determine the first position of the target vertex in the first human internal anatomy model based on the target parameters; the first human internal anatomy model is a human internal anatomy model with the same parameterization as the skinned multi-person linear model. The target navigation and positioning unit 530 is used to transform the first position to the second position of the human body image based on the spatial transformation of the virtual camera parameters, and to determine the target navigation and positioning position based on the second position.

[0090] The apparatus provided in this invention directly acquires human features from human images and / or human video streams through a deep learning network, outputting target parameters for a skinned multi-person linear model, including shape parameters, pose parameters, and virtual camera parameters. This achieves rapid and objective parameterization of the human model based on appearance data. Furthermore, the shape parameters accurately depict the human body shape, the pose parameters precisely capture the human posture, and the virtual camera parameters accurately describe the intrinsic and extrinsic parameters of the virtual camera in the digital image space. Based on the target parameters and combined with a first internal anatomical model of the human body, the first position of the target vertex in the first internal anatomical model can be quickly located. Through spatial transformation of the virtual camera parameters, this position is mapped to a second position in the human image, thereby determining the target navigation and positioning location. This process eliminates the need for manual modification of model parameters, directly utilizing deep learning technology to efficiently predict internal organ structures based on human appearance images and / or human video streams. It effectively solves the technical problems of relying on manual adjustment of model parameters and lacking the ability to predict internal structures based on appearance data in personalized three-dimensional human anatomical reconstruction, greatly improving the efficiency and accuracy of three-dimensional human anatomical reconstruction and providing more reliable technical support for fields such as medical navigation and positioning.

[0091] Based on any of the above embodiments, a model building unit is further included, wherein the model building unit is specifically used for: Obtain the shape space mapping relationship between the skin information in the second human internal anatomical model and the skin information in the skinned multi-person linear model; Based on the shape space consistency of the statistical shape model and the shape space mapping relationship, the internal organs of the second human internal anatomy model are mapped to the skinned multi-person linear model to obtain the first human internal anatomy model.

[0092] Based on any of the above embodiments, a model determination unit is further included, wherein the model determination unit is specifically used for: Human structure segmentation and 3D reconstruction were performed on N medical image data to obtain N sets of initial human skin and internal organ anatomy reconstruction models; Based on shape statistics techniques, the average statistical shape and high-dimensional shape space of the human body structure model are obtained from N sets of reconstructed models; the average statistical shape represents the average human body shape of the samples covered by N medical image data; the high-dimensional shape space represents the group-specific human body shape characteristics of the samples covered by N medical image data. Based on the average statistical model and the high-dimensional shape space, the second human internal anatomical model is determined.

[0093] Based on any of the above embodiments, a reference anatomical positioning determination unit is further included, wherein the reference anatomical positioning determination unit is specifically used for: If the organ to be located is not of a preset type, obtain the reference anatomical location of the organ to be located in the first human internal anatomical model; The target navigation and positioning position is determined based on the first human internal anatomical model, the reference anatomical location, and the offset between the organ to be located and the reference anatomical location.

[0094] Based on any of the above embodiments, the human body image includes an RGB human body image and / or a human body depth image; the deep learning network includes a feature extraction encoder, a target parameter decoder, and a digital human mapping module; The step of inputting the human body features into a deep learning network to obtain the target parameters of the skinned multi-person linear model output by the deep learning network includes: The first human body feature corresponding to the RGB human body image and the second human body feature corresponding to the human body depth image are input into the feature extraction encoder to obtain human body features; The human body features are input into the target parameter decoder to obtain the target parameters output by the target parameter decoder; The target parameters include human pose decoding, the shape parameters, and the virtual camera parameters; the human pose decoding includes the human joint positions of the digital human and the pose parameters, wherein the pose parameters are represented by rotation matrices, angles, and quaternions.

[0095] Based on any of the above embodiments, a mapping unit is further included, wherein the mapping unit is specifically used for: The pose parameters, shape parameters, and virtual camera parameters are input into the digital human mapping module to obtain the first human internal anatomical model and the skinned multi-person linear model reconstructed based on the input image of the deep learning network, which are output by the digital human mapping module.

[0096] Based on any of the above embodiments, the target navigation and positioning unit 530 is specifically used for: Based on the intrinsic and extrinsic parameters of the real camera hardware, combined with the depth information of the human body image and / or the depth information of the human body video stream, the second position is transformed to a real-world spatial position based on the real camera hardware through spatial transformation, thereby obtaining the target navigation and positioning position.

[0097] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logical instructions in the memory 630 to execute a navigation and positioning method based on personalized digital human anatomical organ reconstruction. This method includes: acquiring human features from human images and / or human video streams, and inputting the human images and / or human video streams into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters characterize the body shape features of the human body; the pose parameters characterize the posture features of the human body; the virtual camera parameters characterize the intrinsic and extrinsic parameters of the virtual camera in the image digital space; acquiring the position index of the organ to be located, obtaining the target vertex corresponding to the position index in a first human internal anatomical model, and determining the first position of the target vertex in the first human internal anatomical model based on the target parameters; the first human internal anatomical model is a human internal anatomical model with the same parameterization as the skinned multi-person linear model; transforming the first position to a second position in the human image based on the spatial transformation of the virtual camera parameters, and determining the target navigation and positioning position based on the second position.

[0098] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the navigation and positioning method based on personalized digital human anatomical organ reconstruction provided by the above methods. The method includes: acquiring human features from human images and / or human video streams, and inputting the human images and / or human video streams into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters are used to characterize the human body. The body shape characteristics; the posture parameters are used to characterize the posture characteristics of the human body; the virtual camera parameters are used to characterize the intrinsic and extrinsic parameters of the virtual camera in the digital space of the image; the position index of the organ to be located is obtained, the target vertex corresponding to the position index is obtained in the first human internal anatomy model, and the first position of the target vertex in the first human internal anatomy model is determined based on the target parameters; the first human internal anatomy model is a human internal anatomy model with the same parameterization as the skinned multi-person linear model; based on the spatial transformation of the virtual camera parameters, the first position is transformed to the second position of the human body image, and the target navigation and positioning position is determined based on the second position.

[0100] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the navigation and positioning method based on personalized digital human anatomical organ reconstruction provided by the above methods. This method includes: acquiring human features from human images and / or human video streams, and inputting the human images and / or the human video streams into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters characterize the body shape features of the human body; the pose parameters characterize the posture features of the human body; the virtual camera parameters characterize the intrinsic and extrinsic parameters of a virtual camera in the image digital space; acquiring the position index of the organ to be located, obtaining the target vertex corresponding to the position index in a first human internal anatomical model, and determining a first position of the target vertex in the first human internal anatomical model based on the target parameters; the first human internal anatomical model is a human internal anatomical model with the same parameterization as the skinned multi-person linear model; transforming the first position to a second position in the human image based on the spatial transformation of the virtual camera parameters, and determining the target navigation and positioning position based on the second position.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A navigation and positioning method based on personalized digital human anatomical organ reconstruction, characterized in that, include: The system acquires human features from human images and / or human video streams, and inputs the human images and / or human video streams into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters are used to characterize the body shape features of the human body; The posture parameters are used to characterize the posture characteristics of the human body; the virtual camera parameters are used to characterize the intrinsic and extrinsic parameters of the virtual camera in the image digital space; Obtain the location index of the organ to be located, obtain the target vertex corresponding to the location index in the first human internal anatomy model, and determine the first position of the target vertex in the first human internal anatomy model based on the target parameters; the first human internal anatomy model is a human internal anatomy model with the same parameterization as the skinned multi-person linear model. Based on the spatial transformation of the virtual camera parameters, the first position is transformed to the second position of the human body image, and the target navigation and positioning position is determined based on the second position.

2. The navigation and positioning method based on personalized digital human anatomical organ reconstruction according to claim 1, characterized in that, The steps for constructing the first human internal anatomical model include: Obtain the shape space mapping relationship between the skin information in the second human internal anatomical model and the skin information in the skinned multi-person linear model; Based on the shape space consistency of the statistical shape model and the shape space mapping relationship, the internal organs of the second human internal anatomy model are mapped to the skinned multi-person linear model to obtain the first human internal anatomy model.

3. The navigation and positioning method based on personalized digital human anatomical organ reconstruction according to claim 2, characterized in that, The steps for determining the second human internal anatomical model include: Human structure segmentation and 3D reconstruction were performed on N medical image data to obtain N sets of initial human skin and internal organ anatomy reconstruction models; Based on shape statistics techniques, the average statistical shape and high-dimensional shape space of the human body structure model are obtained from N sets of reconstructed models; the average statistical shape represents the average human body shape of the samples covered by N medical image data; the high-dimensional shape space represents the group-specific human body shape characteristics of the samples covered by N medical image data. Based on the average statistical model and the high-dimensional shape space, the second human internal anatomical model is determined.

4. The navigation and positioning method based on personalized digital human anatomical organ reconstruction according to any one of claims 1 to 3, characterized in that, The method further includes: If the organ to be located is not of a preset type, obtain the reference anatomical location of the organ to be located in the first human internal anatomical model; The target navigation and positioning position is determined based on the first human internal anatomical model, the reference anatomical location, and the offset between the organ to be located and the reference anatomical location.

5. The navigation and positioning method based on personalized digital human anatomical organ reconstruction according to any one of claims 1 to 3, characterized in that, The human body image includes an RGB human body image and / or a human body depth image; the deep learning network includes a feature extraction encoder, a target parameter decoder, and a digital human mapping module. The step of inputting the human body features into a deep learning network to obtain the target parameters of the skinned multi-person linear model output by the deep learning network includes: The first human body feature corresponding to the RGB human body image and the second human body feature corresponding to the human body depth image are input into the feature extraction encoder to obtain human body features; The human body features are input into the target parameter decoder to obtain the target parameters output by the target parameter decoder; The target parameters include human pose decoding, the shape parameters, and the virtual camera parameters; the human pose decoding includes the human joint positions of the digital human and the pose parameters, wherein the pose parameters are represented by rotation matrices, angles, and quaternions.

6. The navigation and positioning method based on personalized digital human anatomical organ reconstruction according to claim 5, characterized in that, The step of inputting the human body features into the target parameter decoder to obtain the target parameters output by the target parameter decoder further includes: The pose parameters, shape parameters, and virtual camera parameters are input into the digital human mapping module to obtain the first human internal anatomical model and the skinned multi-person linear model reconstructed based on the input image of the deep learning network, which are output by the digital human mapping module.

7. The navigation and positioning method based on personalized digital human anatomical organ reconstruction according to any one of claims 1 to 3, characterized in that, Determining the target navigation and positioning location based on the second location includes: Based on the intrinsic and extrinsic parameters of the real camera hardware, combined with the depth information of the human body image and / or the depth information of the human body video stream, the second position is transformed to a real-world spatial position based on the real camera hardware through spatial transformation, thereby obtaining the target navigation and positioning position.

8. A navigation and positioning device based on personalized digital human anatomical organ reconstruction, characterized in that, include: An acquisition unit is used to acquire human features from human images and / or human video streams, and input the human images and / or human video streams into a deep learning network to obtain target parameters of a skinned multi-person linear model output by the deep learning network; the target parameters include shape parameters, pose parameters, and virtual camera parameters; the shape parameters are used to characterize the body shape features of the human body; The posture parameters are used to characterize the posture characteristics of the human body; the virtual camera parameters are used to characterize the intrinsic and extrinsic parameters of the virtual camera in the image digital space; A first position unit is determined to obtain the position index of the organ to be located, the target vertex corresponding to the position index is obtained in the first human internal anatomy model, and the first position of the target vertex in the first human internal anatomy model is determined based on the target parameters; the first human internal anatomy model is a human internal anatomy model with the same parameterization as the skinned multi-person linear model. A target navigation and positioning unit is used to transform the first position to a second position of the human body image based on the spatial transformation of the virtual camera parameters, and to determine the target navigation and positioning position based on the second position.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the navigation and positioning method based on personalized digital human anatomical organ reconstruction as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the navigation and positioning method based on personalized digital human anatomical organ reconstruction as described in any one of claims 1 to 7.

Images