Subject body index estimation method for medical imaging and medical imaging system

Abstract

The embodiment of the invention provides a subject body index estimation method for medical imaging, a computer readable storage medium and a medical imaging system. The method comprises the following steps: acquiring image data of a subject shot by a camera device; extracting a first feature vector and key point information of a preset position of the body of the subject from the image data; generating a second feature vector according to the key point information; and estimating at least one of the height and the weight of the subject according to the first feature vector and the second feature vector.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of medical equipment, and in particular to a method for estimating body indicators of a subject for medical imaging and a medical imaging system. Background Art

[0002] In clinical scanning, before medical imaging examinations, it is necessary to routinely record the age, height, weight and other indicators of the subject. For any medical imaging mode, the height and weight of the subject are important parameters. For example, in magnetic resonance imaging systems, height and weight parameters can be used for positioning, as well as to assist in the evaluation of the radio frequency energy absorption rate (SAR) value to prevent the tissue to be imaged from absorbing too much radio frequency energy in a short period of time, causing local burns or even causing greater safety problems. For example, for a computed tomography system, the appropriate tube current can be determined based on height and weight to minimize radiation dose, etc. Summary of the invention

[0003] Embodiments of the present application provide a method for estimating a body index of a subject for medical imaging and a medical imaging system.

[0004] According to one aspect of an embodiment of the present application, a method for estimating a body index of a subject for medical imaging is provided, the method comprising:

[0005] Acquiring image data of a subject captured by an imaging device;

[0006] extracting a first feature vector and key point information of a predetermined position of the subject's body from the image data;

[0007] Generate a second feature vector according to the key point information;

[0008] At least one of the height and weight of the subject is estimated based on the first eigenvector and the second eigenvector.

[0009] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the method for estimating body indicators of a subject for medical imaging described in the previous aspect is executed.

[0010] According to one aspect of an embodiment of the present application, a medical imaging system is provided, the system comprising:

[0011] An imaging device that captures image data of a subject;

[0012] A controller is connected to the camera device and is used to execute the subject's body index estimation method in the aforementioned aspect.

[0013] One of the beneficial effects of the embodiment of the present application is: extracting a first feature vector and key point information of a predetermined position of the subject's body from the image data captured by the camera device; generating a second feature vector based on the key point information; and estimating at least one of the height and weight of the subject by combining the first feature vector and the second feature vector. Thus, the height and weight of the human body can be estimated by combining the global first feature in the original image data and the second feature related to the human body morphology, that is, using multi-dimensional features to estimate the height and weight of the human body, so that the estimation result is more accurate.

[0014] With reference to the following description and accompanying drawings, the specific implementation of the embodiment of the present application is disclosed in detail, indicating the way in which the principle of the embodiment of the present application can be adopted. It should be understood that the implementation of the present application is not limited in scope. Within the scope of the spirit and clauses of the attached claims, the implementation of the present application includes many changes, modifications and equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without creative work. In the drawings:

[0016] Figure 1 is a schematic diagram of a magnetic resonance imaging system according to an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of a method for estimating a subject's body index for medical imaging according to an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of a medical imaging system according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of a magnetic resonance imaging system according to an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a subject's body index estimation device for medical imaging according to an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of a method for estimating a subject's body index for medical imaging according to an embodiment of the present application;

[0022] Figure 7 is a schematic diagram of a method for estimating a subject's body index for medical imaging according to an embodiment of the present application;

[0023] Figure 8 is a schematic diagram of a subject's body index estimation device for medical imaging according to an embodiment of the present application;

[0024] Fig. 9 is a schematic diagram of a convolutional neural network according to an embodiment of the present application;

[0025] Fig.10 It is a schematic diagram of key points of the predetermined positions of an embodiment of the present application. DETAILED DESCRIPTION

[0026] With reference to the accompanying drawings, the foregoing and other features of the embodiments of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the embodiments of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the embodiments of the present application include all modifications, variations and equivalents that fall within the scope of the attached claims.

[0027] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0028] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0029] Features described and / or shown for one embodiment may be used in one or more other embodiments in the same or similar manner, combined with features in other embodiments, or replace features in other embodiments. The term "includes / comprising" when used herein refers to the presence of a feature, an entirety, a step, or a component, but does not exclude the presence or addition of one or more other features, entireties, steps, or components.

[0030] The medical imaging systems described herein include, but are not limited to, computed tomography (CT) systems, magnetic resonance imaging (MRI) systems, C-arm imaging systems, positron emission tomography (PET) systems, single photon emission computed tomography (SPECT) systems, ultrasound systems, X-ray imaging systems, or any other suitable medical imaging systems.

[0031] The following description is made by taking a magnetic resonance imaging system as an example, but the embodiments of the present application are not limited thereto.

[0032] For ease of understanding, Figure 1 A magnetic resonance imaging (MRI) system 100 of some embodiments of the present invention is shown.

[0033] The MRI system 100 includes a scanning unit 111. The scanning unit 111 is used to perform magnetic resonance scanning on an object (eg, a human body) 170 to generate a reconstructed image of a region of interest of the object 170, where the region of interest may be a predetermined anatomical site or anatomical tissue.

[0034] The operation of the MRI system 100 is controlled by an operator workstation 110, which includes an input device 114, a control panel 116, and a display 118. The input device 114 can be a joystick, a keyboard, a mouse, a trackball, a touch-activated screen, voice control, or any similar or equivalent input device. The control panel 116 can include a keyboard, a touch-activated screen, voice control, buttons, a slider, or any similar or equivalent control device. The operator workstation 110 is coupled to and communicates with a computer system 120, which enables an operator to control the generation and viewing of images on the display 118. The computer system 120 includes multiple components that communicate with each other via electrical and / or data connection modules 122. The connection module 122 can be a direct wired connection, an optical fiber connection, a wireless communication link, etc. The computer system 120 can include a central processing unit (CPU) 124, a memory 126, and an image processor 128. In some embodiments, the image processor 128 can be replaced by image processing functions implemented in the CPU 124. The computer system 120 may be connected to an archival media device, permanent or backup storage, or a network. The computer system 120 may be coupled to and communicate with a separate MRI system controller 130 .

[0035] The MRI system controller 130 includes a set of components that communicate with each other via an electrical and / or data connection module 132. The connection module 132 can be a direct wired connection, an optical fiber connection, a wireless communication link, etc. The MRI system controller 130 can include a CPU 131, a sequence pulse generator 133 that communicates with the operator workstation 110, a transceiver (or RF transceiver) 135, a memory 137, and an array processor 139. In some embodiments, the sequence pulse generator 133 can be integrated into the resonance assembly 140 of the scanning unit 111 of the MRI system 100. The MRI system controller 130 can receive commands from the operator workstation 110, coupled to the scanning unit 111, to indicate the MRI scanning sequence to be performed during the MRI scan, so as to control the scanning unit 111 to perform the above-mentioned magnetic resonance scanning process. The MRI system controller 130 is also coupled to and communicates with a gradient driver system 150, which is coupled to the gradient coil assembly 142 to generate a magnetic field gradient during the MRI scan.

[0036] The sequence pulse generator 133 may also receive data from a physiological acquisition controller 155, which receives signals from a plurality of different sensors (such as electrocardiogram (ECG) signals from electrodes attached to the patient) connected to a subject or patient 170 undergoing an MRI scan. The sequence pulse generator 133 is coupled to and in communication with a scan room interface system 145, which receives signals from various sensors associated with the state of the resonance assembly 140. The scan room interface system 145 is also coupled to and in communication with a patient positioning system 147, which sends and receives signals to control the movement of the patient table to a desired position for an MRI scan.

[0037] The MRI system controller 130 provides gradient waveforms to the gradient driver system 150, which includes a G x (x direction), G y (y direction) and G z (z direction) amplifier, etc. Each G x , G y and G zThe gradient amplifiers all excite corresponding gradient coils in the gradient coil assembly 142 to generate magnetic field gradients for spatially encoding MR signals during MRI scanning. The gradient coil assembly 142 is disposed within a resonance assembly 140, which also includes a superconducting magnet having superconducting coils 144, which in operation provide a static uniform longitudinal magnetic field B0 throughout a cylindrical imaging volume 146. The resonance assembly 140 also includes an RF body coil 148, which in operation provides a transverse magnetic field B1 that is substantially perpendicular to B0 throughout the cylindrical imaging volume 146. The resonance assembly 140 may also include an RF surface coil 149 for imaging different anatomical structures of a patient undergoing an MRI scan. The RF body coil 148 and the RF surface coil 149 may be configured to operate in a transmit and receive mode, a transmit mode, or a receive mode.

[0038] The x direction can also be called the frequency encoding direction or the k direction in K space. x The y direction can be called the phase encoding direction or the k direction in K space. y Direction. x It can be used for frequency encoding or signal readout and is usually called frequency encoding gradient or readout gradient. y Can be used for phase encoding, usually called phase encoding gradient. G z It can be used for slice (layer) position selection to obtain K-space data. It should be noted that the slice selection direction, phase encoding direction and frequency encoding direction can be modified according to actual needs.

[0039] An MRI scanned subject or patient 170 may be positioned within a cylindrical imaging volume 146 of the resonance assembly 140. A transceiver 135 in the MRI system controller 130 generates RF excitation pulses that are amplified by an RF amplifier 162 and provided to an RF body coil 148 via a transmit / receive switch (T / R switch) 164.

[0040] As described above, the RF body coil 148 and the RF surface coil 149 can be used to transmit RF excitation pulses and / or receive the resulting MR signals from the patient undergoing the MRI scan. The MR signals emitted by the nuclei excited in the patient's body of the MRI scan can be sensed and received by the RF body coil 148 or the RF surface coil 149 and sent back to the preamplifier 166 through the T / R switch 164. The T / R switch 164 can be controlled by a signal from the sequence pulse generator 133 to electrically connect the RF amplifier 162 to the RF body coil 148 during the transmit mode, and to connect the preamplifier 166 to the RF body coil 148 during the receive mode. The T / R switch 164 can also enable the RF surface coil 149 to be used in the transmit mode or the receive mode.

[0041] In some embodiments, the MR signals sensed and received by the RF body coil 148 or the RF surface coil 149 and amplified by the preamplifier 166 are stored as a raw k-space data array in the memory 137 for post-processing. A reconstructed magnetic resonance image can be acquired by transforming / processing the stored raw k-space data.

[0042] In some embodiments, the MR signals sensed and received by the RF body coil 148 or the RF surface coil 149 and amplified by the preamplifier 166 are demodulated, filtered, and digitized in the receive portion of the transceiver 135 and transmitted to the memory 137 in the MRI system controller 130. For each image to be reconstructed, the data is rearranged into a separate k-space data array, and each of these separate k-space data arrays is input to the array processor 139, which is operated to Fourier transform the data into an array of reconstructed images.

[0043] The array processor 139 uses a transform method, most commonly a Fourier transform, to create images from the received MR signals. These images are transmitted to the computer system 120 and stored in the memory 126. In response to commands received from the operator workstation 110, the data used to reconstruct the image can be stored in long-term storage or can be further processed by the image processor 128 and transmitted to the operator workstation 110 for presentation on the display 118.

[0044] In various embodiments, components of computer system 120 and MRI system controller 130 may be implemented on the same computer system or multiple computer systems. Figure 1 The MRI system 100 shown is for illustration purposes. Suitable MRI systems may include more, fewer, and / or different components.

[0045] The MRI system controller 130 and the image processor 128 may include a computer processor and a storage medium, respectively or in common, on which a program for predetermined data processing to be executed by the computer processor is recorded, for example, the storage medium may store a program for implementing scanning processing (e.g., scanning process, imaging sequence), image reconstruction, image processing, etc. For example, a program for implementing the magnetic resonance imaging method of the embodiment of the present invention may be stored. The above-mentioned storage medium may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-volatile memory card.

[0046] The above-mentioned "imaging sequence" (hereinafter also referred to as scanning sequence or pulse sequence) refers to a combination of pulses with specific amplitude, width, direction and timing applied when performing magnetic resonance imaging scanning, and these pulses can generally include, for example, radio frequency pulses and gradient pulses. The radio frequency pulses can include, for example, radio frequency excitation pulses, radio frequency refocusing pulses, inversion recovery pulses, etc. The gradient pulses can include, for example, the above-mentioned gradient pulses for layer selection, gradient pulses for phase encoding, gradient pulses for frequency encoding, gradient pulses for phase offset (phase shift), gradient pulses for discrete phase (dephasing), etc.

[0047] Generally, a plurality of scanning sequences can be pre-set in a magnetic resonance system so that a sequence suitable for clinical detection requirements can be selected. The clinical detection requirements may include, for example, imaging site, imaging function, imaging effect, and the like.

[0048] At present, in medical imaging systems (such as magnetic resonance imaging systems or computer tomography imaging systems, etc.), a camera device can be introduced to obtain visual and morphological information of the human body. Usually, the camera device can be installed near the examination bed, so as to collect auxiliary information of the subject to the greatest extent in a non-contact manner. In the related art, a method for estimating the body shape information of the subject based on the 3D image obtained by the camera device is proposed.

[0049] In an embodiment of the present application, a method for estimating the physical index of a subject for medical imaging is proposed, comprising: extracting a first feature vector and key point information of a predetermined position of the subject's body from image data captured by a camera; generating a second feature vector based on the key point information; and estimating at least one of the height and weight of the subject by combining the first feature vector and the second feature vector. Thus, the height and weight of a human body can be estimated by combining the global first feature in the original image data and the second feature related to the human body morphology, that is, using multi-dimensional features to estimate the height and weight of a human body, so that the estimation result is more accurate.

[0050] The following is a description with reference to embodiments.

[0051] The present application provides a method for estimating a subject's body index for medical imaging. Figure 2 is a schematic diagram of a method for estimating a subject's body index for medical imaging according to an embodiment of the present application. Figure 2 As shown, the method includes:

[0052] 201, acquiring image data of a subject photographed by a camera device;

[0053] 202, extracting a first feature vector and key point information of a predetermined position of the subject's body from the image data;

[0054] 203, generating a second feature vector according to the key point information;

[0055] 204 : Estimate at least one of the height and weight of the subject according to the first eigenvector and the second eigenvector.

[0056] In some embodiments, the image data of the subject may be acquired by a camera device, and the image data may include two-dimensional optical image data and depth image data, but this is not intended to be limiting. For example, the camera device may be a 3D camera, which may capture the entire body of the subject in real time and generate a video stream, each frame in the video stream includes the two-dimensional optical image and the depth image, the two-dimensional optical image may be a two-dimensional RGB image, and each pixel value of the depth image reflects the distance between the camera and the corresponding position of the subject. The camera device may be installed on the ceiling or wall directly above the examination bed of the medical imaging system, and the embodiments of the present application are not intended to be limiting.

[0057] In the above, the camera device is taken as an example of a 3D camera, but the present application is not limited to this. For example, at least one of the above-mentioned two-dimensional optical image data and depth image data can also be obtained by an independent 2D optical camera and a depth sensor respectively, and examples are not given one by one here.

[0058] In some embodiments, after obtaining the two-dimensional optical image data and the depth image data, the two can be fused to generate image data. For example, the two-dimensional optical image data and the depth image data are fused to generate RGBD structured data as image data, which is convenient for subsequent processing and analysis. The fusion process includes aligning the two-dimensional optical image data and the depth image data (for example, using the perspective transformation principle), and then synthesizing the RGBD structured data. For details, please refer to the relevant technology, which will not be repeated here.

[0059] In some embodiments, in 202, a convolutional neural network may be used to extract a first feature vector from the image data, including inputting the image data into a convolutional neural network and using an output vector of the convolutional neural network as the first feature vector. Fig. 9 is a schematic diagram of a convolutional neural network according to an embodiment of the present application, such as Fig. 9As shown, the convolutional neural network includes at least a convolutional layer 901, a residual block layer 902, a pooling layer 903, and a fully connected layer 904, and each layer is composed of multiple processing nodes that can be called neurons. Each neuron in each layer can perform a processing function and pass the processed information to one of the multiple neurons in the downstream layer for further processing. For example, the neurons in the first layer can learn to recognize structural edges in image data. The neurons in the second layer can learn to recognize shapes based on the detected edges from the first layer, etc., which are not exemplified one by one here. For example, the convolutional neural network can adopt a ResNet model, a VGG model, a GoogleNet model, etc., and the embodiments of the present application are not limited to this.

[0060] In some embodiments, the above-mentioned image data of multiple volunteers can be collected in advance, and the convolutional neural network can be trained using the image data, and the trained convolutional neural network can be used to extract the first eigenvector. The first eigenvector can reflect (represent) at least one of the features such as edges, textures, shapes, and colors in the image data. The output dimension of the convolutional neural network can be W1×1. The value of W1 can be determined according to the convolutional neural network and actual needs. For example, W1=4096, but the embodiments of the present application are not limited to this.

[0061] In some embodiments, in addition to extracting the first eigenvector from the image data, key point information of a predetermined position of the subject's body can also be extracted from the image data, wherein the key point information of the predetermined position can be extracted from the image data center using a human posture estimation model, and the key point information can be a coordinate in a camera coordinate system or a coordinate in a medical imaging coordinate system, and the coordinate is a three-dimensional coordinate in space. That is, the image data is input into a pre-trained human posture estimation model, and the output of the human posture estimation model is the coordinates of the key points of multiple predetermined positions of the human body in the camera coordinate system or the medical imaging coordinate system. The coordinate origin of the medical imaging coordinate system can be the scanning center position (for example, the isocenter of a magnetic resonance imaging system), but the embodiments of the present application are not limited to this.

[0062] In some embodiments, the predetermined position may be a joint position on the human body with a certain degree of freedom, or an anatomical part of the human body that is easily identifiable in the image data, for example, including at least two of the top of the head, shoulders, nose, eyes, ears, arms, elbows, wrists, hips, knees, ankles, etc. For a predetermined position, if there are two symmetrical body positions on the left and right, at least one of the two symmetrical body positions on the left and right may be included. For example, the predetermined position may include, but is not limited to, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, left upper arm, right upper arm, left forearm, right forearm, left thigh, right thigh, left calf, right calf, top of the head, head, neck, heart, abdomen, pelvis. The embodiment of the present application does not limit the number of the predetermined positions.

[0063] In some embodiments, the human posture estimation model is implemented based on a deep learning algorithm, and specific reference can be made to relevant technologies, such as using a densepose model, an openpose model, an HRNet model, etc. to extract key point information from image data, wherein the aforementioned pre-collected image data of multiple volunteers can be used as an input parameter set, and the pre-calibrated key point information of multiple predetermined positions corresponding to each image data can be used as an output parameter set, the input parameter set and the output parameter set are used to train the human posture estimation model, and the trained human posture estimation model is used to extract key point information.

[0064] In some embodiments, the human body posture estimation model extracts the coordinates (two-dimensional pixel coordinates) of the key points at predetermined positions in the image coordinate system from the image data, wherein the camera parameters of the imaging device can be determined, wherein the camera parameters include camera intrinsic parameters and extrinsic parameters, wherein the intrinsic parameters include the focus, the physical imaging plane, and the distance between the optical center (i.e., the focal length f), and the extrinsic parameters include a rotation matrix, a scaling parameter, and a translation vector. According to the camera parameters, the two-dimensional pixel coordinates can be converted into the three-dimensional space coordinates of the camera coordinate system or the three-dimensional space coordinates of the medical imaging coordinate system to obtain the key point information. For the coordinate conversion, reference can be made to the relevant technology, which will not be repeated here.

[0065] In some embodiments, in 203, a second feature vector may be generated based on the key point information. As described above, the key point reflects the anatomical structure of the human body. Therefore, a morphological feature may be determined based on the position of the anatomical structure as the second feature vector. The second feature vector may reflect (represent) the distance or proportional relationship between the anatomical structures of the human body. The following describes how to generate the second feature vector based on the key point information.

[0066] In some embodiments, the distance between at least two pairs (W2 pairs) of key points can be calculated based on the key point information; the second feature vector is generated based on the distance. Wherein, the output dimension of the second feature vector can be W2×1, W2 is greater than or equal to 2, W2 pairs of key points are selected from the key points at the predetermined position, each pair of key points includes two key points in the length direction of the human body or two key points in the width direction, and the distance between each pair of key points is calculated. The distance can be the distance in the length direction of the human body or the distance in the width direction, wherein it can be obtained by calculating the coordinate difference of each pair of key points in the length direction or the width direction. The W2 distances are distances with no overlapping areas at all, or distances with overlapping areas, and the present application is not limited to this. The calculated W2 distances between the W2 pairs of key points are arranged according to a predetermined rule to generate the second feature vector. Through the above embodiment, since the height and weight of a person are related to the distance or the proportional relationship between the distances between the key points, the accuracy of height and weight estimation can be improved based on the above second feature vector.

[0067] In some embodiments, when each pair of key points includes two key points in the length direction of the human body, the two key points may be key points on the same side of the body (left or right side); when each pair of key points includes two key points in the width direction of the human body, the two key points may be left and right key points of the same anatomical part, but the embodiments of the present application are not limited to this.

[0068] For example, for the aforementioned key points at multiple predetermined positions, W2=10 pairs of key points can be selected. Fig.10 is a schematic diagram of key points of the predetermined positions of the embodiment of the present application, such as Fig.10As shown, the 10 pairs of key points are (top of the head, left shoulder), (left shoulder, left elbow), (left elbow, left wrist), (left shoulder, left hip), (left hip, left knee), (left knee, left ankle), (top of the head, ankle), (top of the head, left hip), (left shoulder, right shoulder), (left thigh, right thigh). The distance L1 between the top of the head and the shoulder in the length direction is calculated based on the top of the head coordinates and the left shoulder coordinates. The distance L2 between the shoulder and the elbow in the length direction is calculated based on the left shoulder coordinates and the left elbow coordinates. The distance L3 between the elbow and the wrist in the length direction is calculated based on the left elbow coordinates and the left wrist coordinates. The distance L4 between the shoulder and the hip in the length direction is calculated based on the left hip coordinates and the left knee coordinates, the distance L5 between the hip and the knee in the length direction is calculated based on the left knee coordinates and the left ankle coordinates, the distance L6 between the knee and the ankle in the length direction is calculated based on the left knee coordinates and the left ankle coordinates, the distance L7 between the head and the ankle in the length direction is calculated based on the head top coordinates and the left ankle coordinates, the distance L8 between the head top and the hip in the length direction is calculated based on the head top coordinates and the left hip coordinates, the distance L9 between the left shoulder and the right shoulder in the width direction is calculated based on the left shoulder coordinates and the right shoulder coordinates, and the distance L10 between the left thigh and the right thigh in the width direction is calculated based on the left thigh coordinates and the right thigh coordinates. The above L1~L3, L5~L6 are independent distances that do not overlap with each other, L4, L7, L8 and L1~L3, L5~L6 have overlapping areas, L1~L8 are distances in the length direction, and L9~L10 are distances in the width direction. The output dimension of the second feature vector may be W2×1, that is, the vector (L1, L2, L3, L4, L5, L6, L7, L8, L9, L10)×1 generated by arranging L1, L2, L3, L4, L5, L6, L7, L8, L9, L10 is used as the second feature vector. The above is only an example, and the embodiments of the present application are not limited thereto.

[0069] It should be noted that the above distance refers to the distance in the medical imaging coordinate system, that is, the distance reflects the real distance between the anatomical parts of the human body. When the key point information is the three-dimensional space coordinate of the camera coordinate system, it is necessary to combine the camera parameters, perform coordinate conversion, and calculate the corresponding distance in the medical imaging coordinate system. For details, please refer to the relevant technology, which will not be repeated here.

[0070] In some embodiments, in 204, at least one of the height and weight of the subject can be estimated based on the first eigenvector and the second eigenvector, wherein at least one of the height and weight of the subject can be estimated based only on the first eigenvector and the second eigenvector, or at least one of the height and weight of the subject can be estimated based on the first eigenvector, the second eigenvector and a third eigenvector.

[0071] In some embodiments, in 204, an input feature vector may be generated based only on the splicing of the first feature vector and the second feature vector. Wherein, splicing the first feature vector and the second feature vector includes cascading the first feature vector and the second feature vector, and the cascading method may be that all the first feature vectors are in front and all the second feature vectors are in the back, or all the second feature vectors are in front and all the first feature vectors are in the back, or the first feature vector and the second feature vector are arranged in a cross-spaced manner, and the embodiment of the present application is not limited to this. For example, the dimension of the first feature vector is W1×1, and the dimension of the second feature vector is W2×1, then the dimension of the input feature vector is (W1+W2)×1.

[0072] In some embodiments, a deep learning algorithm or a machine learning algorithm can be used to estimate at least one of the height and weight of the subject, wherein the concatenated input feature vector can be input into one or two regressors to obtain at least one of the height and weight of the subject.

[0073] In some embodiments, the input feature vector can be input into a regressor to estimate the height and weight of the subject at the same time, or the input feature vector can be input into a height regressor to estimate the height of the subject, or the input feature vector can be input into a weight regressor to estimate the weight of the subject, or the input feature vector can be input into a weight regressor and a height regressor respectively to estimate the weight and height of the subject, and the present application is not limited to this.

[0074] In some embodiments, the above-mentioned regressor can be a support vector based regressor (SVR), that is, the intrinsic relationship of the input feature vector is determined by the regressor, and various types of data in the input feature vector are fitted to predict birth height and weight. For the principle of SVR, reference can be made to the relevant technology, and the embodiments of the present application are not limited to this. In addition, the regressor of the present application is not limited to SVR. For example, it can also be a linear regressor, etc., which will not be exemplified one by one here. Alternatively, other deep learning algorithms can be used instead of a regressor, and the embodiments of the present application are not limited to this.

[0075] In some embodiments, one or both regressors may be pre-trained. For example, for a height regressor, the aforementioned method may be used to obtain input feature vectors of the aforementioned multiple volunteers, and the heights calibrated by the volunteers may be used as outputs to train the height regressor. For a weight regressor, the aforementioned method may be used to obtain input feature vectors of the aforementioned multiple volunteers, and the weights calibrated by the volunteers may be used as outputs to train the weight regressor.

[0076] In some embodiments, before inputting the data into the regressor, the data may be preprocessed for standardization. For example, the input feature vector may be passed through a fully connected layer before being input into the regressor. If there are two different regressors, the input feature vector also needs to be input into the fully connected layers corresponding to each regressor before being input into the regressor.

[0077] Figure 7 is a schematic diagram of a method for estimating a subject's body index for medical imaging according to an embodiment of the present application. Figure 7 As shown, the method includes:

[0078] 701, acquiring image data of a subject photographed by a camera device;

[0079] 702, extracting a first feature vector and key point information of a predetermined position of the subject's body from the image data;

[0080] 703, generating a second feature vector according to the key point information;

[0081] 704, generating a third feature vector according to the inherent information of the subject;

[0082] 705 : Estimate at least one of the height and weight of the subject according to the first eigenvector, the second eigenvector, and the third eigenvector.

[0083] In some embodiments, the implementations of 701 to 703 can refer to the aforementioned implementations and will not be repeated here. In 705, the difference from 204 is that the method of generating the input feature vector is different, that is, in 705, the input feature vector can be generated by splicing the first feature vector, the second feature vector and the third feature vector, wherein the inherent information of the subject, especially the gender and age, is directly associated with the height and weight of the subject. Therefore, in 704, the third feature vector can be generated according to the inherent information of the subject, and the inherent information includes at least one of age and gender. For example, the gender and age of the subject can be extracted from the hospital database, the gender and age can be encoded, and a third feature vector with a dimension of W3×1 can be generated. The value of W3 is related to the number of categories of the inherent information. When only gender or age is extracted, W3=1, and when gender and age are extracted, W3=2. In addition, the inherent information can also include other types of human attributes, and the embodiments of the present application are not limited to this. Concatenating the first feature vector with the second feature vector and the third feature vector includes cascading the first feature vector, the second feature vector and the third feature vector (the cascading order is not limited). For example, the dimension of the first feature vector is W1×1, the dimension of the second feature vector is W2×1, and the dimension of the third feature vector is W3×1, then the dimension of the input feature vector is (W1+W2+W3)×1.

[0084] It is worth noting that the above Figure 7 The embodiments of the present application are only schematically described, but the present application is not limited thereto. For example, the execution order of the various operations can be appropriately adjusted, and other operations can be added or some operations can be reduced. Those skilled in the art can make appropriate modifications based on the above content, and are not limited to the above appended Figure 7 Records of.

[0085] Through the above embodiment, a first feature vector and key point information of a predetermined position of the subject's body are extracted from the image data captured by the camera device; a second feature vector is generated based on the key point information; and at least one of the height and weight of the subject is estimated by combining the first feature vector and the second feature vector. Thus, the height and weight of the human body can be estimated by combining the global first feature in the original image data and the second feature related to the human body morphology, that is, the height and weight of the human body are estimated using multi-dimensional features, so that the estimation result is more accurate.

[0086] The present application also provides a medical imaging system. Figure 3 is a schematic diagram of a medical imaging system according to an embodiment of the present application. Figure 3 As shown, the system 300 includes:

[0087] An imaging device 301 for capturing image data of a subject;

[0088] A controller 302 is connected to the camera 301 and is configured to: receive the image data from the camera; extract a first feature vector and key point information of a predetermined position of the subject's body from the image data; generate a second feature vector based on the key point information; and estimate at least one of the height and weight of the subject based on the first feature vector and the second feature vector.

[0089] The implementation of the camera device 301 and the controller 302 can refer to the above embodiments, which will not be repeated here. The medical imaging system includes but is not limited to a computed tomography (CT) system, a magnetic resonance imaging (MRI) system, a C-arm imaging system, a positron emission tomography (PET) system, a single photon emission computed tomography (SPECT) system, an ultrasound system, an X-ray imaging system, or any other suitable medical imaging system.

[0090] In some embodiments, the controller 302 can be configured separately from the controller of the medical imaging system itself. For example, the controller 302 can be configured as a chip connected to the controller of the medical imaging system, and the two can control each other. Alternatively, the functions of the controller 302 can be integrated into the controller of the medical imaging system itself. The embodiments of the present application are not limited to this.

[0091] In some embodiments, the controller 302 includes a computer processor and a storage medium, on which a program for predetermined data processing to be executed by the computer processor is recorded. For example, the storage medium may store a program for implementing the estimation of the subject's body indicators for medical imaging. The storage medium may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-volatile memory card.

[0092] The medical imaging system may also include other structural components not shown in the figure, and the relevant technology may be referred to for details. The embodiments of the present application are not limited to this. Taking the magnetic resonance imaging system as an example, Figure 4 is a schematic diagram of a magnetic resonance imaging system according to an embodiment of the present application, such as Figure 4 As shown, Figure 1 The difference is that the magnetic resonance imaging system also includes a camera 41 for capturing image data of the subject, for example, the image data includes two-dimensional optical image data and depth image data. In addition, its controller 130 or computer system 120 is connected to the camera 41 and is configured to: receive the image data from the camera; extract a first feature vector and key point information of a predetermined position of the subject's body from the image data; generate a second feature vector based on the key point information; estimate at least one of the height and weight of the subject based on the first feature vector and the second feature vector. The specific implementation method can refer to the aforementioned embodiment and will not be repeated here.

[0093] The present application also provides a device for estimating a body index of a subject for medical imaging. Figure 5 Schematic diagram of a device for estimating a body index of a subject for medical imaging according to an embodiment of the present application. Figure 5 As shown, the device 500 includes:

[0094] An acquisition unit 501 acquires image data of a subject captured by an imaging device;

[0095] A first extraction unit 502 extracts a first feature vector from the image data;

[0096] A second extraction unit 503 extracts key point information of a predetermined position of the subject's body from the image data;

[0097] A generating unit 504 generates a second feature vector according to the key point information;

[0098] The estimating unit 505 estimates at least one of the height and weight of the subject according to the first eigenvector and the second eigenvector.

[0099] Figure 8 Schematic diagram of a device for estimating a body index of a subject for medical imaging according to an embodiment of the present application. Figure 8 As shown, the device 800 includes:

[0100] An acquisition unit 801 acquires image data of a subject captured by an imaging device;

[0101] A first extraction unit 802 extracts a first feature vector from the image data;

[0102] A second extraction unit 803 extracts key point information of a predetermined position of the subject's body from the image data;

[0103] A generating unit 804 generates a second feature vector according to the key point information;

[0104] A determination unit 805 generates a third feature vector according to the inherent information of the subject;

[0105] The estimating unit 806 estimates at least one of the height and weight of the subject according to the first eigenvector, the second eigenvector, and the third eigenvector.

[0106] The implementation of the above-mentioned unit modules can refer to 201 to 204, and the repeated parts will not be repeated.

[0107] The following combination Figure 6 The process of estimating the physical indicators of the subject by each of the above-mentioned unit modules is explained.

[0108] like Figure 6As shown, the camera device captures image data of the subject, and the acquisition unit 501, 801 acquires the image data, which is respectively input into the first extraction unit 502, 802 and the second extraction unit 503, 803. The first extraction unit 502, 802, for example, uses a convolutional neural network to extract a first feature vector, and the second extraction unit 503, 803 uses a human posture estimation model to extract key point information. The generation unit 504, 804 generates a second feature vector according to the key point information. The determination unit 805 can also obtain the inherent information of the subject from the database, and generate a third feature vector according to the inherent information. The estimation unit 505 cascades the first feature vector and the second feature vector to generate an input feature vector, or the estimation unit 806 cascades the first feature vector, the second feature vector and the third feature vector to generate an input feature vector. The input feature vector passes through the fully connected layer 61, the fully connected layer 62, and the height regressor 63 in sequence to predict the height. The input feature vector passes through the fully connected layer 61, the fully connected layer 64, and the weight regressor 65 in sequence to predict the weight.

[0109] An embodiment of the present application also provides a computer-readable program, wherein when the program is executed in a device or a medical imaging system, the program enables the computer to execute the method for estimating the body indicators of a subject for medical imaging described in the aforementioned embodiment in the device or the medical imaging system.

[0110] An embodiment of the present application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the method for estimating a body index of a subject for medical imaging described in the aforementioned embodiment in a device or a medical imaging system.

[0111] The above devices and methods of the present application can be implemented by hardware, or by hardware combined with software. The present application relates to such a computer-readable program, which, when executed by a logic component, enables the logic component to implement the above-mentioned devices or components, or enables the logic component to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0112] The method / device described in conjunction with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to the various steps shown in the figure, respectively. These hardware modules may be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0113] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if a device (such as a mobile terminal) uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0114] For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in the present application. For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0115] The present application is described above in conjunction with specific implementation methods, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles of the present application, and these modifications and variations are also within the scope of the present application.

Claims

1. A method for estimating a subject's body index for medical imaging, characterized in that: The method comprises: Acquiring image data of a subject captured by an imaging device; extracting a first feature vector and key point information of a predetermined position of the subject's body from the image data; Generate a second feature vector according to the key point information; At least one of the height and weight of the subject is estimated based on the first eigenvector and the second eigenvector.

2. The method according to claim 1, wherein: The first feature vector represents at least one feature of edge, texture, shape, and color in the image data.

3. The method according to claim 1, wherein: The second feature vector represents the distance between the anatomical structures of the subject or reflects the proportional relationship between the distances between the anatomical structures of the subject.

4. The method according to claim 1, wherein: The image data includes two-dimensional optical image data and depth image data.

5. The method according to claim 1, characterized in that: Extracting the first feature vector and key point information of a predetermined position of the subject's body from the image data includes: Extracting the first feature vector from the image data using a convolutional neural network; Key point information of predetermined positions of the subject's body is extracted from the image data using a human posture estimation model.

6. The method according to claim 1, characterized in that The predetermined positions include: at least two of the top of the head, shoulders, nose, eyes, ears, arms, elbows, wrists, hips, knees, and ankles.

7. The method according to claim 1, characterized in that Generating a second feature vector according to the key point information includes: Calculate the distance between at least two pairs of key points according to the key point information; The second feature vector is generated according to the distance.

8. The method according to claim 4, characterized in that Each pair of key points includes two key points in the length direction of the human body or two key points in the width direction.

9. The method according to claim 1, characterized in that: Estimating at least one of the height and weight of the subject according to the first feature vector and the second feature vector comprises: Concatenate the first feature vector and the second feature vector to generate an input feature vector; The input feature vector is input into one or two regressors to obtain at least one of the height and weight of the subject.

10. The method according to claim 1, characterized in that Estimating at least one of the height and weight of the subject according to the first feature vector and the second feature vector comprises: generating a third feature vector according to the inherent information of the subject; Concatenate the first feature vector, the second feature vector and the third feature vector to generate an input feature vector; The input feature vector is input into one or two regressors to obtain at least one of the height and weight of the subject.

11. The method according to claim 10, characterized in that The inherent information includes at least one of age and gender.

12. A computer-readable storage medium, the computer-readable storage medium comprising a stored computer program, wherein: When the computer program is executed, the method for estimating a body index of a subject for medical imaging according to any one of claims 1 to 11 is executed.

13. A medical imaging system, characterized in that: The system comprises: An imaging device that captures image data of a subject; A controller is connected to the camera device and is used to execute the subject's body index estimation method as described in any one of claims 1-11.

Images