Shielding detection method for medical imaging and medical imaging method and system

Abstract

The embodiment of the invention provides an occlusion detection method for medical imaging, a medical imaging method and a medical imaging system.The occlusion detection method for medical imaging comprises the steps that an image sequence is obtained, and the image sequence comprises a plurality of images of an object in the time dimension; and determining a shielding state of the key point according to at least one of confidence coefficient change information and depth change information of the key point of the object in the plurality of images.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of medical equipment technology, and in particular, to an occlusion detection method for medical imaging, a medical imaging method, and a system. Background Art

[0002] In the scenario where a medical imaging system is used to scan and image an object to be detected, the positioning information of the object to be detected can be determined based on the image of the object to be detected acquired by the camera device, so as to locate the object to be detected. In the positioning operation process, it is necessary to determine whether the surface of the object to be detected is covered with an obstruction (for example, a coil, a blanket, complex clothing, a mask, etc.). Summary of the invention

[0003] Embodiments of the present application provide an occlusion detection method for medical imaging, a medical imaging method, and a system.

[0004] According to one aspect of an embodiment of the present application, there is provided an occlusion detection method for medical imaging, the method comprising: acquiring an image sequence, the image sequence comprising a plurality of images of an object in a time dimension; determining the occlusion state of the key points based on at least one of confidence change information and depth change information of the key points of the object in the plurality of images.

[0005] According to one aspect of an embodiment of the present application, a medical imaging method is provided, the method comprising: determining the occlusion state of a key point according to the above-mentioned occlusion detection method for medical imaging; determining the positioning information of an object according to the occlusion state of the key point; and performing a scanning operation according to the determined positioning information.

[0006] According to one aspect of an embodiment of the present application, a medical imaging system is provided, the system comprising: a controller configured to execute the above-mentioned occlusion detection method for medical imaging, determine the occlusion state of key points, and determine the positioning information of the object based on the occlusion state of the key points; and a scanning component, which performs a scanning operation based on the determined positioning information.

[0007] One of the beneficial effects of the embodiments of the present application is that determining the occlusion state of a key point based on at least one of the confidence change information and depth change information of the key points of multiple images in an image sequence can improve the accuracy and reliability of the occlusion state, and the above-mentioned method of determining the occlusion state does not depend on the shape, color or pattern of the occluding object, and has good universality.

[0008] With reference to the following description and accompanying drawings, the specific implementation of the embodiment of the present application is disclosed in detail, indicating the way in which the principle of the embodiment of the present application can be adopted. It should be understood that the implementation of the present application is not limited in scope. Within the scope of the spirit and clauses of the attached claims, the implementation of the present application includes many changes, modifications and equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without creative work. In the drawings:

[0010] Figure 1 is a schematic diagram of a magnetic resonance imaging system according to an embodiment of the present application;

[0011] Figure 2 is a schematic diagram of an occlusion detection method for medical imaging according to an embodiment of the present application;

[0012] Figure 3 It is a schematic diagram of the key points of the embodiment of the present application;

[0013] Figure 4 is a schematic diagram of the occlusion state of the key points of an embodiment of the present application;

[0014] Figure 5 is another schematic diagram of the occlusion state of the key points of the embodiment of the present application;

[0015] Figure 6 It is a schematic diagram of the preset area of ​​this application;

[0016] Figure 7 is a schematic diagram of an implementation of step 202 of an embodiment of the present application;

[0017] Figure 8 It is an occlusion detection device for medical imaging according to an embodiment of the present application;

[0018] Fig. 9 is a schematic diagram of a medical imaging method according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] With reference to the accompanying drawings, the foregoing and other features of the embodiments of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the embodiments of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the embodiments of the present application include all modifications, variations and equivalents that fall within the scope of the attached claims.

[0020] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0021] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0022] In the embodiments of the present application, the term "landmark" can be equivalently replaced by "key point", "key coordinate point" or "landmark point"; the term "object" can be equivalently replaced by "detected object", "scanned object", "object to be scanned", "patient" or "research object", which can be a human or an animal, etc.

[0023] In the embodiments of the present application, the term "include / comprises" when used herein refers to the existence of features, whole parts, steps or components, but does not exclude the existence or addition of one or more other features, whole parts, steps or components.

[0024] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0025] The occlusion detection method of the embodiment of the present application can be applied to various medical imaging scenarios, including but not limited to magnetic resonance imaging (MRI), computed tomography (CT), ultrasound imaging (Ultrasound Imaging), positron emission tomography (PET), single photon emission computed tomography (SPECT), PET / CT, PET / MR or any other suitable medical imaging scenario.

[0026] In the embodiments of the present application, the method, device and system of the present application are exemplarily described by taking the MRI scenario as an example. It can be understood that the contents of the embodiments of the present application are also applicable to other medical imaging scenarios.

[0027] For ease of understanding, Figure 1 FIG. 1 is a schematic diagram of a magnetic resonance imaging (MRI) system 100 according to an embodiment of the present application.

[0028] The MRI system 100 includes a scanning unit 111. The scanning unit 111 is used to perform magnetic resonance scanning on an object (eg, a human body) 170 to generate image data of a region of interest of the object 170, where the region of interest may be a predetermined anatomical site or anatomical tissue.

[0029] The operation of the MRI system 100 is controlled by an operator workstation 110, which includes an input device 114, a control panel 116, and a display 118. The input device 114 can be a joystick, a keyboard, a mouse, a trackball, a touch-activated screen, voice control, or any similar or equivalent input device. The control panel 116 can include a keyboard, a touch-activated screen, voice control, buttons, a slider, or any similar or equivalent control device. The operator workstation 110 is coupled to and communicates with a computer system 120, which enables an operator to control the generation and viewing of images on the display 118. The computer system 120 includes multiple components that communicate with each other via electrical and / or data connection modules 122. The connection module 122 can be a direct wired connection, an optical fiber connection, a wireless communication link, etc. The computer system 120 can include a central processing unit (CPU) 124, a memory 126, and an image processor 128. In some embodiments, the image processor 128 can be replaced by a medical imaging function implemented in the CPU 124. The computer system 120 may be connected to an archival media device, permanent or backup storage, or a network. The computer system 120 may be coupled to and communicate with a separate MRI system controller 130 .

[0030] The MRI system controller 130 includes a set of components that communicate with each other via an electrical and / or data connection module 132. The connection module 132 can be a direct wired connection, an optical fiber connection, a wireless communication link, etc. The MRI system controller 130 may include a CPU 131, a sequence pulse generator (also referred to as a pulse generator) 133 that communicates with the operator workstation 110, a transceiver (also referred to as an RF transceiver) 135, a memory 137, and an array processor 139.

[0031] In some embodiments, the sequence pulse generator 133 can be integrated into the resonance assembly 140 of the scanning unit 111 of the MRI system 100. The MRI system controller 130 can receive commands from the operator workstation 110, coupled to the scanning unit 111, to indicate the MRI scanning sequence to be performed during the MRI scan, so as to control the scanning unit 111 to perform the above-mentioned magnetic resonance scanning process. The MRI system controller 130 is also coupled to and communicates with the gradient drive system (also referred to as gradient drive) 150, which is coupled to the gradient coil assembly 142 to generate a magnetic field gradient during the MRI scan.

[0032] The sequence pulse generator 133 may also receive data from a physiological acquisition controller 155, which receives signals from a plurality of different sensors (e.g., electrocardiogram (ECG) signals from electrodes attached to the patient, etc.) connected to a subject or patient 170 undergoing an MRI scan. The sequence pulse generator 133 is coupled to and in communication with a scan room interface system 145, which receives signals from various sensors associated with the state of the resonance assembly 140. The scan room interface system 145 is also coupled to and in communication with a patient positioning system 147, which sends and receives signals to control the movement of the patient table to a desired position for an MRI scan.

[0033] The MRI system controller 130 provides gradient waveforms to the gradient driver system 150, which includes a G x (x direction), G y (y direction) and G z (z direction) amplifier, etc. Each G x , G y and G z The gradient amplifiers all excite corresponding gradient coils in the gradient coil assembly 142 to generate magnetic field gradients for spatially encoding MR signals during MRI scanning. The gradient coil assembly 142 is disposed within a resonance assembly 140, which also includes a superconducting magnet having superconducting coils 144, which in operation provide a static uniform longitudinal magnetic field B0 throughout a cylindrical imaging volume 146. The resonance assembly 140 also includes an RF body coil 148, which in operation provides a transverse magnetic field B1 that is substantially perpendicular to B0 throughout the cylindrical imaging volume 146. The resonance assembly 140 may also include an RF surface coil 149 for imaging different anatomical structures of a patient undergoing an MRI scan. The RF body coil 148 and the RF surface coil 149 may be configured to operate in a transmit and receive mode, a transmit mode, or a receive mode.

[0034] The x direction can also be called the frequency encoding direction or the k direction in K space. x The y direction can be called the phase encoding direction or the k direction in K space. y Direction, the z direction can be called the layer selection direction. x It can be used for frequency encoding or signal readout and is usually called frequency encoding gradient or readout gradient. y Can be used for phase encoding, usually called phase encoding gradient. G z It can be used for slice (layer) position selection to obtain K-space data. It should be noted that the slice selection direction, phase encoding direction and frequency encoding direction can be modified according to actual needs.

[0035] An MRI scanned subject or patient 170 may be positioned within a cylindrical imaging volume 146 of the resonance assembly 140. The transceiver 135 in the MRI system controller 130 generates RF excitation pulses that are amplified by an RF amplifier 162 and provided to the RF body coil 148 via a transmit / receive switch (also referred to as a T / R switch or switch) 164.

[0036] As described above, the RF body coil 148 and the RF surface coil 149 can be used to transmit RF excitation pulses and / or receive the resulting MR signals from the patient undergoing the MRI scan. The MR signals emitted by the nuclei excited in the patient's body of the MRI scan can be sensed and received by the RF body coil 148 or the RF surface coil 149 and sent back to the preamplifier 166 through the T / R switch 164. The T / R switch 164 can be controlled by a signal from the sequence pulse generator 133 to electrically connect the RF amplifier 162 to the RF body coil 148 during the transmit mode, and to connect the preamplifier 166 to the RF body coil 148 during the receive mode. The T / R switch 164 can also enable the RF surface coil 149 to be used in the transmit mode or the receive mode.

[0037] In some embodiments, the MR signals sensed and received by the RF body coil 148 or the RF surface coil 149 and amplified by the preamplifier 166 are stored as a raw k-space data array in the memory 137 for post-processing. A reconstructed magnetic resonance image can be acquired by transforming / processing the stored raw k-space data.

[0038] In some embodiments, the MR signals sensed and received by the RF body coil 148 or the RF surface coil 149 and amplified by the preamplifier 166 are demodulated, filtered, and digitized in the receive portion of the transceiver 135 and transmitted to the memory 137 in the MRI system controller 130. For each image to be reconstructed, the data is rearranged into a separate k-space data array, and each of these separate k-space data arrays is input to an array processor 139, which is operated to Fourier transform the data into an array of image data.

[0039] The array processor 139 uses a transform method, most commonly a Fourier transform, to create images from the received MR signals. These images are transmitted to the computer system 120 and stored in the memory 126. In response to commands received from the operator workstation 110, the image data may be stored in long-term storage or may be further processed by the image processor 128 and transmitted to the operator workstation 110 for presentation on the display 118.

[0040] In various embodiments, components of computer system 120 and MRI system controller 130 may be implemented on the same computer system or multiple computer systems. Figure 1 The MRI system 100 shown is for illustration purposes. Suitable MRI systems may include more, fewer, and / or different components.

[0041] The MRI system controller 130 and the image processor 128 may include a computer processor and a storage medium, respectively or in common, on which a program for predetermined data processing to be executed by the computer processor is recorded, for example, the storage medium may store a program for implementing scanning processing (e.g., scanning process, imaging sequence), image reconstruction, medical imaging, etc. For example, a program for implementing the magnetic resonance imaging method of the embodiment of the present invention may be stored. The above-mentioned storage medium may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-volatile memory card.

[0042] The MRI system 100 also includes an imaging device 180. The imaging device 180 uses the principle of optical imaging to obtain visual and morphological information of the object being detected. The imaging device 180 can be a camera, a video camera, etc. Usually, the imaging device 180 can be installed near the examination bed, so that the image information of the object being detected can be collected to the greatest extent in a non-contact manner. The image information can be used to assist medical imaging operations. For example, the image information obtained by the imaging device 180 can be used to determine the positioning information of the object being detected, so as to optimize the positioning process of the object being detected.

[0043] The inventors have found that since the detected object is covered by an obstruction, it is necessary to determine the obstruction range of the detected object in order to determine the positioning information based on the obstruction range. However, existing occlusion detection methods have certain limitations. For example, since there are a large number of obstructions of different types and styles in the MRI system, a lot of effort needs to be spent on collecting and marking the training data of these obstructions in order to train the detection model based on these training data. In addition, if a new obstruction is introduced, the training data needs to be collected and marked again to train a detection model that can detect the new obstruction. The detection model obtained in the above manner depends on the shape, color or pattern of the obstruction, etc., and its applicable scenarios are very limited.

[0044] In response to at least one of the above problems, embodiments of the present application provide an occlusion detection method for medical imaging, a medical imaging method, and a medical imaging system.

[0045] The following is a description with reference to embodiments.

[0046] The present application provides an occlusion detection method for medical imaging. Figure 2is a schematic diagram of an occlusion detection method for medical imaging according to an embodiment of the present application. Figure 2 As shown, the method includes:

[0047] Step 201: Acquire an image sequence, where the image sequence includes multiple images of an object in a time dimension;

[0048] Step 202: Determine the occlusion state of the key points according to at least one of the confidence change information and the depth change information of the key points of the objects in the multiple images.

[0049] According to the above embodiment, when determining the occlusion state of a key point, the change in confidence or depth value of the key point of the detected object caused by the covering of the occlusion object is taken into consideration. The occlusion state of the key point is determined based on the change in confidence or depth value of the key point, which can improve the accuracy and reliability of the occlusion state. In addition, the above method of determining the occlusion state does not depend on the shape, color or pattern of the occlusion object, and has good universality.

[0050] In some embodiments, the image sequence may be generated by the aforementioned camera device 180. The image sequence includes multiple images in a time dimension, that is, the multiple images may be images acquired at different times.

[0051] In some embodiments, the image sequence may include various types of image sequences, for example, the image sequence includes a color image sequence, or a depth image sequence, or a color image sequence and a depth image sequence corresponding to each other in the time dimension, wherein the color image sequence includes a plurality of color images, and the depth image sequence includes a plurality of depth images.

[0052] In other words, the images in the image sequence may include various types of images. For example, the image may include a color image, or a depth image, or a color image and a depth image corresponding to each other in the time dimension. The color image and the depth image corresponding to each other in the time dimension may refer to a color image and a depth image acquired at the same time.

[0053] In some embodiments, the color image may include a plurality of pixels, and the value of each pixel may include the value of each color component, wherein the color component may include red, green, and blue (RGB) components, etc.

[0054] In some embodiments, the depth image may include multiple pixels, and the value of each pixel may include the depth value of the pixel. The depth value may be the distance between the detected object and the reference plane, etc. The reference plane is, for example, the camera plane of the camera device 180.

[0055] In some embodiments, when the image sequence includes a color image sequence, the confidence change information of the key point can be determined based on multiple color images in the image sequence. Since the color image can provide more image information, determining the confidence change information of the key point based on the color image can improve the accuracy of the confidence change information. The present application is not limited to this, and the confidence change information can also be determined based on other images. For example, the confidence change information can also be determined based on the depth image in the depth image sequence.

[0056] In some embodiments, the confidence change information of the key point may be confidence change information of the same type of key points of a first image and a second image in the plurality of images, wherein the second image is located after the first image in the time dimension.

[0057] The confidence change information may be in various forms. For example, the confidence change information may be at least one of a difference and a ratio of confidences of key points of the same type in the first image and the second image.

[0058] In some embodiments, the key points of the detected object can be, for example, feature points related to the body parts (anatomical structures) of the detected object that can be detected in the image. Among them, the types of key points can include at least one of the following: head, nose, left eye, right eye, left ear, right ear, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, chest, abdomen, left hip, right hip, left knee, right knee, left ankle, or right ankle. The present application is not limited to this, and the key point type can also include other content.

[0059] In some embodiments, the confidence of a key point may be in the form of probability. For example, the confidence of a key point may represent the probability that the true position of the key point is located at the detection position. The higher the confidence of a key point, the greater the possibility that the key point is located at the detection position; the lower the confidence of a key point, the smaller the possibility that the key point is located at the detection position. The present application is not limited thereto, and the confidence information of a key point may also be in other forms or have other meanings.

[0060] In some embodiments, key points and confidence levels can be generated by performing key point recognition on an image using a deep learning model (Deep Learning Module). The deep learning model can be a convolutional neural network, a deep belief network, a stacked auto-encoder network model, and the like. The input of the deep learning model can be an image in an image sequence, and the output of the deep learning model can include at least one of the following: key points (position information, type information, etc.), confidence information, and the like. The specific method of using a deep learning model to recognize key points can refer to the relevant technology and will not be described in detail here.

[0061] In some embodiments, the position information of the key point can be represented by the pixel position in the image. The present application is not limited thereto, and the position information of the key point can also be represented by other methods.

[0062] Figure 3 Schematic diagram of the key points of the embodiment of the present application. Figure 3 As shown, the deep learning model can detect 16 key points of the human body. For example, the 16 key points are head key point 0, right shoulder key point 1, left shoulder key point 2, chest key point 3, abdomen key point 4, right elbow key point 5, left elbow key point 6, right wrist key point 7, left wrist key point 8, right hip key point 9, left hip key point 10, right knee key point 11, left knee key point 12, right ankle key point 13, left ankle key point 14 and neck key point 15. Among them, the information of the 16 key points can be represented by the position information of the key points, the type information of the key points, etc. In addition, the 16 key points correspond to a confidence information respectively.

[0063] In some embodiments, when the image sequence includes a depth image sequence, the depth change information of the key point may be determined based on a plurality of depth images.

[0064] In some embodiments, the depth change information of the key point may be depth change information of the same type of key points in a first image and a second image of the plurality of images.

[0065] The depth change information may be in various forms. For example, the depth change information may include at least one of a difference and a ratio of depths of key points of the same type in the first image and the second image.

[0066] The following is an exemplary description of a specific method of determining the occlusion state of a key point according to the confidence change information of the key point.

[0067] In some embodiments, when the confidence of a keypoint of the second image is reduced by a first value compared to the confidence of a keypoint of the same type of the first image, and the first value is greater than a first threshold, the keypoint is determined to be an occluded keypoint.

[0068] In some embodiments, the first image may be an image adjacent to the second image in the time dimension. For example, the image sequence includes N images, and the i+1th image is taken as the second image as an example, and the first image is the i-th image, where i is greater than or equal to 1 and less than or equal to N-1.

[0069] The present application is not limited to this, and the first image may also be an image in which the key points of at least a part of the area of ​​the detected object are not obstructed. That is, the image sequence includes an image in which the key points of at least a part of the area of ​​the detected object are not obstructed, so that the image can be used as the first image (reference image) for confidence comparison. Among them, the first image can be determined based on information input by the user. For example, the information input by the user directly indicates the first image; or the information input by the user is the first moment when the detected object is covered by an obstruction. In this case, the image before the first moment can be selected from the image sequence as the first image, and so on.

[0070] Below, we use deep learning model to detect Figure 3 Taking the 16 key points shown and the detected object in the first image not being covered by an occlusion object as an example, a specific method for determining the occlusion state is exemplarily described.

[0071] Table 1 shows the confidence of each key point in the first image. As shown in Table 1, since the detected object in the first image is not covered by an occluder, the confidence level of the key point output by the deep learning model is relatively high.

[0072] Table 1: Confidence of each key point in the first image

[0073] 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 88 83 77 53 53 59 59 80 73 57 53 55 51 78 73 53

[0074] The first row of Table 1 indicates the number of each key point, and the second row of Table 1 indicates the confidence of each key point (the confidence is in the form of percentage, for example, the confidence of the head key point 0 is 88%).

[0075] Figure 4 Schematic diagram of the occlusion state of the key points of the embodiment of the present application. Figure 4As shown in FIG. 4 , in the second image, a portion of the detected object is covered by a blanket 400. Table 2 shows the confidence of each key point in the second image. As shown in Table 2, since a portion of the detected object is covered by the blanket, the confidence of the key point corresponding to the portion is reduced or not detected, and the confidence of the undetected key point is 0 or does not exist.

[0076] Table 2: Confidence of each key point in the second image

[0077] 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 88 30 23 23 23 13 0 0 0 34 26 37 32 68 69 23

[0078] The first row of Table 2 indicates the number of each key point, and the second row of Table 2 indicates the confidence of each key point.

[0079] After determining the confidence of each key point of the first image and the second image, the confidence of each key point of the second image is subtracted from the confidence of each key point of the first image to obtain the confidence change information. Table 3 shows the confidence change information of the second image relative to the first image. As shown in Table 3, the confidence of the right shoulder key point 1, the left shoulder key point 2, the chest key point 3, the abdomen key point 4, the right elbow key point 5, the left elbow key point 6, the right wrist key point 7, the left wrist key point 8, the right hip key point 9, the left hip key point 10 and the neck key point 15 decreases, and the decrease rate exceeds 20%. Therefore, the occlusion state of the right shoulder key point 1, the left shoulder key point 2, the chest key point 3, the abdomen key point 4, the right elbow key point 5, the left elbow key point 6, the right wrist key point 7, the left wrist key point 8, the right hip key point 9, the left hip key point 10 and the neck key point 15 in the second image is occluded.

[0080] Table 3: Confidence of each key point in the second image

[0081] 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 -53 -54 -30 -30 -46 -59 -80 -73 -23 -27 -18 -19 -10 -4 -30

[0082] The first row of Table 3 indicates the serial number of each key point, and the second row of Table 3 indicates the confidence change information of each key point.

[0083] Figure 5 is another schematic diagram of the occlusion state of the key points of the embodiment of the present application. Figure 5 As shown, in the second image, most of the area of ​​the detected object is covered by the blanket 500. Table 4 shows the confidence of each key point in the second image. As shown in Table 4, since most of the area of ​​the detected object is covered by the blanket, most of the key points are not detected.

[0084] Table 4: Confidence of each key point in the second image

[0085] 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 88 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0086] The first row of Table 4 indicates the number of each key point, and the second row of Table 4 indicates the confidence of each key point.

[0087] As shown in Table 4, except for the head key point, the confidences of the other 15 key points decrease, and the decrease rate exceeds 20%. Therefore, the occlusion status of the other 15 key points except for the head key point in the second image is occluded.

[0088] In some embodiments, there are a third image and a fourth image in the image sequence, wherein the occlusion state of some key points of the third image is determined to be "occluded", and the fourth image is located after the third image in the time dimension. If the confidence of the key point of the fourth image is increased by a third value compared with the confidence of the key point of the same type of the third image, and the third value is greater than a third threshold, the occlusion state of the key point of the fourth image can be determined to be "unoccluded".

[0089] The third image and the fourth image may be images adjacent to each other in the time dimension, or the occlusion state of at least some key points in the third image is occluded.

[0090] The third threshold may be the same value as the first threshold, but the present application is not limited thereto. The third threshold may also be a different value from the first threshold.

[0091] For example, if the occlusion state of the abdominal key point of the third image is "occluded" and the confidence of the abdominal key point is 23%, if the confidence of the abdominal key point of the fourth image is 78%, the confidence of the abdominal key point of the fourth image increases relative to the confidence of the abdominal key point of the third image and the increase of 55% is greater than the third threshold value of 20%, then the occlusion state of the abdominal key point in the fourth image can be determined as "unoccluded".

[0092] The following is an exemplary description of a specific method of determining the occlusion state of a key point according to the depth change information of the key point.

[0093] In some embodiments, the keypoint is an occluded keypoint if the depth of the keypoint in the second image is reduced by a second value from the depth of the same type of keypoint in the first image, and the second value is greater than a second threshold.

[0094] In some embodiments, the first image may be an image adjacent to the second image in the time dimension. The present application is not limited thereto, and the first image may also be an image in which the key points of at least a portion of the detected object are not blocked. That is, the image sequence includes an image in which the key points of at least a portion of the detected object are not blocked, thereby enabling the image to be used as the first image (reference image) for depth value comparison.

[0095] In some embodiments, when determining the occlusion state of a key point based on the depth change information of the key point, the position information of the key point can be determined first, and then the depth change information of the key point can be determined based on the position information of the key point. In this way, the amount of calculation can be reduced and computing power can be saved. The present application is not limited to this, and the depth change information of each pixel can also be determined within the entire image range, and then the depth change information of the key point can be determined based on the position information of the key point.

[0096] In some embodiments, the key points of the second image may be determined in various ways. Taking the example that the image sequence includes a depth image sequence, and the first image and the second image are depth images, the way of determining the key points of the second image is exemplarily described.

[0097] In some embodiments, the key points of the second image may be generated by performing key point recognition on the second image using a deep learning model.

[0098] In some embodiments, the key points of the second image may be generated by performing key point recognition on a color image corresponding to the second image using a deep learning model, that is, the coordinate information of the key points in the color image is used as or converted into the coordinate information of the key points of the same type in the second image.

[0099] In some embodiments, the key points of the second image may be generated by performing key point recognition on the second image and the color image respectively through a deep learning model, and performing weighted averaging on the recognition results. For example, the coordinates of the same type of key points of the second image and the corresponding color image are weighted according to the confidence level, and the weighted results are used as the key point coordinates of the second image.

[0100] In some embodiments, the key points of the second image may be generated by identifying key points of a reference image in an image sequence using a deep learning model. That is, the coordinate information of the key points in the reference image is used as or converted into the coordinate information of the same type of key points of the second image. The reference image may be an image in which the key points of at least a portion of the detected object are not obstructed. In some embodiments, the reference image may be the first image.

[0101] In some embodiments, after the key points of the second image are determined, the depth of the key points may be determined in various ways.

[0102] For example, the depth of the key point is the depth of the key point itself. For example, a pixel in the second image is determined based on the position information of the key point in the second image, and the depth value of the pixel is used as the depth of the key point. In this way, the depth of the key point can be determined in a simple manner.

[0103] For another example, the depth of a key point is determined based on the depth of pixels within a preset area in the image that includes the key point. Thus, the reliability of the depth of the key point can be improved. For example, when there is a hole in the coil covering the object to be detected, and the position of the hole coincides with the position of the key point, the depth of the key point does not change. If the depth of the key point itself is used to determine the occlusion state, the key point may be determined as unoccluded. However, the anatomical structure corresponding to the key point is in a state covered by the coil, so it is more reasonable to determine the key point as being occluded. By considering the depth of other pixels around the key point in the process of determining the depth of the key point, the reliability of the depth of the key point can be improved.

[0104] The key point may be located at the center of the preset area. The present application is not limited thereto, and the key point may also be located at other positions of the preset area.

[0105] In which, when determining the depth of a key point based on the depth of pixels within a preset area, the depth of the key point may be at least one of the following: an average value of the depths of all pixels within the preset area; a weighted average value of the depths of all pixels within the preset area; or the depth of the key point obtained by convolution processing the depths of the pixels within the preset area.

[0106] Figure 6 is a schematic diagram of the preset area of ​​this application. Figure 6 As shown, a preset area K can be selected in the second image with the key point P as the center. The preset area K includes 5*5 pixels, for example.

[0107] The average value of the depth values ​​of the 25 pixels may be used as the depth value of the key point P.

[0108] Alternatively, a weighted average may be calculated for the depth values ​​of the 25 pixels, and the result of the weighted average may be used as the depth value of the key point P. The weight of each pixel in the preset area may be set in various ways. For example, the weight of each pixel may be determined based on the distance between each pixel and the key point P. For example, the weight of the key point P may be set to 60%, the weight of the 8 pixels around the key point P may be set to 30%, and the weight of the outermost 18 pixels may be set to 10%.

[0109] Alternatively, a convolution operation may be performed on the depth of pixels in a preset area using a convolution kernel, and the result of the convolution operation is used as the depth of the key point P. The size of the convolution kernel is, for example, the size of the preset area.

[0110] In some embodiments, the second threshold for determining the occlusion state of the key point may be a value related to the depth value error of the camera device. The depth value error is, for example, a depth change value that the camera device cannot recognize. The depth value error is, for example, 2 cm.

[0111] The second threshold value may be a value greater than or equal to the depth value error of the camera device. For example, the second threshold value may be 3 centimeters, and so on.

[0112] In some embodiments, there are a fifth image and a sixth image in the image sequence, wherein the occlusion state of some key points of the fifth image is determined to be "occluded", and the sixth image is located after the fifth image in the time dimension. If the depth value of the key point of the sixth image is increased by a fourth value compared to the depth value of the key point of the same type of the fifth image, and the fourth value is greater than a fourth threshold, the occlusion state of the key point of the sixth image can be determined to be "unoccluded".

[0113] The fifth image and the sixth image may be adjacent images in the time dimension, or the occlusion state of at least some key points in the fifth image is occluded.

[0114] The fourth threshold may be the same value as the third threshold, but the present application is not limited thereto. The fourth threshold may also be a value different from the third threshold.

[0115] For example, if the occlusion state of the abdominal key point of the fifth image is "occluded" and the depth value of the abdominal key point is 70 cm, if the confidence of the abdominal key point of the sixth image is 78 cm, the depth value of the abdominal key point of the sixth image increases relative to the depth value of the abdominal key point of the fifth image and the increase of 8 cm is greater than the fourth threshold of 3 cm, then the occlusion state of the abdominal key point in the sixth image can be determined as "unoccluded".

[0116] In some embodiments, the occlusion state of the key point can be determined based on both the confidence change information and the depth change information. For example, when the following conditions 1 and 2 are met, the occlusion state of the key point is determined to be occluded, condition 1: the confidence of the key point of the second image is reduced by a first value compared to the confidence of the key point of the same type in the first image, and the first value is greater than a first threshold; condition 2: the depth of the key point of the second image is reduced by a second value compared to the depth of the key point of the same type in the first image, and the second value is greater than a second threshold.

[0117] In some embodiments, the same method of determining the occlusion state may be used for different types of key points. For example, for all key points, the occlusion state of the key points is determined based on both the confidence change information and the depth change information; or, for all key points, the occlusion state of the key points is determined based on the confidence change information; or, for all key points, the occlusion state of the key points is determined based on the depth change information.

[0118] In some embodiments, an appropriate method for determining the occlusion state of a key point may be selected according to the type of the key point. Figure 7 FIG. 2 is a schematic diagram of an implementation of step 202 of an embodiment of the present application. Figure 7 As shown, step 202 includes:

[0119] Step 701: Select at least one of confidence change information and depth change information according to the type of key points;

[0120] Step 702: Determine the occlusion state of the key point according to the selected information.

[0121] Since the confidence of the deep learning model output may not drop significantly when the occlusion area is small, the method of determining the occlusion state based on the confidence change information is more suitable for situations where the occlusion area is large. For example, for the key points of the torso (abdomen, chest, etc.), the area of ​​the occluded object is usually large, and the confidence change information can be selected to determine the occlusion state of the key points of the torso.

[0122] For example, for the key points of the limbs (elbows, wrists, knees, ankles, etc.), the area of ​​the coil is usually small, and the depth change information can be selected to determine the occlusion state of the key points of the limbs.

[0123] For example, for a head key point, both confidence change information and depth change information may be selected to determine the occlusion state of the head key point.

[0124] According to the above embodiment, when determining the occlusion state of a key point, the change in confidence or depth value of the key point of the detected object caused by the covering of the occlusion object is taken into consideration. The occlusion state of the key point is determined based on the change in confidence or depth value of the key point, which can improve the accuracy and reliability of the occlusion state. In addition, the above method of determining the occlusion state does not depend on the shape, color or pattern of the occlusion object, and has good universality.

[0125] The embodiment of the present application further provides an occlusion detection device for medical imaging, and the contents that are the same as those of the embodiment of the first aspect are not repeated here.

[0126] Figure 8 is a schematic diagram of an occlusion detection device for medical imaging according to an embodiment of the present application, such as Figure 8 As shown, the occlusion detection device 800 for medical imaging includes:

[0127] An acquisition unit 801, which acquires an image sequence, wherein the image sequence includes a plurality of images of an object in a time dimension;

[0128] A determining unit 802 is configured to determine an occlusion state of the key point according to at least one of confidence change information and depth change information of the key point of the object in the multiple images.

[0129] In some embodiments, the confidence change information is confidence change information of key points of the same type in a first image and a second image of the multiple images, wherein the second image is located after the first image in the time dimension.

[0130] In some embodiments, the confidence change information includes at least one of a difference and a ratio of confidences of key points of the same type in the first image and the second image.

[0131] In some embodiments, when the confidence of a keypoint of the second image is reduced by a first value compared to the confidence of a keypoint of the same type of the first image, and the first value is greater than a first threshold, the keypoint is an occluded keypoint.

[0132] In some embodiments, the key points and the confidence levels are generated by performing key point recognition on an image using a deep learning model.

[0133] In some embodiments, the depth change information is depth change information of key points of the same type in a first image and a second image of the multiple images, wherein the second image is located after the first image in a time dimension.

[0134] In some embodiments, the depth change information includes at least one of a difference and a ratio of depths of key points of the same type in the first image and the second image.

[0135] In some embodiments, when the depth of a keypoint of the second image is reduced by a second value compared to the depth of a keypoint of the same type of the first image, and the second value is greater than a second threshold, the keypoint is an occluded keypoint.

[0136] In some embodiments, the depth of the key point is the depth of the key point itself, or the depth of the key point is determined based on the depth of pixels in a preset area in the image, and the key point is located in the preset area.

[0137] In some embodiments, the key point is located at the center of the preset area.

[0138] In some embodiments, the depth of the keypoint is at least one of the following:

[0139] The average value of the depths of the pixels in the preset area;

[0140] A weighted average of the depths of the pixels within the preset area; or

[0141] The depth of the key point is obtained by performing convolution processing on the depth of the pixels in the preset area.

[0142] In some embodiments, the first image is an image adjacent to the second image in the time dimension; or, the first image is an image in which key points of at least a partial area of ​​the object are not blocked.

[0143] In some embodiments, the determining unit 802 selects at least one of the confidence change information and the depth change information according to the type of the key point, and determines the occlusion state of the key point according to the selected information.

[0144] In some embodiments, the image sequence includes a color image sequence, or a depth image sequence, or a color image sequence and a depth image sequence corresponding to each other, the color image sequence includes multiple color images, the depth image sequence includes multiple depth images, the confidence change information is determined based on the multiple color images, and the depth change information is determined based on the multiple depth images.

[0145] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The medical imaging occlusion detection device 800 may also include other components or modules, and the specific contents of these components or modules may refer to the relevant technology.

[0146] To keep it simple, Figure 8 The connection relationship or signal direction between various components or modules is only exemplified, but it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors and memories; the embodiments of the present application are not limited to this.

[0147] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0148] When determining the occlusion state of a key point, the change in confidence or depth value of the key point of the detected object caused by the occlusion covering the detected object is taken into account. The occlusion state of the key point is determined based on the change in confidence or depth value of the key point, which can improve the accuracy and reliability of the occlusion state. The above method of determining the occlusion state does not depend on the shape, color or pattern of the occlusion, and has good universality.

[0149] The embodiment of the present application also provides a medical imaging method. Fig. 9 FIG. 1 is a schematic diagram of a medical imaging method according to an embodiment of the present application. Fig. 9 As shown, the medical imaging method includes:

[0150] Step 901: determining the occlusion state of key points according to the image sequence;

[0151] Step 902: Determine the location information of the object according to the occlusion state of the key point; and

[0152] Step 903: Perform a scanning operation according to the determined positioning information.

[0153] In step 901, the occlusion state of the key point may be determined according to the occlusion detection method for medical imaging described in the above-mentioned embodiment, the contents of which are incorporated herein and will not be further described.

[0154] In some embodiments, in step 902, when determining the positioning information of the object according to the image sequence, the occlusion state of the key points of the object is also considered, thereby ensuring the accuracy and reliability of the positioning information.

[0155] The positioning information may be the position information of the key point. In step 902, the positioning information may be determined in various ways according to the occlusion state. For example, for a key point that is occluded in the image, the position information of a key point of the same type before being occluded is used as the position information of the occluded key point.

[0156] In some embodiments, step 903 may include positioning the scanned object according to the positioning information, for example, determining a region of interest to be imaged of the current object, and adjusting the relative position between the object and the center of the scanning device according to the positioning information to align the coordinate center of the determined region of interest with the scanning center of the imaging device.

[0157] In some embodiments, step 903 further includes performing a scanning operation on the positioned scanning object.

[0158] According to the above embodiment, since the occlusion detection method for medical imaging described in the above embodiment is used to determine the occlusion state of the key point, the occlusion state of the key point has high accuracy and reliability. When the positioning information is determined according to the occlusion state of the key point and the scanning operation is performed according to the positioning information, the accuracy and reliability of the scanning result can be guaranteed.

[0159] The present application also provides a medical imaging system. The medical imaging system is composed of Figure 1 As shown, the repeated parts will not be repeated.

[0160] In some embodiments, Figure 1 The difference between the aforementioned medical imaging system and the aforementioned medical imaging system is that the controller 130 is configured to execute the aforementioned occlusion detection method for medical imaging, that is, to obtain an image sequence, the image sequence comprising a plurality of images of an object in a time dimension; and to determine the occlusion state of the key point according to at least one of the confidence change information and the depth change information of the key point of the object in the plurality of images.

[0161] In some embodiments, the controller 130 (which may also be a processor) includes a computer processor and a storage medium, on which a program for predetermined data processing to be executed by the computer processor is recorded. For example, the storage medium may store programs for implementing scanning processing (for example, including waveform design / conversion, etc.), image reconstruction, medical imaging, etc. For example, a program for implementing an occlusion detection method for medical imaging according to an embodiment of the present invention may be stored. The occlusion detection method for medical imaging includes: acquiring an image sequence, wherein the image sequence includes multiple images of an object in a time dimension; determining the occlusion state of the key point according to at least one of the confidence change information and the depth change information of the key point of the object in the multiple images. The specific implementation method is as described above and will not be repeated here. The above-mentioned storage medium may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-volatile memory card.

[0162] The present application is not limited thereto, and the aforementioned occlusion detection method for medical imaging may also be executed in other processors in the MRI system 100 , or executed by a cloud processor.

[0163] In some embodiments, the controller 130 may also determine the positioning information of the object based on the occlusion status of the key points.

[0164] In some embodiments, the scanning component can perform a scanning operation according to the positioning information determined by the controller 130. The scanning component can include a positioning component, which can position the scanned object according to the positioning information. The positioning component is, for example, the patient positioning system 147 of the MRI system 100.

[0165] In some embodiments, the scanning component may further include a scanning unit 111, and the scanning unit 111 performs a scanning operation on the positioned scanning object.

[0166] According to the above embodiment, since the occlusion detection method for medical imaging described in the above embodiment is used to determine the occlusion state of the key point, the occlusion state of the key point has high accuracy and reliability. When the positioning information is determined according to the occlusion state of the key point and the scanning operation is performed according to the positioning information, the accuracy and reliability of the scanning result can be guaranteed.

[0167] An embodiment of the present application also provides a computer-readable program, wherein when the program is executed in a medical imaging system, the program enables the computer to execute the occlusion detection method for medical imaging described in the aforementioned embodiment in the medical imaging system.

[0168] An embodiment of the present application further provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the occlusion detection method for medical imaging described in the aforementioned embodiment in a medical imaging system.

[0169] The above devices and methods of the present application can be implemented by hardware, or by hardware combined with software. The present application relates to such a computer-readable program, which, when executed by a logic component, enables the logic component to implement the above-mentioned devices or components, or enables the logic component to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0170] The method / device described in conjunction with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to the various steps shown in the figure, respectively. These hardware modules may be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0171] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if a device (such as a mobile terminal) uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0172] For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in the present application. For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0173] The present application is described above in conjunction with specific implementation methods, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles of the present application, and these modifications and variations are also within the scope of the present application.

Claims

1. An occlusion detection method for medical imaging, characterized in that: The method comprises: Acquire an image sequence, wherein the image sequence includes a plurality of images of the object in a time dimension; The occlusion state of the key point is determined according to at least one of the confidence change information and the depth change information of the key point of the object in the multiple images.

2. The method according to claim 1, characterized in that The confidence change information is confidence change information of key points of the same type in a first image and a second image among the multiple images, wherein the second image is located after the first image in the time dimension.

3. The method according to claim 2, characterized in that The confidence change information includes at least one of a difference and a ratio of confidences of key points of the same type in the first image and the second image.

4. The method according to claim 2, characterized in that: When the confidence of the key point of the second image is less than the confidence of the key point of the same type in the first image by a first value, and the first value is greater than a first threshold, the key point is an occluded key point.

5. The method according to claim 2, characterized in that: The key points and the confidence levels are generated by performing key point recognition on an image using a deep learning model.

6. The method according to claim 1, characterized in that The depth change information is depth change information of key points of the same type in a first image and a second image of the multiple images, wherein the second image is located after the first image in a time dimension.

7. The method according to claim 6, characterized in that The depth change information includes at least one of a difference and a ratio of depths of key points of the same type in the first image and the second image.

8. The method according to claim 6, characterized in that When the depth of a key point in the second image is less than the depth of a key point of the same type in the first image by a second value, and the second value is greater than a second threshold, the key point is an occluded key point.

9. The method according to claim 6, characterized in that The depth of the key point is determined according to the depth of pixels in a preset area in the image, and the key point is located in the preset area.

10. The method according to claim 9, characterized in that The key point is located at the center of the preset area.

11. The method according to claim 9, characterized in that The depth of the key point is at least one of the following: The average value of the depths of the pixels in the preset area; A weighted average of the depths of the pixels within the preset area; or The depth of the key point is obtained by performing convolution processing on the depth of the pixels in the preset area.

12. The method according to claim 2 or 6, characterized in that: The first image is an image adjacent to the second image in a time dimension; or, the first image is an image in which key points of at least a partial area of ​​the object are not blocked.

13. The method according to claim 1, characterized in that Determining the occlusion state of the key point according to at least one of the confidence change information and the depth change information of the key point of the object in the multiple images includes: At least one of the confidence change information and the depth change information is selected according to the type of the key point, and the occlusion state of the key point is determined according to the selected information.

14. The method according to claim 1, characterized in that The image sequence includes a color image sequence, or a depth image sequence, or a color image sequence and a depth image sequence corresponding to each other, the color image sequence includes a plurality of color images, and the depth image sequence includes a plurality of depth images, The confidence change information is determined based on the multiple color images, and the depth change information is determined based on the multiple depth images.

15. A medical imaging method, characterized in that: The method comprises: Determining the occlusion state of a key point according to the method of any one of claims 1 to 14; and Determining the positioning information of the object according to the occlusion state of the key point; A scanning operation is performed according to the determined positioning information.

16. A medical imaging system, characterized in that: The system comprises: A controller configured to execute the method according to any one of claims 1 to 14 to determine an occlusion state of a key point, and determine positioning information of an object according to the occlusion state of the key point; and A scanning component performs a scanning operation according to the determined positioning information.

Images