Scanning plane determination method, apparatus, device, and storage medium

Abstract

The application discloses a scanning plane determination method and device, equipment and a storage medium, and belongs to the technical field of medical treatment. The determination equipment can predict distance information of multiple positioning images, the distance information of the positioning image is used to reflect the distance between each pixel in the positioning image and the first intersection line of the positioning image, and the first intersection line of the positioning image is the intersection line of the positioning image and the target scanning plane to be determined. Then, the determination equipment can automatically determine the target scanning plane based on the distance information of the multiple positioning images. Since the target scanning plane is automatically determined, manual determination of the target scanning plane according to experience is not needed, thereby on the one hand, the operation of the staff is reduced, the determination efficiency of the scanning plane is effectively improved, and on the other hand, the problem that the accuracy of the determined target scanning plane is low due to manual errors can be avoided, so that the accuracy of the scanning image based on the scanning plane can be ensured.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to a method, apparatus, device and storage medium for determining a scanning plane. Background Technology

[0002] When using cardiac magnetic resonance imaging (CMR) to scan the heart of a patient, it is usually necessary to first determine the scanning plane for the CMR of that patient, and then scan the heart of the patient based on the scanning plane to ensure that the scan image obtained based on the scanning plane can accurately reflect the functional characteristics of the heart.

[0003] In related technologies, the scanning plane is determined by the operator based on personal experience. This approach is inefficient in determining the scanning plane, and the accuracy of the determined scanning plane is easily compromised by human error, thus affecting the precision of the scanned image. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for determining a scanning plane, which can solve the problem of low efficiency in determining the scanning plane in related technologies, which easily affects the accuracy of the scanned image. The technical solution is as follows:

[0005] According to one aspect of this application, a method for determining a scanning plane is provided, the method comprising:

[0006] Multiple positioning images are acquired, the positioning images being determined based on multiple scan images obtained by scanning a target object from different angles, the target object including target parts;

[0007] Predict the distance information of the plurality of positioning images. The distance information of any positioning image is used to reflect the distance of each pixel in the positioning image to the first intersection line of the positioning image. The first intersection line of the positioning image is the intersection line of the positioning image with the target scanning plane of the target part to be determined.

[0008] The target scanning plane is determined based on the distance information of the multiple positioning images.

[0009] According to another aspect of this application, a scanning plane determination apparatus is provided, the apparatus comprising:

[0010] An acquisition module is used to acquire multiple positioning images, which are determined based on multiple scan images. The multiple scan images are obtained by scanning a target object from different angles, and the target object includes target parts.

[0011] The prediction module is used to predict the distance information of the plurality of positioning images. The distance information of any positioning image is used to reflect the distance of each pixel in the positioning image to the first intersection line of the positioning image. The first intersection line of the positioning image is the intersection line of the positioning image with the target scanning plane of the target part to be determined.

[0012] The determination module is used to determine the target scanning plane based on the distance information of the multiple positioning images.

[0013] In an optional design, the determining module is used for:

[0014] Based on the distance information of the multiple positioning images, perform at least one scanning plane determination process until the cutoff condition is met;

[0015] The scan plane obtained by performing the second scan plane determination process is determined as the target scan plane;

[0016] The scanning plane determination process includes:

[0017] Multiple candidate planes are obtained, and each candidate plane is a plane in the space where the target part is located that intersects with all of the multiple positioning images;

[0018] Based on the distance information of the plurality of positioning images, the target distance of each of the plurality of candidate planes is obtained. For each candidate plane, the target distance of the candidate plane is the sum of the reference distances of the plurality of positioning images. The reference distance of any positioning image is the sum of the distances from each pixel of the second intersection line of any positioning image to the first intersection line of any positioning image. The second intersection line of any positioning image is the intersection line of any positioning image with the candidate plane.

[0019] The candidate plane with the smallest target distance among the multiple candidate planes is determined as the scanning plane.

[0020] In an optional design, the candidate plane is determined based on position parameters, and the position parameters of any two candidate planes are different, including: spatial point, polar angle, and orientation angle;

[0021] Wherein, the spatial point is a pixel point in the intersection line of any two positioning images in the space where the target part is located.

[0022] In an optional design, the determining module is used to perform a multi-scan plane determination process based on the distance information of the multiple positioning images;

[0023] In the sequentially adjacent first scan plane determination process and second scan plane determination process, multiple candidate planes in the second scan plane determination process are determined based on the first scan plane, and the first scan plane is the scan plane determined by the first scan plane determination process;

[0024] Wherein, at least one of the parameters used to determine the position of the second candidate plane satisfies:

[0025] The parameter value is the value between the target parameter value and the adjacent parameter value in the first parameter set, and the tolerance of the parameter values ​​in the second parameter value set is less than the tolerance of the parameter values ​​in the first parameter value set;

[0026] The target parameter value is the parameter value of the first scanning plane. The first parameter value set is the parameter value set used to determine the first candidate plane. The second parameter value set is the parameter value set used to determine the second candidate plane. The parameter values ​​in the first parameter value set and the second parameter value set are arranged in an arithmetic sequence. The first candidate plane is the candidate plane obtained in the process of determining the first scanning plane. The second candidate plane is the candidate plane obtained in the process of determining the second scanning plane.

[0027] In an optional design, the distance information of any positioning image is reflected by a heatmap of the positioning image, the heatmap having heat values ​​that correspond one-to-one with the pixels of the positioning image, each heat value being used to reflect the distance from the corresponding pixel to the first intersection line of the positioning image;

[0028] The reference distance for any positioning image is the sum of the heat values ​​corresponding to each pixel of the second intersection line of the positioning image on the heat map.

[0029] In an alternative design, each of the thermal values ​​is negatively correlated with the distance from the pixel corresponding to the thermal value to the first intersection line.

[0030] In an optional design, the plurality of localization images correspond one-to-one with a plurality of convolutional neural networks, and the prediction module is used for:

[0031] The target positioning image is input into the target convolutional neural network to obtain the distance information of the target positioning image output by the target convolutional neural network. The target convolutional neural network is used to process positioning images of the type to which the target positioning image belongs, and the target positioning image is any one of the plurality of positioning images.

[0032] In an optional design, the acquisition module is further used for:

[0033] Multiple training samples are acquired, including: historical distance information of the historical positioning image, the historical distance information being determined based on the distance from each pixel in the historical positioning image to a third intersection line, the third intersection line being the intersection line between the historical positioning image and the historical scanning plane, the historical positioning image and the target positioning image having the same scanning angle, and the historical scanning plane and the target scanning plane having the same scanning angle; the target convolutional neural network is trained based on the multiple training samples.

[0034] In an optional design, the acquisition module is used for:

[0035] Multiple medical samples are acquired, including: the historical positioning image and the historical scanning plane;

[0036] Based on the multiple medical samples, the multiple training samples are obtained;

[0037] The plurality of medical samples correspond one-to-one with the plurality of training samples. The historical distance information of the historical positioning images in the training samples is determined based on the position of each pixel included in the historical positioning image in the corresponding medical sample and the position of the third intersection line. The third intersection line is the intersection line between the historical positioning image and the historical scanning plane in the corresponding medical sample.

[0038] According to another aspect of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the scan plane determination method as described above.

[0039] According to another aspect of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the scan plane determination method as described above.

[0040] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the scan plane determination method provided in various alternative implementations of the above aspects.

[0041] The beneficial effects of the technical solution provided in this application include at least the following:

[0042] This application provides a method, apparatus, device, and storage medium for determining a scanning plane. The determining device can predict distance information from multiple positioning images. This distance information reflects the distance between each pixel in the positioning image and a first intersection line of the positioning image, which is the intersection line between the positioning image and the target scanning plane to be determined. Then, the determining device can automatically determine the target scanning plane based on the distance information from the multiple positioning images. Since the target scanning plane is automatically determined, there is no need for manual determination based on experience. Therefore, on the one hand, it reduces the workload of operators and effectively improves the efficiency of scanning plane determination; on the other hand, it avoids the problem of low accuracy in determining the target scanning plane due to human error, thereby ensuring the accuracy of the scanned image obtained based on the scanning plane. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a scanning plane determination method provided in an exemplary embodiment of this application;

[0045] Figure 2 This is a flowchart illustrating a scanning plane determination method provided in another exemplary embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the process for determining the target scanning plane provided in an exemplary embodiment of this application;

[0047] Figure 4 This is a flowchart of a method for determining a scanning plane from multiple candidate planes, provided in an exemplary embodiment of this application.

[0048] Figure 5 This is a schematic diagram of a pyramid search process provided in an exemplary embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the structure of a scanning plane determination device provided in an exemplary embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the structure of a determining device provided in an exemplary embodiment of this application.

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0053] Assuming the long axis of the human body extends along its height, the first plane is the scanning plane determined based on the long axis (LAX) of the human body, and the second plane is the scanning plane determined based on either the long axis or short axis (SAX) of the heart. The long axis of the heart is the line connecting the apex of the left ventricle (LV) to the mitral valve. The short axis is a straight line perpendicular to the long axis.

[0054] Because the long axis of the heart is not parallel to the long axis of the human body, images obtained by scanning the heart using CMR technology based on a first plane cannot reflect the functional characteristics of the heart. Therefore, in order for images obtained by CMR technology to accurately reflect the functional characteristics of the heart, a second plane needs to be determined first. This second plane is the scanning plane described in the subsequent embodiments of this application. Currently, the scanning plane is determined by staff based on personal experience, which is inefficient.

[0055] This application provides a method for determining a scanning plane, which can solve the aforementioned problems. This method can be applied to a scanning plane determining device (hereinafter referred to as the determining device). Optionally, the scanning plane determining device can be a smartphone, laptop, or desktop computer. See also... Figure 1 The method includes:

[0056] Step 101: Obtain multiple positioning images.

[0057] The multiple positioning images can be of different types, and are determined based on multiple scan images. For example, each positioning image is determined based on multiple scan images. These multiple scan images are obtained by scanning the target object from different angles. For instance, the multiple scan images could be obtained by scanning the target object from the coronal plane (i.e., directly in front or behind), sagittal plane (i.e., left or right side), and axial plane (i.e., directly above or below). The target object includes the target region.

[0058] In this embodiment of the application, taking the heart as the target site as an example, the acquired multiple localization images can be determined by the target scanning plane (also called the target imaging plane). This target scanning plane can be a two-chamber (2C) long-axis scanning plane (referred to as the 2C scanning plane), a three-chamber (3C) long-axis scanning plane (referred to as the 3C scanning plane), a four-chamber (4C) long-axis scanning plane (referred to as the 4C scanning plane), or a SAX scanning plane. If the target scanning plane to be determined is the 2C scanning plane, the multiple localization images can include pseudo (P) 4C localization images and PSAX localization images. If the target scanning plane to be determined is the 3C scanning plane, the multiple localization images can include P2C localization images, P4C localization images, and PSAX localization images. If the target scanning plane to be determined is the 4C scanning plane, the multiple localization images can include P2C localization images and PSAX localization images. If the target scanning plane to be determined is the SAX scanning plane, the multiple localization images can include P2C localization images and P4C localization images. Among them, the P2C positioning image is of type P2C, the P4C positioning image is of type P4C, and the PSAX positioning image is of type PSAX.

[0059] Step 102: Predict distance information from multiple localization images.

[0060] The distance information of any positioning image is used to reflect the distance between each pixel in the positioning image and the first intersection line of the positioning image. The first intersection line of the positioning image is the intersection line between the positioning image and the target scanning plane of the target region to be determined.

[0061] In one alternative example, the device may acquire multiple convolutional neural networks (CNNs), each CNN processing a type of localization image to obtain distance information for that type of localization image. For example, the multiple types of localization images corresponding to the multiple CNNs may include P2C, P4C, and PSAX types. That is, the multiple CNNs are used to process P2C localization images (i.e., P2C type localization images), P4C localization images (i.e., P4C type localization images), and PSAX localization images (i.e., PSAX type localization images), respectively.

[0062] After the device acquires multiple positioning images, for each positioning image, the device can select the corresponding convolutional neural network from among the aforementioned multiple convolutional neural networks based on the type of the positioning image. That is, it selects a neural network capable of processing positioning images of the type to which the positioning image belongs. Then, the positioning image is input into the corresponding convolutional neural network to obtain the distance information of the positioning image.

[0063] By employing convolutional neural networks to obtain distance information, the efficiency of determining the distance information for each localization image can be improved.

[0064] In another alternative example, the device can determine the distance information of a location image based on a prediction formula corresponding to the distance information of that location image.

[0065] Step 103: Determine the target scanning plane based on the distance information of multiple positioning images.

[0066] For example, after determining the distance information of each positioning image, the determining device can obtain the target distance of each of the multiple candidate planes based on the distance information of each positioning image. Then, the determining device can determine the candidate plane with the smallest target distance among the multiple candidate planes as the target scanning plane. Here, the target distance of each candidate plane is the sum of the reference distances of the multiple positioning images, and the sum of the reference distances of any positioning image is the sum of the distances from each pixel of the second intersection line of that positioning image to the first intersection line of that positioning image. The second intersection line of that positioning image is the intersection line of that positioning image with the candidate plane.

[0067] Alternatively, the determining device can identify multiple candidate pixels included in a first intersection line in each positioning image based on distance information from multiple positioning images, and identify multiple candidate planes based on multiple groups of pixels among the multiple candidate pixels. Each group of pixels includes at least three candidate pixels that are not collinear. Then, the determining device can identify the candidate plane that includes the most candidate pixels among the multiple candidate planes as the target scanning plane.

[0068] In summary, this application provides a method for determining a scanning plane. The determining device can predict distance information from multiple positioning images. This distance information reflects the distance between each pixel in the positioning image and a first intersection line of the positioning image, which is the intersection line between the positioning image and the target scanning plane to be determined. Then, the determining device can automatically determine the target scanning plane based on the distance information from the multiple positioning images. Since the target scanning plane is automatically determined, there is no need for manual determination based on experience. Therefore, on the one hand, it reduces the workload of operators and effectively improves the efficiency of scanning plane determination; on the other hand, it avoids the problem of low accuracy in determining the target scanning plane due to human error, thereby ensuring the accuracy of the scanned image obtained based on the scanning plane.

[0069] In this embodiment, the target object can be a human body or other mammals, and the target body part can be the heart, knee, shoulder bone, or other parts. For ease of explanation, subsequent embodiments will use a human body as the target object and the heart as the target body part as an example. Furthermore, this embodiment uses the determination of distance information corresponding to a positioning image based on a convolutional neural network as an example to exemplify the scanning plane determination method provided in this embodiment. See also... Figure 2 The method may include:

[0070] Step 201: Obtain multiple convolutional neural networks.

[0071] In this embodiment, since determining the target scanning plane of a heart requires distance information from multiple different types of positioning images, the determining device needs to acquire multiple convolutional neural networks corresponding one-to-one with the multiple different types of positioning images to obtain the distance information of each positioning image. The multiple convolutional neural networks in step 201 are the multiple convolutional neural networks corresponding one-to-one with the multiple positioning images.

[0072] Each convolutional neural network (CNN) is used to process one type of localization image, and the types of localization images processed by multiple CNNs are different. Each CNN corresponds to a localization image because it can process localization images of the type to which that image belongs. For a detailed explanation, please refer to step 102 above.

[0073] Optionally, each convolutional neural network can be a fully convolutional network (FCN).

[0074] In this embodiment, the target scanning plane can be a 2C scanning plane, a 3C scanning plane, a 4C scanning plane, or a SAX scanning plane. Furthermore, the convolutional neural network is trained specifically for the target scanning plane. The convolutional neural network acquired by the device differs for different target scanning planes.

[0075] For example, if the target scanning plane is the 2C scanning plane, since the multiple positioning images include P4C positioning images and PSAX positioning images, the device needs to obtain the convolutional neural network corresponding to the P4C positioning image and the convolutional neural network corresponding to the PSAX positioning image.

[0076] If the target scanning plane is a 3C scanning plane, since there are multiple positioning images including P2C positioning images, P4C positioning images and PSAX positioning images, it is determined that the device needs to acquire the convolutional neural network corresponding to the P2C positioning image, the convolutional neural network corresponding to the P4C positioning image, and the convolutional neural network corresponding to the PSAX positioning image.

[0077] If the target scanning plane is a 4C scanning plane, since the multiple positioning images include P2C positioning images and PSAX positioning images, the device needs to obtain the convolutional neural network corresponding to the P2C positioning image and the convolutional neural network corresponding to the PSAX positioning image.

[0078] If the target scanning plane is the SAX scanning plane, since the multiple positioning images include P2C positioning images and P4C positioning images, the device needs to obtain the convolutional neural network corresponding to the P2C positioning image and the convolutional neural network corresponding to the P4C positioning image.

[0079] It should be noted that the convolutional neural networks corresponding to the same type of localization image differ depending on the target scanning plane. For example, the convolutional neural network corresponding to the P4C localization image is different when the target scanning plane is the 2C scanning plane, compared to the convolutional neural network corresponding to the P4C localization image when the target scanning plane is the SAX scanning plane.

[0080] Assuming the target localization image is any localization image used to determine the target scanning plane, and the target convolutional neural network is the convolutional neural network corresponding to the target localization image among multiple convolutional neural networks, that is, the target convolutional neural network is used to determine the distance information of the localization image of the type to which the target localization image belongs, then the process of obtaining the target convolutional neural network is as follows:

[0081] Step A1: Obtain multiple training samples.

[0082] Each training sample may include historical distance information from historical localization images. This historical distance information can be determined based on the distance from each pixel in the historical localization image to a third intersection line, which is the line of intersection between the historical localization image and the historical scan plane.

[0083] The historical positioning image and the target positioning image share the same scanning angle, and the historical scanning plane and the target scanning plane to be determined share the same scanning angle. Among the historical distance information of multiple historical positioning images included in the multiple training samples, the historical distance information of any one historical positioning image is used to reflect the distance between each pixel in that historical positioning image and the third intersection line of that historical positioning image.

[0084] For example, assuming the target scanning plane is the 2C scanning plane, the target localization image is the P4C localization image, and the target convolutional neural network is the convolutional neural network corresponding to the P4C localization image, then each training sample acquired by the device can include: historical distance information of historical P4C localization images. This historical distance information can be determined based on the distance from each pixel in the historical P4C localization image to the third intersection line, which is the intersection line between the historical P4C localization image and the historical 2C scanning plane.

[0085] In this embodiment, each training sample may be pre-stored in the determining device. Alternatively, each training sample may be acquired by the determining device. The process of the determining device acquiring multiple training samples may include:

[0086] Step S1: Obtain multiple medical samples.

[0087] Each medical sample includes historical positioning data and historical scanning planes. The historical positioning data includes historical positioning images and their positions. The historical scanning planes can be represented by their positions. Furthermore, for the same medical sample, the positions of the historical positioning images and the historical scanning planes can be represented by two coordinates in the same three-dimensional coordinate system. This three-dimensional coordinate system can be the anatomical coordinate system of the target object or another coordinate system.

[0088] The historical location images mentioned above are independent of the location of the historical location images. It can be understood that the location of the historical location images can also be carried in the historical location images.

[0089] Optionally, the determining device can pre-store multiple medical samples. Alternatively, when the target neural network needs to be trained, the determining device can send a sample retrieval request to other devices storing medical samples to obtain multiple medical samples. This can save storage resources on the determining device.

[0090] Furthermore, the device or other equipment can store each medical sample based on historical positioning images and the spatial relationships between historical scanning planes. For example, the device or other equipment can store each medical sample according to the storage format defined by the Digital Imaging and Communications in Medicine (DICOM) protocol. In this way, it can be ensured that when the device acquires each medical sample, it can obtain the location of the historical positioning images included in that medical sample, as well as the location of the historical scanning planes.

[0091] Step S2: Based on multiple medical samples, obtain multiple training samples.

[0092] In this system, multiple medical samples correspond one-to-one with multiple training samples. The historical distance information of the historical positioning image in each training sample is determined based on the position of each pixel in the historical positioning image of the corresponding medical sample, as well as the position of the third intersection line. This third intersection line is the intersection line between the historical positioning image and the historical scanning plane in the medical sample corresponding to the training sample.

[0093] Assuming the first medical sample is any one of the multiple medical samples, and the first training sample is the training sample corresponding to the first medical sample, then the process by which the device obtains the first training sample based on the first medical sample includes the following steps:

[0094] Step S21: For the first medical sample, based on the position of the historical positioning image and the position of the historical scanning plane, determine the position of the third intersection line between the historical positioning image and the historical scanning plane.

[0095] In one alternative example, the position of the third intersection line can be represented by its position coordinates in the three-dimensional coordinate system of the historical location image in the first medical sample; in another alternative example, the position of the third intersection line can be represented by its position coordinates in the image coordinate system of the historical location image. Since the image coordinate system is a two-dimensional coordinate system, representing the position of the third intersection line with two-dimensional coordinates is more concise and facilitates subsequent determination of the distance from each pixel in the historical location image to the third intersection line.

[0096] Optionally, if the position of the third intersection line is represented by position coordinates in the image coordinate system, the determining device can determine the expression of the third intersection line with the historical scanning plane in the historical positioning image to characterize the position of the third intersection line. The general expression of the third intersection line satisfies the following formula:

[0097] Formula (1)

[0098] In formula (1), A , B and C All are constants. x 1 represents the x-coordinate value of each pixel in the third intersection line. y 1 represents the ordinate value of the pixel.

[0099] Optionally, the determining device can input the position of the historical positioning image and the position of the historical scanning plane into a deep convolutional neural network, thereby obtaining the position of the third intersection line between the historical positioning image and the historical scanning plane in the historical positioning image, as output by the deep convolutional neural network. This improves the efficiency of determining the position of the third intersection line.

[0100] The deep convolutional neural network is obtained by self-supervised learning of the spatial relationship between the device and historical positioning images and historical scanning planes. This self-supervised learning can refer to a training method for deep convolutional networks that constructs supervision signals and training tasks through the attributes of the data itself, and the training process does not require human intervention (such as manual annotation).

[0101] Step S22: Based on the position of each pixel in the historical positioning image and the position of the third intersection line, determine the historical distance information of the historical positioning image in the first training sample.

[0102] For each pixel in the historical positioning image, the determining device can determine the distance from the pixel to the third intersection line based on the position of the pixel and the position of the third intersection line, thereby obtaining the historical distance information of the historical positioning image.

[0103] Optionally, the historical distance information of the historical positioning image can be reflected by the historical heatmap of the historical positioning image. The historical heatmap has historical heatmap values ​​that correspond one-to-one with the pixels of the historical positioning image. Each historical heatmap value is used to reflect the distance from the corresponding pixel in the historical positioning image to the third intersection line.

[0104] Since heatmaps can intuitively reflect the magnitude of data, using historical heatmaps to reflect historical distance information of historical positioning images can ensure the intuitiveness of this historical distance information.

[0105] Optionally, each historical heatmap value can be negatively correlated with the distance from the pixel corresponding to that historical heatmap value to the third intersection line. For example, the historical heatmap value corresponding to each pixel included in the historical location image can satisfy the following formula:

[0106] Formula (2)

[0107] In formula (2), exp[] represents an exponential function with the natural constant e as its base. It is a hyperparameter, and the It can be 0.5 times the layer thickness of the historical scan plane, which can be determined by pre-selected storage in the device. x This represents the x-coordinate value of a pixel in a historical location image. y This is the ordinate value of the pixel.

[0108] Step A2: Train the target convolutional neural network based on multiple training samples.

[0109] Once the device acquires multiple training samples, it can train those samples to obtain the target convolutional neural network.

[0110] Optionally, the device can use a loss function to train the target convolutional neural network on the multiple training samples. Optionally, this training process involves updating the network parameters of the initial convolutional neural network through backpropagation until the loss function converges (i.e., the function value continuously decreases and eventually fluctuates within a preset numerical range) to obtain the target convolutional neural network. The loss function, also called the cost function, is used to evaluate the accuracy of the convolutional neural network. For example, this loss function is the L2 loss function. The target convolutional neural network can be a U-Net convolutional neural network, a Res-U-Net convolutional neural network, or a Dense-U-Net convolutional neural network.

[0111] Furthermore, since the historical positioning image can also be used to determine other scanning planes, in order to ensure the reliability of the target convolutional neural network trained, each training sample in step A1 can also include other distance information of the historical positioning image. This other distance information can be determined based on the distance of each pixel in the historical positioning image to other intersection lines, which are the intersection lines of the historical positioning image with other scanning planes. The other distance information of any historical positioning image is used to reflect the distance of each pixel in that historical positioning image to other intersection lines of that historical positioning image. Then, in step A2, the determining device can use the L2 loss function to train the training sample to obtain multiple more reliable convolutional neural networks, which include the target convolutional neural network.

[0112] The process of determining other distance information of the historical positioning image acquired by the device can refer to the relevant implementation process of determining the historical distance information of the historical positioning image acquired by the device in S21 and S22 above, and will not be repeated here in this embodiment. Furthermore, this other distance information can be reflected by other heatmaps of the historical positioning image. These other heatmaps have other heatmap values ​​that correspond one-to-one with the pixels of the historical positioning image. Each other heatmap value reflects the distance from the corresponding pixel in the historical positioning image to the other intersection line. Optionally, each other heatmap value can be negatively correlated with the distance from the pixel corresponding to the other heatmap value to the other intersection line.

[0113] In this scenario, where historical distance information for historical positioning images is reflected by historical heatmaps, and other distance information is reflected by other heatmaps, and the heatmap values ​​are all negatively correlated with the corresponding distances, the L2 loss function can satisfy the following formula:

[0114] Formula (3)

[0115] In formula (3), T is the total number of different scanning planes involved in the determination of the historical positioning image, which includes the historical scanning plane and other scanning planes. The historical location image includes the total number of pixels. (x, y) represents each pixel in this historical location image. H t (x,y) The heatmap of the historical positioning image under the t-th scanning plane in multiple different scanning planes. This is a predicted heatmap of the historical localization image predicted by the convolutional neural network under the t-th scan plane. t is an integer greater than or equal to 1 and less than or equal to T.

[0116] For example, suppose the target scanning plane to be determined is the 2C scanning plane, and the historical positioning image is the historical P4C positioning image. Since the historical P4C positioning image is involved in determining not only the historical 2C scanning plane, but also the historical 3C scanning plane and the historical SAX scanning plane, in order to ensure the reliability of the convolutional neural network corresponding to the P4C positioning image trained when the target scanning plane is the 2C scanning plane, the determining device can also obtain the heat map of the historical P4C positioning image under the historical 3C scanning plane and the heat map of the historical P4C positioning image under the historical SAX scanning plane. Then, the determining device can train the convolutional neural network corresponding to the P4C positioning image when the target scanning plane is the 2C scanning plane, the convolutional neural network corresponding to the P4C positioning image when the target scanning plane is the 3C scanning plane, and the convolutional neural network corresponding to the P4C positioning image when the target scanning plane is the SAX scanning plane, based on the above formula (3).

[0117] The methods for obtaining each heatmap of the historical P4C positioning image under the historical 2C scanning plane, the historical P4C positioning image under the historical 3C scanning plane, and the historical P4C positioning image under the historical SAX scanning plane can all refer to the methods for obtaining the heatmaps described in steps S1 and S2 above.

[0118] As can be seen from the above description, the method provided in this application embodiment can perform self-supervised learning on the spatial relationship between historical positioning images and historical scanning planes in medical samples during the training of the convolutional neural network corresponding to the positioning image. This enables the training of the convolutional neural network without relying on manual annotation, and then the automatic planning of the target scanning plane can be achieved based on the convolutional neural network, thus effectively improving the efficiency of determining the target scanning plane.

[0119] Step 202: Obtain multiple positioning images.

[0120] The multiple positioning images can be of different types, and are determined based on multiple scan images. For example, each positioning image is determined based on multiple scan images obtained from scanning the human body from different angles. For instance, multiple electronic scanning planes can be obtained by scanning the human body from the coronal plane (i.e., directly in front or behind), sagittal plane (i.e., left or right side), and axial plane (i.e., directly above or below). The human body includes the heart.

[0121] Optionally, the device can input multiple scanned images into a neural network to obtain multiple localization images output by the neural network. This can improve the efficiency of acquiring multiple localization images.

[0122] In this embodiment, the multiple positioning images acquired by the determination device can correspond one-to-one with the multiple convolutional neural networks acquired in step 201. That is, if the target scanning plane of the heart to be determined is a 2C scanning plane, then acquiring the multiple convolutional networks in step 201 includes: a convolutional neural network for processing P4C type positioning images when the target scanning plane is a 2C scanning plane, and a convolutional neural network for processing PSAX type positioning images. Accordingly, the multiple positioning images include: P4C positioning images and PSAX positioning images.

[0123] Step 203: Input each localization image into the corresponding convolutional neural network to obtain the distance information of the localization image output by the convolutional neural network.

[0124] As mentioned earlier, each of the multiple acquired positioning images can correspond to a convolutional neural network acquired in step 201. Still assuming the target positioning image is any one of the multiple positioning images used to determine the target scanning plane, and the target convolutional neural network is the convolutional neural network corresponding to that target positioning image, that is, the target convolutional neural network is the positioning image used to process the type to which the target positioning image belongs.

[0125] The device can input the target image into a target convolutional neural network to obtain distance information of the target localization image output by the target convolutional neural network. This distance information of the target localization image is used to reflect the distance of each pixel in the target localization image to the first intersection line of the target localization image, which is the intersection line of the target localization image with the target scanning plane of the heart to be determined.

[0126] Optionally, the distance information of the target positioning image can also be reflected by a heatmap of the target positioning image under the target scanning plane. This ensures the intuitiveness of the distance information of the target positioning image.

[0127] The heatmap of the target location image has heat values ​​that correspond one-to-one with each pixel of the target location image. Each heat value reflects the distance from the corresponding pixel to the first intersection line between the pixel and the scanning plane of the target to be determined in the target location image. Optionally, each heat value is negatively correlated with the distance from the pixel corresponding to the heat value to the first intersection line.

[0128] For example, see Figure 3 A circle with a number n represents a convolutional neural network n, where n is an integer greater than or equal to 1. Furthermore, to facilitate quick differentiation for the reader... Figure 3 Different target scanning plane determination processes, in Figure 3 The process of determining different target scanning planes is represented by lines of different thicknesses.

[0129] If the target scanning plane is a 2C scanning plane, refer to Figure 3 The device can input the P4C positioning image into convolutional neural network 1 and the PSAX positioning image into convolutional neural network 2 to obtain the thermal image of the P4C positioning image under the 2C scanning plane output by convolutional neural network 1. Figure 1 The thermal analysis of PSAX localization images under the 2C scan plane output by convolutional neural network 2 Figure 2 When the target scanning plane is the 2C scanning plane, the convolutional neural network corresponding to the P4C positioning image is convolutional neural network 1, and the convolutional neural network corresponding to the PSAX positioning image is convolutional neural network 2.

[0130] If the target scanning plane is a 3C scanning plane, then as follows Figure 3 As shown, the device can input the P2C positioning image into convolutional neural network 3, the P4C positioning image into convolutional neural network 4, and the PSAX positioning image into convolutional neural network 5 to obtain the thermal image of the P2C positioning image under the 3C scanning plane output by the convolutional neural network 3. Figure 3 The thermal analysis of the P4C localization image under the 3C scanning plane output by the convolutional neural network 4 Figure 4 And the thermal analysis of the PSAX localization image under the 3C scanning plane output by the convolutional neural network 5. Figure 5 When the target scanning plane is the 3C scanning plane, the convolutional neural network corresponding to the P2C positioning image is convolutional neural network 3, the convolutional neural network corresponding to the P4C positioning image is convolutional neural network 4, and the convolutional neural network corresponding to the PSAX positioning image is convolutional neural network 5.

[0131] Please continue reading Figure 3If the target scanning plane is the 4C scanning plane, then the device can input the P2C positioning image into convolutional neural network 6 and the PSAX positioning image into convolutional neural network 7 to obtain the thermal image of the P2C positioning image under the 4C scanning plane output by convolutional neural network 6. Figure 6 The thermal analysis of PSAX localization images under the 4C scan plane output by the convolutional neural network 7 Figure 7 When the target scanning plane is the 4C scanning plane, the convolutional neural network corresponding to the P2C positioning image is convolutional neural network 6, and the convolutional neural network corresponding to the PSAX positioning image is convolutional neural network 7.

[0132] like Figure 3 As shown, if the target scanning plane is the SAX scanning plane, the device can input the P2C positioning image into convolutional neural network 8 and the P4C positioning image into convolutional neural network 9, obtaining a heatmap 8 of the P2C positioning image under the SAX scanning plane output by convolutional neural network 8 and a heatmap 9 of the P4C positioning image under the SAX scanning plane output by convolutional neural network 9. Specifically, when the target scanning plane is the SAX scanning plane, the convolutional neural network corresponding to the P2C positioning image is convolutional neural network 8, and the convolutional neural network corresponding to the P4C positioning image is convolutional neural network 9.

[0133] Step 204: Based on the distance information of multiple positioning images, perform the scanning plane determination process.

[0134] For example, once the device has determined the distance information for each of multiple positioning images, it can perform the scan plane determination process. See also Figure 4 The scanning plane determination process may include:

[0135] Step 2041: Obtain multiple candidate planes.

[0136] Each candidate plane can be a plane in the space where the heart is located that intersects with multiple localization images. Furthermore, each candidate plane can be determined based on position parameters, and the position parameters of any two candidate planes are different. These position parameters include: a spatial point p, a polar angle θ, and a direction angle φ. For example, each candidate plane P can be labeled as (p, θ, φ).

[0137] Optionally, since an image intersects with multiple intersecting images, it can include pixels along any intersection line of those intersecting images. Based on this, the spatial point can be a pixel along the intersection line of any two positioning images in the space where the heart is located. This effectively reduces the computational complexity of the determining device and improves the efficiency of determining the target scanning plane.

[0138] For example, if multiple positioning images include a P4C positioning image and a PSAX positioning image, then the spatial point of each candidate plane can be a pixel point in the intersection line of the P4C positioning image and the PSAX positioning image.

[0139] Step 2042: Based on the distance information of multiple positioning images, obtain the target distance of multiple candidate planes.

[0140] For each candidate plane, since it intersects with each of the multiple positioning images, the target distance of the candidate plane can be the sum of the reference distances of the multiple positioning images to ensure the reliability of the determined target scanning plane. Specifically, the reference distance of any positioning image is the sum of the distances from each pixel of the second intersection line of that positioning image to the first intersection line of that positioning image. The second intersection line of that positioning image is the intersection line with the candidate plane in that positioning image.

[0141] In this embodiment of the application, for a scenario where the distance information of any positioning image is reflected by the heat map of that positioning image, the reference distance of any positioning image among multiple positioning images is the sum of the heat values ​​corresponding to each pixel of the second intersection line of that positioning image on the heat map of that positioning image.

[0142] That is, the target distance for each candidate plane satisfies the following formula:

[0143] Formula (4)

[0144] Where V is the total number of multiple location images, This is a heatmap of the v-th localization image among multiple localization images, located on the target scanning plane. v is an integer greater than or equal to 1 and less than or equal to V. l v This is the second intersection line between the v-th localization image and the candidate plane. l v This can be represented by the set of all pixels on the second intersection line in the v-th positioning image, that is, the... l v Satisfy the following formula:

[0145] Formula (5)

[0146] In formula (5), P is the candidate plane. For the v-th localization image among multiple localization images, Indicate P and intersect.

[0147] For examples, please continue to refer to them. Figure 3If the target scanning plane is a 2C scanning plane, then the device can be determined based on... Figure 3 The heat in Figure 1 and heat Figure 2 Determine the target distance for each candidate plane. If the target scanning plane is a 3C scanning plane, then determine the device can be based on... Figure 3 The heat in Figure 3 To heat Figure 5 The target distance for each candidate plane is determined. If the target scanning plane is a 4C scanning plane, then the device can be determined based on... Figure 3 The heat in Figure 6 and heat Figure 7 The target distance for each candidate plane is determined. If the target scanning plane is an SAX scanning plane, then the device can be determined based on... Figure 3 Heatmaps 8 and 9 are used to determine the target distance for each candidate plane.

[0148] Step 2043: Among multiple candidate planes, the candidate plane with the smallest target distance is determined as the scanning plane.

[0149] After determining the target distance of each candidate plane among multiple candidate planes, the determining device can compare the magnitudes of the multiple target distances of the multiple candidate planes to determine the candidate plane with the smallest target distance from the multiple candidate planes, and determine the candidate plane with the smallest target distance as the scanning plane in the current scanning plane determination process.

[0150] For any location image whose distance information is reflected by its heatmap, and in a scenario where a certain heatmap value is negatively correlated with the distance from the corresponding pixel to the first intersection line, the determining device can identify the candidate plane with the largest sum of heatmap values ​​corresponding to all pixels of all intersection lines of all location images among multiple candidate planes as the scanning plane.

[0151] In this embodiment, for scenarios where the target number is an integer greater than 1 (i.e., when the determining device performs multiple scanning plane determination processes based on distance information from multiple positioning images), adjacent scanning plane determination processes are related. For example, suppose any two adjacent scanning plane determination processes are a first scanning plane determination process and a second scanning plane determination process, respectively, with the first scanning plane determination process preceding the second scanning plane determination process, and the first scanning plane being the scanning plane determined by the first scanning plane determination process. Then, the multiple candidate planes acquired by the determining device during the second scanning plane determination process can be determined based on this first scanning plane.

[0152] For example, this relationship is a pyramid search relationship, which means that in the process of determining two adjacent scanning planes, the range of the parameter value set to which the same parameter in the position parameters belongs decreases successively, and the difference between adjacent parameter values ​​in the parameter value set decreases successively. Therefore, at least one parameter in the position parameters used to determine the second candidate plane satisfies this pyramid search relationship.

[0153] For each parameter x that satisfies the pyramid search relation, assume that the first parameter value set is the set of parameter values ​​for parameter x used to determine the first candidate plane, and the second parameter value set is the set of parameter values ​​for parameter x used to determine the second candidate plane. The first candidate plane is the candidate plane obtained during the determination of the first scan plane, and the second candidate plane is the candidate plane obtained during the determination of the second scan plane. The parameter values ​​in both the first and second parameter value sets are arranged in an arithmetic sequence. Then the pyramid search relation satisfies the following: the parameter value of parameter x is the value between the target parameter value and its adjacent parameter values ​​in the first parameter value set, and the tolerance of the parameter values ​​in the second parameter value set is less than the tolerance of the parameter values ​​in the first parameter value set. For example, parameter x can be a spatial point p, a polar angle θ, or a direction angle φ.

[0154] It should be noted that if the target parameter value is an endpoint value in the first parameter set, the number of parameter values ​​adjacent to that target parameter value is 1; if the target parameter value is a non-endpoint value in the first parameter set, the number of parameter values ​​adjacent to that target parameter value is 2. For example, if the parameter is a polar angle, the first parameter set is {65°, 70°, 75°, 80°, 85°}, and the target parameter value is 65°, then the only parameter value adjacent to 65° is 70°. If the target parameter value is 70°, then the parameter values ​​adjacent to 70° include 65° and 75°.

[0155] As can be seen from the above description, the determining device provided in this application embodiment can use a pyramid search method to determine the target scanning plane, so that the determining device successively narrows the range of the parameter set to which the parameters used to determine the candidate plane belong, and the tolerance of the parameter values ​​in the parameter value set successively narrows, thereby achieving accurate and efficient positioning of the target scanning plane.

[0156] As mentioned earlier, the position parameters include three parameters: spatial point p, polar angle θ, and orientation angle φ. When parameter x is a spatial point, this parameter is the coordinate of the spatial point. When parameter x is a polar angle, this parameter is the angle of the polar angle. When parameter x is an orientation angle, this parameter is the angle of the polar angle, and the difference is the difference between the angles.

[0157] It should also be noted that during the multiple scanning plane determination process, the multiple candidate planes obtained in the first scanning plane determination process can be determined by the determining device from multiple spatial points in the space where the heart is located, the angle range of the polar angle, and the angle range of the azimuth angle, based on pre-stored preset differences. The angle range of the polar angle is [0, 180°], and the angle range of the azimuth angle is [0, 360°]. ° represents degrees.

[0158] Figure 5 This is a schematic diagram of a pyramid search process provided in an embodiment of this application. Assuming the target number is 3, the spatial point p, polar angle θ, and orientation angle φ all satisfy the pyramid search relationship. For each parameter among the spatial point p, polar angle θ, and orientation angle φ, in the first scanning plane determination process, the tolerance of the parameter value set used to determine the candidate plane is 15; in the second scanning plane determination process, the tolerance of the parameter value set used to determine the candidate plane is 5; and in the third scanning plane determination process, the tolerance of the parameter value set used to determine the candidate plane is 1.

[0159] Assuming that the multiple pixels in the intersection line of any two positioning images are: point 0 to point 90, then the set of spatial points used to determine multiple candidate planes during the first scan is {point 0, point 15, point 30, point 45, point 60, point 75, point 90}, the set of polar angles is {0°, 15°, 30°, 45°, ..., 165°, 180°}, and the set of orientation angles is {0°, 15°, 30°, 45°, ..., 345°, 360°}.

[0160] If, during the initial scan plane determination process, the determined scan plane is a plane with a spatial point of 45°, a polar angle of 75°, and a direction angle of 150°, then... Figure 5 As shown, the determining device identifies pixels between points 30 and 60 as the multiple candidate planes used in the second scanning plane determination process, with polar angles between 60° and 90° and azimuth angles between 135° and 165°. Subsequently, the determining device can determine that, in the second scanning plane determination process, the spatial point set used to determine the multiple candidate planes is {point 35, point 40, point 45, point 50, point 55}, the polar angle set is {65°, 70°, 75°, 80°, 85°}, and the azimuth angle set is {140°, 145°, 150°, 155°, 160°}.

[0161] If, during the second scan plane determination process, the determined scan plane is a plane with spatial point 45, a polar angle of 70°, and a direction angle of 145°, then continue to refer to... Figure 5The determining device identifies pixels between points 40 and 50 as the multiple candidate planes used in the second scanning plane determination process, with polar angles between 65° and 75° and azimuth angles between 140° and 150°. Subsequently, the determining device can further define the spatial point set used to determine the multiple candidate planes during the second scanning plane determination process as {point 41, point 42, point 43, ..., point 48, point 49}, the polar angle set as {66°, 67°, 68°, ..., 73°, 74°}, and the azimuth angle set as {141°, 142°, 143°, ..., 148°, 149°}.

[0162] Step 205: Check if the cutoff condition is met.

[0163] The cutoff condition can be that the number of times the scan plane determination process is executed equals the target number. Optionally, the target number can be 1. Alternatively, the target number can be an integer greater than 1. For example, the target number can be 3.

[0164] If the device is determined to meet the cutoff condition, step 207 can be executed. If the device is determined not to meet the cutoff condition, step 205 can be executed again until the cutoff condition is met.

[0165] Step 206: Determine the scan plane obtained from the last scan plane determination process as the target scan plane.

[0166] In this embodiment of the application, if the device determines that the cutoff condition is met, the scan plane obtained by performing the last scan plane determination process is determined as the target scan plane.

[0167] For scenarios where the target number of scans is 1, the device can directly determine the scan plane determined after performing one scan plane determination process as the target scan plane.

[0168] For scenarios where the target number is an integer greater than 1, the device can determine the target scan plane as the scan plane obtained during the process of determining the last scan plane.

[0169] Optionally, after determining the target scanning plane, the device can display the position of the target scanning plane by showing the position of the intersection line between the target scanning plane and its corresponding positioning image in the positioning image.

[0170] For example, see Figure 3 After determining the 2C scanning plane, the device can display the position of the intersection line 01 between the 2C scanning plane and the P4C positioning image in the P4C positioning image, and the position of the intersection line 02 between the 2C scanning plane and the PSAX positioning image in the PSAX positioning image.

[0171] After determining the 3C scanning plane, the device can display the position of the intersection line 03 between the 3C scanning plane and the P2C positioning image in the P2C positioning image, the position of the intersection line 04 between the 3C scanning plane and the P4C positioning image in the P4C positioning image, and the position of the intersection line 05 between the 3C scanning plane and the PSAX positioning image in the PSAX positioning image.

[0172] After determining the 4C scanning plane, the device can display the position of the intersection line 06 between the 4C scanning plane and the P2C positioning image in the P2C positioning image, and the position of the intersection line 07 between the 4C scanning plane and the PSAX positioning image in the PSAX positioning image.

[0173] After determining the SAX scanning plane, the device can display the position of the intersection line 08 between the SAX scanning plane P2C and the positioning image in the P2C positioning image, and the position of the intersection line 09 between the SAX scanning plane and the P4C positioning image in the P4C positioning image.

[0174] In related technologies, to improve the efficiency of determining the scanning plane and to avoid the problem of low reliability of the determined scanning plane due to human error, one of the following methods can be adopted:

[0175] Method 1: First, a 3D model of the heart is established. Then, this 3D model is input into a first keypoint detection network to obtain a 3D model marked with keypoints. Next, the scanning plane is determined based on this 3D model with marked keypoints. This first keypoint detection network is trained on a large number of historical 3D labeled models, which are obtained by marking keypoints on historical 3D models during the training process. These keypoints may include the apex of the left ventricle.

[0176] Method 2: During the human body scanning process, staff determine the scanning plane based on experience and continuously input the determined and accurate scanning plane into the scanning equipment for the equipment to learn from, thereby enabling the scanning equipment to automatically determine the scanning plane. Afterward, the scanning equipment can automatically determine the scanning plane.

[0177] Method 3: The localization image of the heart is input into a pre-trained second keypoint network to obtain a labeled image with key points output by the keypoint network. The scanning plane can then be determined based on this labeled image. The second keypoint network is trained using a large number of historical localization images and their corresponding labeled images. The labeled images are obtained by marking key points on historical localization images during the training of the keypoint network. The localization image is determined by scanning the heart of the target object using the first scanning plane.

[0178] However, the aforementioned techniques all require manual labeling of key points or manual determination of the scanning plane, resulting in low efficiency and reliability. The method provided in this application, however, utilizes self-supervised learning of the spatial relationship between historical positioning images and historical scanning planes in the medical sample during the training of the convolutional neural network corresponding to the positioning image. This process eliminates the need for manual labeling of key points or manual determination of the scanning plane, thus completely eliminating manual intervention. This improves the efficiency and reliability of acquiring multiple convolutional neural networks, thereby effectively enhancing the efficiency and reliability of determining the target scanning plane.

[0179] In this embodiment of the application, a dataset was used to test the method provided in this embodiment of the application, and the error between the target scanning surface and the actual scanning plane (i.e. the real scanning plane) obtained by the test was compared with the error between the scanning plane automatically determined by the above method and the actual scanning plane.

[0180] The comparison results are shown in Table 1. This error can be used to evaluate two indicators: normal deviation and point-to-plane distance. Furthermore, this error is positively correlated with both the smaller the normal deviation and the smaller the point-to-plane distance. In other words, the smaller the normal deviation and the smaller the point-to-plane distance, the smaller the error.

[0181] The normal deviation can refer to the absolute angular difference between the normals of the automatically planned scanning plane and the actual scanning plane, expressed in degrees (°). The point-to-surface distance can refer to the distance from the center of the actual scanning plane to the automatically planned scanning plane, expressed in millimeters (mm). The center of the actual scanning plane can be the center of the image obtained by scanning along this direction. The automatically planned scanning plane includes the target scanning plane determined by the method provided in the embodiments of this application, as well as scanning planes automatically determined by other methods.

[0182] This dataset contains 181 clinical CMR scans from 99 patients, with 82 patients having two scans and the other patients having only one scan. All CMR data were acquired using a 1.5T Magnetic Resonance Imaging (MRI) system.

[0183] Table 1

[0184]

[0185] In Table 1, '-' indicates that a certain method has no relevant results. '*' indicates a statistically significant difference.

[0186] As can be seen from Table 1, the method provided in this application embodiment achieves the best average results in both metrics. For the four different target scanning planes, the method provided in this application embodiment achieves the minimum point-to-surface distance. Furthermore, for the 2C and 3C scanning planes, the method provided in this application embodiment achieves the minimum normal deviation, while for the 4C and SAX scanning planes, the method provided in this application embodiment also achieves the second smallest normal deviation. Therefore, it is evident that the method provided in this application embodiment determines the scanning plane with higher accuracy than other methods.

[0187] It should be noted that the order of the steps in the scanning plane determination method provided in this application embodiment can be appropriately adjusted, and the steps can also be added or removed as appropriate. For example, step 201, which determines that the device has already stored a trained convolutional neural network, can also be deleted as appropriate. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0188] In summary, this application provides a method for determining a scanning plane. The determining device can predict distance information from multiple positioning images. This distance information reflects the distance between each pixel in the positioning image and a first intersection line of the positioning image, which is the intersection line between the positioning image and the target scanning plane to be determined. Then, the determining device can automatically determine the target scanning plane based on the distance information from the multiple positioning images. Since the target scanning plane is automatically determined, there is no need for manual determination based on experience. Therefore, on the one hand, it reduces the workload of operators and effectively improves the efficiency of scanning plane determination; on the other hand, it avoids the problem of low accuracy in determining the target scanning plane due to human error, thereby ensuring the accuracy of the scanned image obtained based on the scanning plane.

[0189] Figure 6This application also provides a structural block diagram of a scanning plane determination device, which can be configured in the determination device provided in the above method embodiments. See also... Figure 6 The determining device 300 may include:

[0190] The acquisition module 301 is used to acquire multiple positioning images. The positioning images are determined based on multiple scan images. The multiple scan images are obtained by scanning the target object from different angles. The target object includes target parts.

[0191] The prediction module 302 is used to predict the distance information of multiple positioning images. The distance information of any positioning image is used to reflect the distance between each pixel in any positioning image and the first intersection line of any positioning image. The first intersection line of any positioning image is the intersection line of the target scanning plane of the target part to be determined in any positioning image.

[0192] The determination module 303 is used to determine the target scanning plane based on the distance information of multiple positioning images.

[0193] Optionally, the determining module 303 can be used to:

[0194] Based on the distance information from multiple positioning images, perform at least one scanning plane determination process until the cutoff condition is met;

[0195] The scan plane obtained by performing the second scan plane determination process is determined as the target scan plane;

[0196] The scanning plane determination process includes:

[0197] Multiple candidate planes are obtained, each of which is a plane in the space where the target part is located that intersects with multiple localization images;

[0198] Based on the distance information of multiple positioning images, the target distance of each candidate plane in multiple candidate planes is obtained. For each candidate plane, the target distance of the candidate plane is the sum of the reference distances of multiple positioning images. The reference distance of any positioning image is the sum of the distances from each pixel of the second intersection line of any positioning image to the first intersection line of any positioning image. The second intersection line of any positioning image is the intersection line of any positioning image with the candidate plane.

[0199] Among multiple candidate planes, the candidate plane with the smallest target distance is determined as the scanning plane.

[0200] Optionally, the candidate plane is determined based on position parameters. The position parameters of any two candidate planes are different. The position parameters include: spatial point, polar angle, and orientation angle.

[0201] Here, a spatial point is a pixel on the intersection line of any two positioning images in the space where the target part is located.

[0202] Optionally, the determining module 303 can be used to perform multiple scanning plane determination processes based on distance information from multiple positioning images;

[0203] In the sequentially adjacent first scan plane determination process and second scan plane determination process, multiple candidate planes in the second scan plane determination process are determined based on the first scan plane, and the first scan plane is the scan plane determined by the first scan plane determination process;

[0204] Wherein, at least one of the position parameters used to determine the second candidate plane satisfies:

[0205] The parameter value is the value between the target parameter value and the adjacent parameter values ​​in the first parameter set, and the tolerance of the parameter values ​​in the second parameter value set is less than the tolerance of the parameter values ​​in the first parameter value set;

[0206] The target parameter value is the parameter value of the first scanning plane. The first parameter value set is the parameter value set used to determine the first candidate plane. The second parameter value set is the parameter value set used to determine the second candidate plane. The parameter values ​​in the first parameter value set and the second parameter value set are arranged in an arithmetic sequence. The first candidate plane is the candidate plane obtained in the process of determining the first scanning plane. The second candidate plane is the candidate plane obtained in the process of determining the second scanning plane.

[0207] Optionally, the distance information of any positioning image is reflected by a heatmap of any positioning image. The heatmap has heat values ​​that correspond one-to-one with the pixels of any positioning image. Each heat value is used to reflect the distance from the corresponding pixel to the first intersection line of any positioning image.

[0208] The reference distance for any location image is the sum of the heat values ​​of each pixel on the heatmap corresponding to the second intersection line of any location image.

[0209] Optionally, each thermal value is negatively correlated with the distance from the pixel corresponding to the thermal value to the first intersection line.

[0210] Optionally, multiple localization images correspond one-to-one with multiple convolutional neural networks, and the prediction module 302 can be used for:

[0211] The target localization image is input into the target convolutional neural network to obtain the distance information of the target localization image output by the target convolutional neural network. The target convolutional neural network is used to process the localization image of the type to which the target localization image belongs, and the target localization image is any one of multiple localization images.

[0212] Optionally, the acquisition module 301 can also be used for:

[0213] Multiple training samples are acquired, including historical distance information of historical positioning images. The historical distance information is determined based on the distance from each pixel in the historical positioning image to the third intersection line. The third intersection line is the intersection line between the historical positioning image and the historical scanning plane. The historical positioning image and the target positioning image have the same scanning angle, and the historical scanning plane and the target scanning plane have the same scanning angle. Based on the multiple training samples, the target convolutional neural network is trained.

[0214] It should be noted that the acquisition module 301 may include a first acquisition unit and a second acquisition unit. The first acquisition unit can be used to acquire multiple localization images, and the second acquisition unit can be used to acquire multiple training samples, and train a convolutional neural network based on the multiple training samples.

[0215] Optionally, the second acquisition unit in the acquisition module 301 can be used for:

[0216] Acquire multiple medical samples, including historical localization images and historical scan planes;

[0217] Multiple training samples were obtained based on multiple medical samples;

[0218] Among them, multiple medical samples correspond one-to-one with multiple training samples. The historical distance information of the historical positioning images in the training samples is determined based on the position of each pixel included in the historical positioning image of the corresponding medical sample, as well as the position of the third intersection line. The third intersection line is the intersection line between the historical positioning image and the historical scanning plane in the corresponding medical sample.

[0219] In summary, this application provides a scanning plane determination device that can predict distance information from multiple positioning images. This distance information reflects the distance between each pixel in the positioning image and a first intersection line of the positioning image, where the first intersection line is the line of intersection between the positioning image and the target scanning plane to be determined. The device can then automatically determine the target scanning plane based on the distance information from the multiple positioning images. Since the target scanning plane is automatically determined, eliminating the need for manual determination based on experience, it reduces operator workload and effectively improves the efficiency of scanning plane determination. Furthermore, it avoids the problem of low accuracy in determining the target scanning plane due to human error, thereby ensuring the accuracy of the scanned image obtained based on the scanning plane.

[0220] Embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the scan plane determination method provided in the above-described method embodiments.

[0221] Optionally, the computer device is a specific device. For example, Figure 7 This is a schematic diagram of the structure of a determining device provided in an exemplary embodiment of this application. Typically, the determining device 400 includes a processor 401 and a memory 402.

[0222] Processor 401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0223] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 are used to store at least one instruction, which is executed by the processor 401 to implement the scan plane determination method provided in the method embodiments of this application.

[0224] In some embodiments, the terminal 400 may also optionally include a peripheral device interface 403 and at least one peripheral device. The processor 401, memory 402, and peripheral device interface 403 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 404, a display screen 405, a camera assembly 406, an audio circuit 407, a positioning assembly 408, and a power supply 409.

[0225] Peripheral device interface 403 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 401 and memory 402. In some embodiments, processor 401, memory 402 and peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 401, memory 402 and peripheral device interface 403 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0226] The radio frequency (RF) circuit 404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 404 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 404 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 404 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0227] Display screen 405 is used to display a UI (User Interface, horizontal level interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 405 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 401 for processing. In this case, display screen 405 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 405, serving as the front panel of terminal 400; in other embodiments, there may be at least two display screens 405, respectively disposed on different surfaces of terminal 400 or in a folded design; in still other embodiments, display screen 405 may be a flexible display screen, disposed on a curved or folded surface of terminal 400. Furthermore, display screen 405 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 405 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0228] The camera assembly 406 is used to acquire images or videos. Optionally, the camera assembly 406 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal 400, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 406 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0229] The audio circuit 407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting them into electrical signals that are input to the processor 401 for processing, or to the radio frequency circuit 404 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 400. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 407 may also include a headphone jack.

[0230] The positioning component 408 is used to determine the current geographical location of the terminal 400 in order to enable navigation or LBS (Location Based Service). The positioning component 408 can be a positioning component based on GPS (Global Positioning System), BeiDou system, or Galileo system.

[0231] Power supply 409 is used to power the various components in terminal 400. Power supply 409 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 409 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0232] In some embodiments, the terminal 400 further includes one or more sensors 410. The one or more sensors 410 include, but are not limited to: an accelerometer 411, a gyroscope 412, a pressure sensor 413, a fingerprint sensor 414, an optical sensor 415, and a proximity sensor 416.

[0233] Accelerometer 411 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by terminal 400. For example, accelerometer 411 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 401 can control touch screen 405 to display a horizontal or vertical view of the level interface based on the gravitational acceleration signal collected by accelerometer 411. Accelerometer 411 can also be used for collecting game or user motion data.

[0234] The gyroscope sensor 412 can detect the orientation and rotation angle of the terminal 400. The gyroscope sensor 412, in conjunction with the accelerometer sensor 411, can collect 3D motion data from the user on the terminal 400. Based on the data collected by the gyroscope sensor 412, the processor 401 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0235] The pressure sensor 413 can be disposed on the side bezel of the terminal 400 and / or on the lower layer of the touch display screen 405. When the pressure sensor 413 is disposed on the side bezel of the terminal 400, it can detect the user's grip signal on the terminal 400, and the processor 401 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed on the lower layer of the touch display screen 405, the processor 401 can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 405. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0236] The fingerprint sensor 414 is used to collect the user's fingerprint. The processor 401 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as trusted, the processor 401 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 414 can be located on the front, back, or side of the terminal 400. When the terminal 400 has physical buttons or a manufacturer's logo, the fingerprint sensor 414 can be integrated with the physical buttons or manufacturer's logo.

[0237] An optical sensor 415 is used to collect ambient light intensity. In one embodiment, the processor 401 can control the display brightness of the touch screen 405 based on the ambient light intensity collected by the optical sensor 415. Specifically, when the ambient light intensity is high, the display brightness of the touch screen 405 is increased; when the ambient light intensity is low, the display brightness of the touch screen 405 is decreased. In another embodiment, the processor 401 can also dynamically adjust the shooting parameters of the camera assembly 406 based on the ambient light intensity collected by the optical sensor 415.

[0238] The proximity sensor 416, also known as a distance sensor, is typically located on the front panel of the terminal 400. The proximity sensor 416 is used to detect the distance between the user and the front of the terminal 400. In one embodiment, when the proximity sensor 416 detects that the distance between the user and the front of the terminal 400 is gradually decreasing, the processor 401 controls the touchscreen display 405 to switch from a screen-on state to a screen-off state; when the proximity sensor 416 detects that the distance between the user and the front of the terminal 400 is gradually increasing, the processor 401 controls the touchscreen display 405 to switch from a screen-off state to a screen-on state.

[0239] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the defined device 400, and may include more or fewer components than shown, or combine certain components, or employ different component arrangements.

[0240] This application also provides a computer-readable storage medium storing at least one piece of program code. When the program code is loaded and executed by the processor of a computer device, it implements the scanning plane determination method provided in the above-described method embodiments.

[0241] This application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the scan plane determination method provided in the above-described method embodiments.

[0242] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0243] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining a scanning plane, characterized in that, The method includes: Multiple positioning images are acquired, the positioning images being determined based on multiple scan images obtained by scanning a target object from different angles, the target object including target parts; The target positioning image is input into the target convolutional neural network to obtain the distance information of the target positioning image predicted by the target convolutional neural network. The target convolutional neural network is used to process positioning images of the type to which the target positioning image belongs. The target positioning image is any one of the plurality of positioning images. The distance information of any one positioning image is used to reflect the distance of each pixel in the any one positioning image to the first intersection line of the any one positioning image. The first intersection line of the any one positioning image is the intersection line of the any one positioning image with the target scanning plane of the target part to be determined. The target scanning plane is determined based on the distance information of the multiple positioning images.

2. The method according to claim 1, characterized in that, Determining the target scanning plane based on the distance information of the multiple positioning images includes: Based on the distance information of the multiple positioning images, perform at least one scanning plane determination process until the cutoff condition is met; The scan plane obtained by performing the last scan plane determination process is determined as the target scan plane; The scanning plane determination process includes: Multiple candidate planes are obtained, and each candidate plane is a plane in the space where the target part is located that intersects with all of the multiple positioning images; Based on the distance information of the plurality of positioning images, the target distance of each of the plurality of candidate planes is obtained. For each candidate plane, the target distance of the candidate plane is the sum of the reference distances of the plurality of positioning images. The reference distance of any positioning image is the sum of the distances from each pixel of the second intersection line of any positioning image to the first intersection line of any positioning image. The second intersection line of any positioning image is the intersection line of any positioning image with the candidate plane. The candidate plane with the smallest target distance among the multiple candidate planes is determined as the scanning plane.

3. The method according to claim 2, characterized in that, The candidate plane is determined based on position parameters, and the position parameters of any two candidate planes are different. The position parameters include: spatial point, polar angle, and orientation angle. Wherein, the spatial point is a pixel point in the intersection line of any two positioning images in the space where the target part is located.

4. The method according to claim 3, characterized in that, The process of determining the scanning plane at least once based on the distance information of the multiple positioning images includes: Based on the distance information of the multiple positioning images, the scanning plane determination process is performed multiple times; In the sequentially adjacent first scan plane determination process and second scan plane determination process, multiple candidate planes in the second scan plane determination process are determined based on the first scan plane, and the first scan plane is the scan plane determined by the first scan plane determination process; Wherein, at least one of the position parameters used to determine the second candidate plane satisfies: The parameter value is the value between the target parameter value and the adjacent parameter value in the first parameter set, and the tolerance of the parameter values ​​in the second parameter value set is less than the tolerance of the parameter values ​​in the first parameter value set; The target parameter value is the parameter value of the first scanning plane. The first parameter value set is the parameter value set used to determine the first candidate plane. The second parameter value set is the parameter value set used to determine the second candidate plane. The parameter values ​​in the first parameter value set and the second parameter value set are arranged in an arithmetic sequence. The first candidate plane is the candidate plane obtained in the process of determining the first scanning plane. The second candidate plane is the candidate plane obtained in the process of determining the second scanning plane.

5. The method according to claim 2, characterized in that, The distance information of any location image is reflected by the heat map of any location image. The heat map has heat values ​​that correspond one-to-one with the pixels of any location image. Each heat value is used to reflect the distance from the corresponding pixel to the first intersection line of any location image. The reference distance of any positioning image is the sum of the heat values ​​of each pixel of the second intersection line of the positioning image on the heat map.

6. The method according to claim 1, characterized in that, Before inputting the target localization image into the target convolutional neural network to obtain the distance information of the target localization image predicted by the target convolutional neural network, the method further includes: Multiple training samples are acquired, including: historical distance information of historical positioning images, the historical distance information being determined based on the distance from each pixel in the historical positioning image to the third intersection line, the third intersection line being the intersection line between the historical positioning image and the historical scanning plane, the historical positioning image and the target positioning image having the same scanning angle, and the historical scanning plane and the target scanning plane having the same scanning angle; The target convolutional neural network is trained based on the multiple training samples.

7. A scanning plane determination device, characterized in that, The device includes: An acquisition module is used to acquire multiple positioning images, which are determined based on multiple scan images. The multiple scan images are obtained by scanning a target object from different angles, and the target object includes target parts. The prediction module is used to input the target positioning image into the target convolutional neural network to obtain the distance information of the target positioning image predicted by the target convolutional neural network. The target convolutional neural network is used to process positioning images of the type to which the target positioning image belongs. The target positioning image is any one of the plurality of positioning images. The distance information of any one positioning image is used to reflect the distance of each pixel in the any one positioning image to the first intersection line of the any one positioning image. The first intersection line of the any one positioning image is the intersection line of the any one positioning image with the target scanning plane of the target part to be determined. The determination module is used to determine the target scanning plane based on the distance information of the multiple positioning images.

8. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the scan plane determination method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the scan plane determination method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes computer instructions that are executed by a processor to implement the scan plane determination method as described in any one of claims 1 to 6.

Images