Bounding box prediction method and data transmission method and device for three-dimensional image data

Abstract

The embodiment of the present application is applicable to the fields of computer-assisted medical treatment and image processing technology, and provides a method for predicting a bounding box of three-dimensional image data and a data transmission method and device. The prediction method includes: when receiving three-dimensional image data, projecting the three-dimensional image data into a two-dimensional image; determining the feature vector of the two-dimensional image; determining the task label of the three-dimensional image data based on the feature vector of the two-dimensional image; using the prediction model corresponding to the task label to predict the bounding box of the target area of ​​the three-dimensional image data, the input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data, and any type of task label has been pre-trained with an independent prediction model. Using the above method, the target area in the three-dimensional image data can be accurately located and the bounding box of the target area can be output, thereby improving the efficiency and accuracy of bounding box prediction.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of computer-assisted medical treatment and image processing technology, and in particular, relate to a bounding box prediction method and data transmission method and device for three-dimensional image data. Background Art

[0002] Medical image processing technology is developing rapidly. For example, three-dimensional imaging data such as computed tomography (CT) and magnetic resonance imaging (MRI) has been widely used in the diagnosis and treatment of various diseases and in surgical scenarios. However, whether a doctor observes the data to complete a diagnosis or uses image processing algorithms to extract targets from the data, the three-dimensional image data must first be transmitted. Due to the large volume and large amount of three-dimensional image data in the medical field, data transmission speed is slow, which reduces the real-time performance of data processing. This problem is particularly prominent when multi-center transmission relies on multiple image processing algorithms. Summary of the Invention

[0003] In view of this, the embodiments of the present application provide a bounding box prediction method, device and computer program product for three-dimensional image data, which are used to improve the efficiency and accuracy of target area bounding box prediction, and then can extract the data contained in the bounding box for transmission, which can significantly reduce the amount of data to be transmitted and improve the data transmission speed.

[0004] A first aspect of an embodiment of the present application provides a method for predicting a bounding box of three-dimensional image data, comprising:

[0005] When receiving the three-dimensional image data, projecting the three-dimensional image data into a two-dimensional image;

[0006] Determine a feature vector of the two-dimensional image, the feature vector comprising a first vector along a first direction axis and a second vector along a second direction axis;

[0007] determining a task label for the three-dimensional image data according to a feature vector of the two-dimensional image;

[0008] The prediction model corresponding to the task label is used to predict the bounding box of the target area of ​​the three-dimensional image data. The input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data. An independent prediction model is pre-trained for any type of task label.

[0009] Optionally, determining the feature vector of the two-dimensional image includes:

[0010] Accumulating pixel values ​​of each data point in the two-dimensional image along a first direction axis and a second direction axis of the two-dimensional image respectively to obtain pixel accumulation values ​​of each data point along the first direction axis and along the second direction axis;

[0011] Linear interpolation sampling is performed using the pixel accumulation value of each data point to obtain a first vector along the first direction axis and a second vector along the second direction axis.

[0012] Optionally, before determining the feature vector of the two-dimensional image, the method further includes:

[0013] Determining data fingerprint information of the three-dimensional image data, the data fingerprint information including a first position value and a second position value;

[0014] The first position value is used as the lower threshold value and the second position value is used as the upper threshold value, and data clamping is performed on the two-dimensional image so that the pixel value of each data point of the clamped two-dimensional image is within the interval formed by the lower threshold value and the upper threshold value.

[0015] Optionally, the step of clamping the two-dimensional image data by using the first position value as a lower threshold and the second position value as an upper threshold includes:

[0016] Determining a pixel value of each data point in the two-dimensional image;

[0017] The pixel values ​​of data points smaller than the lower threshold are adjusted to be equal to the lower threshold, and the pixel values ​​of data points larger than the upper threshold are adjusted to be equal to the upper threshold.

[0018] Optionally, projecting the three-dimensional image data into a two-dimensional image includes:

[0019] Determining the maximum value of each data point in the three-dimensional image data in the direction of the third axis;

[0020] According to the maximum value of each data point in the third axial direction, the three-dimensional image data is projected along the coronal plane to obtain a two-dimensional image.

[0021] Optionally, determining the task label of the three-dimensional image data according to the feature vector of the two-dimensional image includes:

[0022] The trained classification model is used to process the feature vector of the two-dimensional image and output the corresponding task label of the three-dimensional image data. The classification model is obtained by training a support vector machine model using a feature vector data set marked with the task label. The feature vector data set includes multiple groups of first vectors along the first direction axis of the two-dimensional sample image and second vectors along the second direction axis. Any of the two-dimensional sample images is obtained by projecting the three-dimensional sample image data.

[0023] Optionally, predicting a bounding box of a target area of ​​the three-dimensional image data using a prediction model corresponding to the task label includes:

[0024] The prediction model corresponding to the task label is used to process the size information of the three-dimensional image data, and the coordinate values ​​of the bounding box of the target area are output, wherein the coordinate values ​​include the starting coordinate values ​​and the ending coordinate values ​​of the bounding box on the first direction axis, the second direction axis and the third direction axis.

[0025] Optionally, before predicting the bounding box of the target area of ​​the three-dimensional image data using the prediction model corresponding to the task label, the method further includes:

[0026] Construct linear models;

[0027] The linear model is trained using three-dimensional sample image data to obtain prediction models corresponding to any type of task label. Any of the three-dimensional sample image data has corresponding size information and task labels. The three-dimensional sample image data corresponding to any type of task label includes one or more bounding boxes.

[0028] A second aspect of the embodiments of the present application provides a method for transmitting three-dimensional image data, including:

[0029] Upon receiving the three-dimensional image data, determining a bounding box of at least one target area in the three-dimensional image data;

[0030] extracting data contained in the bounding box from the three-dimensional image data;

[0031] Transmitting the data contained in the extracted bounding box;

[0032] Wherein, the bounding box of at least one of the target areas is predicted using the method described in any one of the first aspects above.

[0033] A third aspect of an embodiment of the present application provides a bounding box prediction device for three-dimensional image data, comprising:

[0034] a projection module, configured to project the three-dimensional image data into a two-dimensional image upon receiving the three-dimensional image data;

[0035] a determination module, configured to determine a feature vector of the two-dimensional image, the feature vector comprising a first vector along a first direction axis and a second vector along a second direction axis;

[0036] a classification module, configured to determine a task label of the three-dimensional image data based on a feature vector of the two-dimensional image;

[0037] A prediction module is used to predict the bounding box of the target area of ​​the three-dimensional image data using the prediction model corresponding to the task label. The input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data. An independent prediction model is pre-trained for any type of task label.

[0038] A fourth aspect of an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the computer device implements the method described in any one of the first and / or second aspects above.

[0039] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the method described in any one of the first and / or second aspects above is implemented.

[0040] A sixth aspect of the embodiments of the present application provides a computer program product, including a computer program, which, when executed, enables the method described in any one of the first and / or second aspects above to be executed.

[0041] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0042] In an embodiment of the present application, upon receiving three-dimensional image data, a computer device may project the three-dimensional image data into a two-dimensional image. By determining a feature vector for the two-dimensional image, the computer device may determine a task label for the three-dimensional image data based on the feature vector, and use a prediction model corresponding to the task label to predict a bounding box for the target area in the three-dimensional image data, outputting the coordinates of the bounding box. Using the above-described method provided in an embodiment of the present application to predict a bounding box can improve the efficiency and accuracy of bounding box prediction.

[0043] The embodiment of the present application can calculate the bounding box of the target area based on a multi-level simple linear model, without the need to use a computationally intensive deep learning method, and instead adopt a lightweight linear prediction model, which significantly reduces the computational complexity. By introducing a preprocessing step of the projection operation, the amount of data processing can be greatly reduced. In addition, the embodiment of the present application simultaneously utilizes a two-stage strategy of support vector machine (SVM) classification combined with linear regression prediction, which can ensure the accuracy of positioning the target area of ​​interest and further improve the accuracy of the bounding box prediction. The lightweight classification model and prediction model design provided by the embodiment of the present application not only makes the model easy to deploy in various medical devices, but is also particularly suitable for data processing requirements in multi-center collaboration scenarios, providing an efficient and feasible solution for medical image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0045] Figure 1 is a schematic diagram of a bounding box prediction method for three-dimensional image data provided by an embodiment of the present application;

[0046] Figure 2 This is a schematic diagram of the overall process of a bounding box prediction method for three-dimensional image data provided by an embodiment of the present application;

[0047] Figure 3 This is a schematic diagram of a data fingerprint information acquisition process provided by an embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of a classification model training process provided in an embodiment of the present application;

[0049] Figure 5 This is a schematic diagram of a prediction model training process provided by an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of a method for transmitting three-dimensional image data provided in an embodiment of the present application;

[0051] Figure 7 is a schematic diagram of a bounding box prediction device for three-dimensional image data provided by an embodiment of the present application;

[0052] Figure 8 This is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details.

[0054] The technical solution of this application is described below through specific embodiments.

[0055] Reference Figure 1 , which shows a schematic diagram of a bounding box prediction method for three-dimensional image data provided by an embodiment of the present application, which may specifically include the following steps:

[0056] S101 : When three-dimensional image data is received, the three-dimensional image data is projected into a two-dimensional image.

[0057] This method can be applied to a computer device, that is, the execution subject of the embodiments of this application is a computer device. By executing the various steps of this method, the computer device can accurately and efficiently predict the bounding box of the target area from the three-dimensional image data. The above-mentioned computer device can be a device with data processing capabilities, such as a desktop computer, a cloud server, a computer-assisted medical device, etc. The embodiments of this application do not limit the type of computer device.

[0058] The three-dimensional image data may be medical image data, such as CT data, MRI data, etc. The three-dimensional image data may be obtained by scanning the entire body or specific body parts of a patient using corresponding scanning equipment during the preoperative stage or the examination stage.

[0059] In one example, the computer device serving as the execution subject of the embodiments of the present application may be a scanning device. Specifically, the scanning device may be configured with corresponding data processing functions so that, after scanning a patient's body part and obtaining three-dimensional image data, the scanning device processes the data using the configured data processing functions to predict a bounding box of a target area in the three-dimensional image data.

[0060] In another example, the computer device executing the embodiments of the present application can also be a standalone device in communication with the scanning device. That is, after scanning and obtaining 3D image data of the patient, the scanning device can transmit the 3D image data to a standalone computer device for processing. The computer device can then predict a bounding box of the target area from the received 3D image data by employing the steps of the method provided in the embodiments of the present application.

[0061] In an embodiment of the present application, when a computer device receives 3D image data, it can first project the 3D image data into a 2D image. Converting the 3D image data into a 2D image by projection can reduce the amount of data to be processed in subsequent steps and increase data processing efficiency.

[0062] In one possible implementation of an embodiment of the present application, a computer device may project the three-dimensional image data along the coronal plane in a maximum projection mode. Specifically, the computer device may determine the maximum value of each data point in the three-dimensional image data in the direction of the third direction axis, and then project the three-dimensional image data along the coronal plane according to the maximum value of each data point in the direction of the third direction axis to obtain a two-dimensional image. The third direction axis is orthogonal to the first direction axis and the second direction axis, and together constitutes a spatial coordinate system of the three-dimensional image data. The direction indicated by the third direction axis can represent the depth direction of the three-dimensional image data. For example, a single three-dimensional image data can be represented as V, and its size information is represented as H×W×D or H×W×L, where H represents the height of the three-dimensional image data V, W represents the width, and D or L represents the depth. The two-dimensional image obtained by projecting along the coronal plane in the maximum projection mode can be represented as I, and its size information is H×W. That is, compared with the three-dimensional image data V, the two-dimensional image I obtained after projection has less data in the depth direction.

[0063] S102: Determine a feature vector of the two-dimensional image, where the feature vector includes a first vector along a first direction axis and a second vector along a second direction axis.

[0064] In an embodiment of the present application, the feature vector of a two-dimensional image may include vectors in two dimensional directions, namely, a first vector along a first direction axis and a second vector along a second direction axis. The first direction axis may be the x-axis in the image coordinate system of the two-dimensional image, and the second direction axis may be the y-axis in the image coordinate system. Therefore, after the computer device completes the projection to obtain the two-dimensional image, it can determine the first vector and the second vector of the two-dimensional image in the x-axis and y-axis directions of the image coordinate system, respectively.

[0065] In one possible implementation of the embodiment of the present application, a computer device may accumulate pixel values ​​of each data point in a two-dimensional image along a first axis and a second axis of the two-dimensional image, respectively, to obtain accumulated pixel values ​​of each data point along the first axis and along the second axis. Then, linear interpolation sampling is performed using the accumulated pixel values ​​of each data point, respectively, to obtain a first vector along the first axis and a second vector along the second axis.

[0066] Exemplarily, taking the first vector determining the first direction axis, i.e., the x-axis direction, as an example, the computer device can accumulate the pixel values ​​of each data point in the x-axis direction of the two-dimensional image to obtain the pixel cumulative value of each data point in the x-axis direction. The process can be expressed as Ix[i] = sum(I[i,:]), where x represents the x-axis direction, i represents each data point, and Ix[i] = sum(I[i,:]) represents the pixel cumulative value corresponding to each data point i. A pixel cumulative value can be calculated for each data point i. Correspondingly, for determining the second vector in the second direction axis, i.e., the y-axis direction, Iy[j] = sum(I[j,:]) can be used to represent the pixel cumulative value of each data point j in the y-axis direction. A pixel cumulative value can also be calculated for each data point j.

[0067] Then, the computer device can obtain two vectors in the x-axis direction and the y-axis direction through linear interpolation sampling, namely the first vector and the second vector.

[0068] In one possible implementation of the embodiment of the present application, after calculating the pixel cumulative value of each data point, in order to reduce the data size, the pixel cumulative value can be divided by a factor γ, and the value obtained by dividing by the factor γ is interpolated and sampled. The factor γ can be any integer value that is not a multiple of 10.

[0069] Therefore, the above formula can be further expressed as: Ix[i] = sum(I[i,:]) / γ, Iy[j] = sum(I[j,:]) / γ. The computer device can perform linear interpolation sampling on Ix and Iy respectively to obtain the first vector I'x and the second vector I'y.

[0070] In an embodiment of the present application, when performing linear interpolation sampling, the computer device may sample Ix and Iy into vectors of a certain dimension, for example, 128 dimensions, that is, the first vector I'x and the second vector I'y obtained by sampling may be 128-dimensional feature vectors.

[0071] In one possible implementation of an embodiment of the present application, in order to improve the accuracy of data processing and reduce the impact of noise and artifacts in the data on subsequent bounding box prediction, after projecting a two-dimensional image, the computer device can clamp the data to clamp each data point in the two-dimensional image within a certain range, thereby suppressing the maximum and minimum values ​​in the two-dimensional image and removing noise and artifacts in the data.

[0072] In an embodiment of the present application, the data fingerprint of the three-dimensional image data can be used to clamp the range of each data point in the two-dimensional image. Specifically, the computer device can determine the data fingerprint information of the three-dimensional image data, where the data fingerprint information can include a first position value and a second position value. The computer device then clamps the data of the two-dimensional image using the first position value as a lower threshold and the second position value as an upper threshold, so that the pixel value of each data point in the clamped two-dimensional image falls within the interval formed by the lower threshold and the upper threshold.

[0073] The above-mentioned data fingerprint can be data that can represent specific characteristics of three-dimensional image data calculated according to certain standards. For example, the data fingerprint of the three-dimensional image data can include the mean (mean), standard deviation (std), maximum value (max_value), minimum value (min_value), etc. of each data point in the foreground data part of the current three-dimensional image data. If the data fingerprint of the three-dimensional image data is represented by fp, then fp={max_value, min_value, std, mean} can be obtained. Among them, the maximum value (max_value) can refer to the pixel value of the data point in the top 99.5% position after sorting the data points in the foreground data by pixel value, and the minimum value (min_value) can be the pixel value of the data point in the top 0.5% position of each data point after the above sorting.

[0074] In a possible implementation of an embodiment of the present application, when using the data fingerprint fp to clamp the two-dimensional image, the first position value may be the smaller value (min_value) in the above data fingerprint fp, and the second position value may be the larger value (max_value).

[0075] When the first position value, i.e., the smaller value (min_value), is used as the lower threshold value and the second position value, i.e., the larger value (max_value), is used as the upper threshold value, when performing data processing on a two-dimensional image, the pixel value of each data point in the two-dimensional image can be determined, so that the pixel value of the data point less than the lower threshold value is adjusted to be equal to the lower threshold value, and the pixel value of the data point greater than the upper threshold value is adjusted to be equal to the upper threshold value, thereby limiting each data point in the two-dimensional image to the range of [min_value, max_value].

[0076] Determining the feature vector of the two-dimensional image may be performed after completing the above-mentioned data clamping step.

[0077] S103 : Determine a task label of the three-dimensional image data according to the feature vector of the two-dimensional image.

[0078] In embodiments of the present application, a task label can be used to represent a target region in the 3D image data, and the bounding box prediction is the same as predicting the bounding box of the target region. For example, if the task label represents the stomach, then the bounding box of the stomach region needs to be predicted from the 3D image data after processing. The computer device can determine the task label of the 3D image data based on the feature vector of the 2D image.

[0079] In a possible implementation of the embodiment of the present application, a trained classification model may be used to process the feature vector of a two-dimensional image and output a task label of the corresponding three-dimensional image data.

[0080] Specifically, a classification model can be pre-trained and deployed in a computer. After the computer completes the aforementioned steps and obtains a feature vector for the two-dimensional image, the classification model can be invoked to process the feature vector and output a corresponding task label. In other words, the feature vector of the two-dimensional image can serve as input data for the classification model, which then processes it to output a task label.

[0081] In an embodiment of the present application, the classification model may be obtained by training an SVM model using a feature vector dataset labeled with task labels. The feature vector dataset, serving as training data for the classification model, may include multiple sets of first vectors along a first axis and second vectors along a second axis of a two-dimensional sample image, and any of the two-dimensional sample images may be projected from three-dimensional sample image data.

[0082] Specifically, multiple three-dimensional image data can be collected as sample data to form three-dimensional sample image data. The computer device can project the three-dimensional sample image data according to the steps described above to obtain multiple two-dimensional sample images. For each two-dimensional sample image, the computer device determines the first vector of the image along the first direction axis x-axis and the second vector along the second direction axis y-axis to form multiple vector groups. Each vector group includes a set of feature vectors of the two-dimensional sample image, that is, the first vector I'x and the second vector I'y determined in the above steps. Each three-dimensional sample image data can be pre-labeled with a corresponding task label, for example, the task label can be the stomach area, the two-hand area, the left hand area, or the right hand area. The computer device can input multiple vector groups into the SVM model for model training, output the corresponding task labels, and compare them with the actual task labels of the three-dimensional sample image data corresponding to each labeled vector group. Through multiple iterations, a classification model that can be used subsequently is obtained.

[0083] In this way, after the training of the classification model is completed, the classification model can be configured in a computer device to process the feature vector of the two-dimensional image determined each time, and output a task label for bounding box prediction.

[0084] S104. Use the prediction model corresponding to the task label to predict the bounding box of the target area of ​​the three-dimensional image data. The input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data. Any type of task label has an independent prediction model pre-trained.

[0085] In an embodiment of the present application, in conjunction with the task labels determined in the aforementioned steps, the computer device can call a pre-trained prediction model to perform predictions and output a bounding box for the target area. Each type of task label can be pre-trained with an independent prediction model. Thus, by training an independent prediction model for each task label, after determining the task label for the three-dimensional image data, the computer device can directly call the prediction model corresponding to the task label for processing, efficiently and accurately completing the bounding box prediction for the target area.

[0086] In one possible implementation of the embodiment of the present application, the prediction model may be a linear model. By constructing the linear model, the computer device may train the linear model using the three-dimensional sample image data to obtain prediction models corresponding to various types of task labels.

[0087] In an embodiment of the present application, the constructed linear model can be expressed as y=wx+k, where w is a model parameter and k is a bias. After constructing the linear model of the above form, the computer device can use three-dimensional sample image data for model training. Among them, any three-dimensional sample image data has corresponding size information and task labels, and the three-dimensional sample image data corresponding to each type of task label can include one bounding box or multiple bounding boxes. For example, for the three-dimensional image data of the hand areas collected, the left and right hand areas belong to different bounding boxes respectively.

[0088] In a possible implementation of an embodiment of the present application, when training the above-mentioned linear model of the form y=wx+k, x=[H,W,D] represents the size information of the three-dimensional sample image data, and y=[x1, x2, y1, y2, z1,z2]*n represents the coordinates of n bounding boxes in the three-dimensional sample image data. Using multiple three-dimensional sample image data, the model parameters w and the bias k values ​​of the prediction model corresponding to the corresponding task label can be optimized using algorithms such as the least squares method. After calculating the values ​​of the model parameters w and the bias k, the prediction model y=wx+k can be determined. By configuring the prediction model in a computer device, after receiving the size information and task label of the three-dimensional image data, the prediction model corresponding to the task label can be used to process the task label and size information, and output the coordinate values ​​of the bounding box of the target area. The coordinate values ​​may include the starting coordinate values ​​and the ending coordinate values ​​of the bounding box on the first direction axis, i.e., the x-axis, the second direction axis, i.e., the y-axis, and the third direction axis, i.e., the z-axis. For example, [x1, x2] can be used to represent the starting and ending coordinate values ​​of one of the bounding boxes on the x-axis, [y1, y2] can be used to represent the starting and ending coordinate values ​​of the bounding box on the y-axis, and [z1, z2] can be used to represent the starting and ending coordinate values ​​of the bounding box on the z-axis. When the three-dimensional image data includes multiple target areas, the corresponding output bounding boxes are also multiple, and the coordinate values ​​of each bounding box include the starting and ending coordinate values ​​of the bounding box on the x-axis, y-axis, and z-axis in the form of [x1, x2, y1, y2, z1, z2]. The number of bounding boxes output by the prediction model is equal to the number of target areas in the three-dimensional image data.

[0089] In one possible implementation of the present invention, after a computer device predicts a bounding box and outputs the coordinate values ​​of the corresponding bounding box in the three-dimensional image data, it can extract the data within the bounding box from the three-dimensional image data according to the coordinate values. In this way, the computer device can transmit the extracted data to other devices for use, such as equipment used by doctors to diagnose and treat diseases based on the three-dimensional image data. Using the steps provided in this method to predict the bounding box and extract data for transmission can avoid transmitting all of the three-dimensional image data, and instead transmit only data from a specific region, thereby increasing the speed of data transmission.

[0090] In an embodiment of the present application, upon receiving three-dimensional image data, a computer device may project the three-dimensional image data into a two-dimensional image. By determining a feature vector for the two-dimensional image, the computer device may determine a task label for the three-dimensional image data based on the feature vector, and use a prediction model corresponding to the task label to predict a bounding box for the target area of ​​the three-dimensional image data, outputting the coordinates of the bounding box. Using the above-described method provided in an embodiment of the present application to predict a bounding box can improve the efficiency and accuracy of bounding box prediction.

[0091] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0092] To facilitate understanding, the bounding box prediction method provided in an embodiment of the present application is introduced below with reference to a complete example.

[0093] like Figure 2 2 is a schematic diagram of the overall process of the bounding box prediction method for three-dimensional image data provided by an embodiment of the present application. Figure 2 The three-dimensional data V in the above embodiments is the three-dimensional image data V, which can be CT data or MRI data. Any three-dimensional data V has corresponding size information, namely Figure 2 As shown in w×h×l (H×W×D or H×W×L in the above embodiments). For three-dimensional data V, it can be converted into a two-dimensional image I through data projection, and its size can be expressed as w×h (That is, H×W).

[0094] Figure 2 The clamp shown in the figure is a numerical clamping function. For a two-dimensional image I, the computer device can use data clamping, combined with the data fingerprint fp, to constrain each value in image I to a range with a lower limit fp0 and an upper limit fp1. The lower limit fp0 can be the first position value (min_value) in the aforementioned embodiments, and the upper limit fp1 can be the second position value (max_value) in the aforementioned embodiments.

[0095] like Figure 3 The figure is a schematic diagram of a data fingerprint information acquisition process provided by an embodiment of the present application. Figure 3As shown, the 3D data set may include multiple 3D image data, such as 3D image data V1, ... 3D image data Vn, etc. For each 3D image data, the corresponding foreground data G can be obtained by image segmentation, and the mean, standard deviation, smaller value (min_value) which can be used as the lower limit fp0, and larger value (max_value) which can be used as the upper limit fp1 in the foreground data G can be calculated. Figure 3 Middle Q 0.005 (G) can represent the pixel value of the data point in the top 0.5% position after each data point in the foreground data G is sorted by pixel value, Q 0.995 (G) represents the pixel value of the data point in the top 99.5% position after each data point in the foreground data G is sorted by pixel value.

[0096] according to Figure 3 After the data fingerprint of the three-dimensional image data is calculated by the process shown, the data fingerprint can be used to limit the pixel value of each data point in the two-dimensional image I to the range of [fp0=min_value, fp1=max_value].

[0097] Return to see Figure 2 After completing the data clamping of the two-dimensional image I, the computer device can respectively determine the feature vectors along the x-axis and y-axis of the image, and sample to obtain the first vector I'x and the second vector I'y.

[0098] like Figure 4 The figure shows a schematic diagram of a classification model training process provided in an embodiment of the present application. Figure 4 In the training process of the classification model shown, the computer device processes the three-dimensional sample image data and samples the corresponding two-dimensional image to obtain the first vector I'x along the x-axis and the second vector I'y along the y-axis. Figure 2 The process shown is similar. The multiple vector groups determined from the multiple three-dimensional sample image data can be used as training data for training the classification model, thereby training the classification model, which can be Figure 4 The SVM model shown in .

[0099] On the other hand, in addition to the classification model, a prediction model needs to be trained. Figure 5 FIG2 is a schematic diagram of a prediction model training process provided by an embodiment of the present application. The prediction model can be obtained based on linear model training. An independent linear prediction model can be trained for each task label and used for subsequent bounding box prediction. The trained classification model and prediction model can be configured in a computer device.

[0100] Return to see Figure 2 , for the three-dimensional data V to be processed, the computer equipment follows Figure 2 After the corresponding first vector I'x and second vector I'y are determined by the process shown in FIG. Figure 4 The SVM model trained by the process shown processes the feature vector and outputs the task label task_label of the three-dimensional data, that is, Figure 2 When the SVM model is called to process the feature vector, the SVM input data is the first vector I'x along the x-axis and the second vector I'y along the y-axis of the 2D image projected from the 3D data V to be processed. The SVM output data is the task label task_label for the corresponding 3D data V.

[0101] The computer device can then call Figure 5 The prediction model obtained by training the process shown in the figure performs bounding box prediction. The input data of the prediction model is the size information of the three-dimensional data V. w×h×l And the corresponding task label task_label, the output data of the prediction model is the coordinate value of the bounding box of at least one target area in the three-dimensional data V.

[0102] In this way, at least one target area can be determined efficiently and accurately from the three-dimensional data, and the position of the bounding box of the target area can be determined by the coordinate values ​​output by the prediction model.

[0103] In combination with the bounding box prediction method described in the above embodiments, the present application also provides a method for transmitting three-dimensional image data. Figure 6 , is a schematic diagram of a method for transmitting three-dimensional image data provided in an embodiment of the present application, which may specifically include the following steps:

[0104] S601: When three-dimensional image data is received, determine a bounding box of at least one target area in the three-dimensional image data.

[0105] S602: Extract data contained in the bounding box from the three-dimensional image data.

[0106] S603: Transmit the data contained in the extracted bounding box.

[0107] The bounding box of at least one target area can be predicted using the methods described in the aforementioned embodiments of the bounding box prediction method.

[0108] This method can be applied to a computer device. For example, the computer device can be the scanning device in the aforementioned embodiment. That is, after the scanning device obtains the three-dimensional image data through scanning, it can extract the data contained in the bounding box of the target area in the three-dimensional image data according to the steps described in this method and transmit the data. Alternatively, the computer device can also be other devices that communicate with the scanning device, such as computer-assisted medical equipment. After the computer device receives the three-dimensional image data obtained by scanning the scanning device, it can extract the data contained in the bounding box of the target area in the three-dimensional image data according to the steps described in this method and transmit the data. For example, the extracted data can be distributed to multiple other devices that need to use the data. This embodiment of the present application is not limited to this.

[0109] By adopting the transmission method provided in the embodiment of the present application, the steps introduced in the aforementioned bounding box prediction methods can be combined to extract the data contained in the bounding box of the target area from the three-dimensional image data, so that in the scenario where the three-dimensional image data needs to be transmitted, only the data of the target area is transmitted, thereby reducing the amount of data that needs to be transmitted and improving data transmission efficiency.

[0110] Reference Figure 7 , shows a schematic diagram of a bounding box prediction device for three-dimensional image data provided by an embodiment of the present application, which may specifically include a projection module 701, a determination module 702, a classification module 703, and a prediction module 704, wherein:

[0111] The projection module 701 is configured to project the received 3D image data into a 2D image;

[0112] A determination module 702 is configured to determine a feature vector of the two-dimensional image, where the feature vector includes a first vector along a first direction axis and a second vector along a second direction axis;

[0113] A classification module 703 is configured to determine a task label of the three-dimensional image data based on a feature vector of the two-dimensional image;

[0114] The prediction module 704 is used to predict the bounding box of the target area of ​​the three-dimensional image data using the prediction model corresponding to the task label. The input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data. An independent prediction model is pre-trained for any type of task label.

[0115] In the embodiment of the present application, the determining module 702 may be specifically configured to:

[0116] Accumulating pixel values ​​of each data point in the two-dimensional image along a first direction axis and a second direction axis of the two-dimensional image respectively to obtain pixel accumulation values ​​of each data point along the first direction axis and along the second direction axis;

[0117] Linear interpolation sampling is performed using the pixel accumulation value of each data point to obtain a first vector along the first direction axis and a second vector along the second direction axis.

[0118] In a possible implementation of the embodiment of the present application, the determining module 702 may also be configured to:

[0119] Determining data fingerprint information of the three-dimensional image data, the data fingerprint information including a first position value and a second position value;

[0120] The first position value is used as the lower threshold value and the second position value is used as the upper threshold value, and data clamping is performed on the two-dimensional image so that the pixel value of each data point of the clamped two-dimensional image is within the interval formed by the lower threshold value and the upper threshold value.

[0121] In another possible implementation of the embodiment of the present application, the determining module 702 may also be configured to:

[0122] Determining a pixel value of each data point in the two-dimensional image;

[0123] The pixel values ​​of data points smaller than the lower threshold are adjusted to be equal to the lower threshold, and the pixel values ​​of data points larger than the upper threshold are adjusted to be equal to the upper threshold.

[0124] In the embodiment of the present application, the projection module 701 can be specifically used for:

[0125] Determining the maximum value of each data point in the three-dimensional image data in the direction of the third axis;

[0126] According to the maximum value of each data point in the third axial direction, the three-dimensional image data is projected along the coronal plane to obtain a two-dimensional image.

[0127] In the embodiment of the present application, the classification module 703 may be specifically used to:

[0128] The trained classification model is used to process the feature vector of the two-dimensional image and output the corresponding task label of the three-dimensional image data. The classification model is obtained by training a support vector machine model using a feature vector data set marked with the task label. The feature vector data set includes multiple groups of first vectors along the first direction axis of the two-dimensional sample image and second vectors along the second direction axis. Any of the two-dimensional sample images is obtained by projecting the three-dimensional sample image data.

[0129] In the embodiment of the present application, the prediction module 704 may be specifically used to:

[0130] The prediction model corresponding to the task label is used to process the size information of the three-dimensional image data, and the coordinate values ​​of the bounding box of the target area are output, wherein the coordinate values ​​include the starting coordinate values ​​and the ending coordinate values ​​of the bounding box on the first direction axis, the second direction axis and the third direction axis.

[0131] In a possible implementation of the embodiment of the present application, the apparatus may further include a training module, which may be specifically configured to:

[0132] Construct linear models;

[0133] The linear model is trained using three-dimensional sample image data to obtain prediction models corresponding to any type of task label. Any of the three-dimensional sample image data has corresponding size information and task labels. The three-dimensional sample image data corresponding to any type of task label includes one or more bounding boxes.

[0134] An embodiment of the present application provides a bounding box prediction device for three-dimensional image data. By using this device, each step in the aforementioned method embodiments can be implemented.

[0135] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0136] Reference Figure 8 , shows a schematic diagram of a computer device provided by an embodiment of the present application. Figure 8 As shown, the computer device 800 in the embodiment of the present application includes: a processor 810, a memory 820, and a computer program 821 stored in the memory 820 and executable on the processor 810. When the processor 810 executes the computer program 821, the steps of each embodiment of the above-mentioned bounding box prediction method and / or data transmission method for three-dimensional image data are implemented, such as Figure 1 Steps S101 to S104 shown, Figure 6 Alternatively, when the processor 810 executes the computer program 821, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 7 Functions of modules 701 to 704 are shown.

[0137] Exemplarily, the computer program 821 may be divided into one or more modules / units, which are stored in the memory 820 and executed by the processor 810 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments may be used to describe the execution process of the computer program 821 in the computer device 800. For example, the computer program 821 may be divided into a projection module, a determination module, a classification module, and a prediction module, with the specific functions of each module being as follows:

[0138] a projection module, configured to project the three-dimensional image data into a two-dimensional image upon receiving the three-dimensional image data;

[0139] a determination module, configured to determine a feature vector of the two-dimensional image, the feature vector comprising a first vector along a first direction axis and a second vector along a second direction axis;

[0140] a classification module, configured to determine a task label of the three-dimensional image data based on a feature vector of the two-dimensional image;

[0141] A prediction module is used to predict the bounding box of the target area of ​​the three-dimensional image data using the prediction model corresponding to the task label. The input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data. An independent prediction model is pre-trained for any type of task label.

[0142] The computer device 800 may be a device capable of implementing the corresponding steps in the aforementioned method embodiments. The computer device 800 may be a desktop computer, a cloud server, or other devices. For example, the computer device 800 may be a computer-assisted medical device. The computer device 800 may include, but is not limited to, a processor 810 and a memory 820. Those skilled in the art will appreciate that Figure 8 This is merely an example of the computer device 800 and does not constitute a limitation of the computer device 800 . The computer device 800 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 800 may also include input and output devices, network access devices, buses, etc.

[0143] The processor 810 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0144] The memory 820 can be an internal storage unit of the computer device 800, such as a hard drive or memory of the computer device 800. The memory 820 can also be an external storage device of the computer device 800, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 800. Furthermore, the memory 820 can include both the internal storage unit of the computer device 800 and an external storage device. The memory 820 is used to store the computer program 821 and other programs and data required by the computer device 800. The memory 820 can also be used to temporarily store data that has been output or is about to be output.

[0145] An embodiment of the present application further discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the methods described in the aforementioned embodiments are implemented.

[0146] An embodiment of the present application further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the methods described in the aforementioned embodiments are implemented.

[0147] An embodiment of the present application further discloses a computer program product, including a computer program. When the computer program is run on a computer, the computer is caused to execute the methods described in the aforementioned embodiments.

[0148] The above embodiments are intended only to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application and should be included within the scope of protection of the present application.

Claims

1. A method for transmitting three-dimensional image data, characterized in that: Applied to a scanning device, the method includes: When receiving the three-dimensional image data, projecting the three-dimensional image data into a two-dimensional image; Determine a feature vector of the two-dimensional image, the feature vector comprising a first vector along a first direction axis and a second vector along a second direction axis; Processing the feature vector of the two-dimensional image using a trained classification model to output a task label corresponding to the three-dimensional image data, wherein the classification model is obtained by training a support vector machine model using a feature vector dataset marked with the task label; A prediction model corresponding to the task label is used to predict a bounding box of a target area in the three-dimensional image data, and coordinate values ​​of the bounding box of the target area are output. The prediction model is a linear model. Input data of the prediction model when performing bounding box prediction is the task label and size information of the three-dimensional image data. An independent prediction model is pre-trained for each type of task label. The three-dimensional image data includes multiple target areas, and the number of bounding boxes output by the prediction model is equal to the number of target areas in the three-dimensional image data. extracting data contained in a plurality of the bounding boxes from the three-dimensional image data according to the coordinate values; The data contained in the extracted multiple bounding boxes are distributed to multiple devices.

2. The method according to claim 1, characterized in that Determining the feature vector of the two-dimensional image includes: Accumulating pixel values ​​of each data point in the two-dimensional image along a first direction axis and a second direction axis of the two-dimensional image respectively to obtain pixel accumulation values ​​of each data point along the first direction axis and along the second direction axis; Linear interpolation sampling is performed using the pixel accumulation value of each data point to obtain a first vector along the first direction axis and a second vector along the second direction axis.

3. The method according to claim 1, characterized in that Before determining the feature vector of the two-dimensional image, the method further includes: Determining data fingerprint information of the three-dimensional image data, the data fingerprint information including a first position value and a second position value; Determining a pixel value of each data point in the two-dimensional image; With the first position value as the lower threshold and the second position value as the upper threshold, data clamping is performed on the two-dimensional image, and the pixel values ​​of the data points smaller than the lower threshold are adjusted to be equal to the lower threshold, and the pixel values ​​of the data points larger than the upper threshold are adjusted to be equal to the upper threshold, so that the pixel values ​​of each data point of the clamped two-dimensional image are within the interval formed by the lower threshold and the upper threshold.

4. The method according to claim 3, characterized in that Projecting the three-dimensional image data into a two-dimensional image includes: Determining the maximum value of each data point in the three-dimensional image data in the direction of the third axis; According to the maximum value of each data point in the third axial direction, the three-dimensional image data is projected along the coronal plane to obtain a two-dimensional image.

5. The method according to any one of claims 1 to 4, characterized in that The feature vector data set includes multiple sets of first vectors along a first direction axis and second vectors along a second direction axis of a two-dimensional sample image, and any of the two-dimensional sample images is obtained by projecting three-dimensional sample image data.

6. The method according to claim 5, characterized in that The predicting of a bounding box of a target area of ​​the three-dimensional image data by using a prediction model corresponding to the task label includes: The prediction model corresponding to the task label is used to process the size information of the three-dimensional image data, and the coordinate values ​​of the bounding box of the target area are output, wherein the coordinate values ​​include the starting coordinate values ​​and the ending coordinate values ​​of the bounding box on the first direction axis, the second direction axis and the third direction axis.

7. The method according to claim 6, characterized in that Before predicting the bounding box of the target area of ​​the three-dimensional image data using the prediction model corresponding to the task label, the method further includes: Construct linear models; The linear model is trained using three-dimensional sample image data to obtain prediction models corresponding to any type of task label. Any of the three-dimensional sample image data has corresponding size information and task labels. The three-dimensional sample image data corresponding to any type of task label includes one or more bounding boxes.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the computer device is caused to implement the method according to any one of claims 1 to 7.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed, the method according to any one of claims 1 to 7 is performed.

Images