Osteotomy length measuring device

Abstract

Disclosed is a bone cutting length measuring device which adopts an image-based bone cutting length measuring method to measure the length of a bone cut to ensure the accuracy of the size adaptation of a specially designed tumor-type artificial joint prosthesis to the length of the bone cut of a patient. That is, based on the proportion between the decoding values obtained by the encoder and decoder with the same network structure for representing the length characteristics of the bone cutting object and the reference object, the size ratio between the bone cutting size to be measured and the reference object is taken as the size ratio between the bone cutting size to be measured and the reference object, so as to construct a bone cutting size measuring scheme based on images. In this way, the accuracy of the length size measurement of the bone cutting to be measured is improved.

Description

Technical Field

[0001] This application relates to the field of smart healthcare, and more specifically to a device for measuring osteotomy length. Background Technology

[0002] With advancements in imaging, chemotherapy, and surgery, limb-sparing surgery has gradually become the mainstream trend in the surgical treatment of malignant bone tumors. Long bone tumor resection is performed, and a specially designed tumor-type artificial joint prosthesis is used for replacement to restore the patient's motor function. Accurate measurement of the length of the tumor bone resection during surgery is a crucial step. Due to the limitations of the surgical field, surrounding blood vessels, nerves, and ligaments, it is not possible to completely separate and measure the tumor bone. Even if it is completely separated, the length of the irregular three-dimensional structure of the tumor bone is often measured with a ruler during surgery, and temporary tools are often needed to assist in the measurement, making errors unavoidable.

[0003] Therefore, an optimized osteotomy length measurement device is needed. Summary of the Invention

[0004] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide an osteotomy length measurement device that employs an image-based cropping length measurement method to measure the length of the osteotomy, ensuring the accuracy of the fit between the specially designed tumor-type artificial joint prosthesis and the patient's osteotomy length dimensions. Specifically, after the osteotomy object and a reference object are encoded using encoders and decoders with the same network structure to obtain decoded values ​​representing length features, the ratio between these decoded values ​​is used as the size ratio between the osteotomy dimension to be measured and the reference object, thereby constructing an osteotomy size measurement scheme based on the image. This improves the accuracy of measuring the length of the osteotomy dimension to be measured.

[0005] According to one aspect of this application, an osteotomy length measuring device is provided, comprising:

[0006] The imaging unit is used to acquire detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length;

[0007] An image denoising unit is used to perform denoising processing on the detection image to obtain a denoised detection image;

[0008] The first target detection unit is used to pass the denoised detection image through the osteotomy object detection network to obtain the first region of interest;

[0009] The second target detection unit is used to pass the denoised detection image through a reference image detection network to obtain a second region of interest.

[0010] A dual encoding unit is used to pass the first region of interest and the second region of interest through a dual encoding network model containing a first image encoder and a second image encoder to obtain the osteotomy object feature vector and the reference object feature vector;

[0011] A feature correction unit is used to perform feature distribution correction on the feature vector of the osteotomy object and the feature vector of the reference object to obtain the corrected feature vector of the osteotomy object and the corrected feature vector of the reference object.

[0012] A dual decoding unit is used to pass the corrected osteotomy object feature vector and the corrected reference object feature vector through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value; and

[0013] The length measurement result generation unit is used to generate the length dimension of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length dimension of the reference object.

[0014] In the aforementioned osteotomy length measurement device, the image denoising unit is further used to input the detection image into a denoiser based on an automatic codec to obtain the denoised detection image.

[0015] In the aforementioned osteotomy length measurement device, the image denoising unit includes: an image feature extraction subunit, used to input the detected image into the encoder of the autoencoder-based denoising unit, wherein the encoder uses a convolutional layer to explicitly spatially encode the detected image to obtain detected image features; and an image feature decoding subunit, used to input the detected image features into the decoder of the autoencoder-based denoising unit, wherein the decoder uses a deconvolutional layer to deconvolve the detected image features to obtain the denoised detected image.

[0016] In the above-mentioned osteotomy length measurement device, the osteotomy object detection network and the reference image detection network are anchor window-based target detection networks, and the anchor window-based target detection network is Fast R-CNN, Faster R-CNN or RetinaNet.

[0017] In the above-mentioned osteotomy length measurement device, the first image encoder and the second image encoder have the same network structure.

[0018] In the aforementioned osteotomy length measurement device, the first image encoder and the second image encoder are deep convolutional neural network models containing multiple hybrid convolutional layers.

[0019] In the aforementioned osteotomy length measurement device, the feature correction unit includes: a first correction factor generation subunit, which calculates the multi-distribution decoding regression continuity factor of the osteotomy object feature vector as the first factor using the following formula; wherein, the formula is:

[0020]

[0021] Where V1 is the feature vector of the osteotomy object, V r It is a reference vector based on the feature vector of the osteotomy object, decodev(·) represents the decoded value obtained by the decoder of the feature vector, and log represents the logarithmic function value with base 2; the second correction factor generation subunit calculates the multi-distribution decoding regression continuity factor of the feature vector of the reference object as the second factor using the following formula; wherein, the formula is:

[0022]

[0023] Where V2 is the feature vector of the reference object, V r It is a reference vector based on the feature vector of the reference object, decodev(·) represents the decoded value obtained by the decoder of the feature vector, log represents the logarithmic function value with base 2; and a correction factor action subunit, used to perform weighted correction on the feature vector of the osteotomy object and the feature vector of the reference object with the first factor and the second factor respectively to obtain the corrected feature vector of the osteotomy object and the corrected feature vector of the reference object.

[0024] In the aforementioned osteotomy length measurement device, the dual decoding unit is further configured to: use multiple fully connected layers of the first decoder to perform decoding regression on the feature vector of the corrected osteotomy object using the following formula to obtain the first decoded value, wherein the formula is: Where X1 is the feature vector of the corrected osteotomy object, Y1 is the first decoded value, W1 is the weight matrix, and B1 is the bias vector. Let h(·) represent matrix multiplication, and h(·) be the activation function; and, the corrected reference object feature vector is decoded and regressed using multiple fully connected layers of the second decoder according to the following formula to obtain the second decoded value, wherein the formula is: Where X2 is the corrected reference object feature vector, Y2 is the second decoded value, W2 is the weight matrix, and B2 is the bias vector. denoted as matrix multiplication, and h(·) is the activation function.

[0025] In the above-mentioned osteotomy length measuring device, the length measurement result generation unit includes: a ratio calculation subunit, used to calculate the ratio between the first decoded value and the second decoded value; and a size calculation subunit, used to multiply the ratio by the known length dimension of the reference object to obtain the length dimension of the osteotomy to be measured.

[0026] According to another aspect of this application, a method for measuring osteotomy length is also provided, comprising:

[0027] Acquire detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length;

[0028] The detected image is denoised to obtain a denoised detected image;

[0029] The denoised detection image is passed through an osteotomy object detection network to obtain a first region of interest.

[0030] The denoised detection image is passed through a reference image detection network to obtain a second region of interest.

[0031] The first region of interest and the second region of interest are passed through a dual-encoding network model containing a first image encoder and a second image encoder to obtain the feature vector of the osteotomy object and the feature vector of the reference object.

[0032] The feature distribution of the osteotomy object feature vector and the reference object feature vector is corrected to obtain the corrected osteotomy object feature vector and the corrected reference object feature vector.

[0033] The corrected osteotomy object feature vector and the corrected reference object feature vector are passed through a dual-decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value; and

[0034] The length of the osteotomy to be measured is generated based on the ratio between the first decoded value and the second decoded value and the known length of the reference object.

[0035] In the above osteotomy length measurement method, the step of denoising the detection image to obtain a denoised detection image includes: inputting the detection image into a denoiser based on an automatic codec to obtain the denoised detection image.

[0036] In the above-described osteotomy length measurement method, the step of denoising the detection image to obtain a denoised detection image includes: inputting the detection image into the encoder of the autoencoder-based denoiser, wherein the encoder uses a convolutional layer to explicitly spatially encode the detection image to obtain detection image features; and inputting the detection image features into the decoder of the autoencoder-based denoiser, wherein the decoder uses a deconvolutional layer to deconvolve the detection image features to obtain the denoised detection image.

[0037] In the above osteotomy length measurement method, the osteotomy object detection network and the reference image detection network are anchor window-based target detection networks, and the anchor window-based target detection network is Fast R-CNN, Faster R-CNN or RetinaNet.

[0038] In the above-described osteotomy length measurement method, generating the length of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length of the reference object includes: calculating the ratio between the first decoded value and the second decoded value; and multiplying the ratio by the known length of the reference object to obtain the length of the osteotomy to be measured.

[0039] Compared with existing technologies, the osteotomy length measurement device provided in this application employs an image-based cropping length measurement method to measure the length of the osteotomy, ensuring the accuracy of the fit between the specially designed tumor-type artificial joint prosthesis and the patient's osteotomy length dimensions. Specifically, after the osteotomy object and a reference object are encoded using encoders and decoders with the same network structure to obtain decoded values ​​representing length features, the ratio between these decoded values ​​is used as the size ratio between the osteotomy dimension to be measured and the reference object, thereby constructing an image-based osteotomy size measurement scheme. This improves the accuracy of measuring the length of the osteotomy dimension to be measured. Attached Figure Description

[0040] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0041] Figure 1 This is an application scenario diagram of the osteotomy length measuring device according to an embodiment of this application.

[0042] Figure 2 This is a block diagram of an osteotomy length measuring device according to an embodiment of this application.

[0043] Figure 3 This is a schematic diagram of the architecture of an osteotomy length measuring device according to an embodiment of this application.

[0044] Figure 4 This is a block diagram of the feature correction unit in the osteotomy length measuring device according to an embodiment of this application.

[0045] Figure 5 This is a flowchart of a method for measuring osteotomy length according to an embodiment of this application. Detailed Implementation

[0046] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0047] Application Overview

[0048] As mentioned above, with advancements in imaging, chemotherapy, and surgery, limb-sparing surgery has gradually become the mainstream trend in the surgical treatment of malignant bone tumors. For long bone tumor resection, the use of specially designed tumor-type artificial joint prostheses for replacement aims to restore the patient's motor function. Accurate intraoperative measurement of the tumor bone resection length is a crucial step. However, due to limitations imposed by the surgical field, surrounding blood vessels, nerves, and ligaments, complete separation and measurement of the tumor bone is not always possible. Even when complete separation is achieved, measuring the length of the irregular three-dimensional structure of the tumor bone using a ruler during surgery often requires the assistance of temporary tools, inevitably leading to errors. Therefore, an optimized osteotomy length measurement device is desired.

[0049] Currently, deep learning and neural networks have been widely applied in fields such as computer vision, natural language processing, and speech signal processing. Furthermore, deep learning and neural networks have demonstrated near-human or even surpassed human-level performance in areas such as image classification, object detection, semantic segmentation, and text translation.

[0050] In recent years, the development of deep learning and neural networks has provided new ideas and solutions for the intelligent measurement of osteotomy length.

[0051] Correspondingly, accurately measuring the length of the osteotomy in tumor-sparing surgery is a crucial step in limb-sparing surgery. However, due to limitations imposed by the surgical field, surrounding blood vessels, nerves, and ligaments, complete separation and measurement of the tumor bone is not always possible. With the continuous maturation of deep learning-based artificial intelligence technology, the technical solution of this application aims to employ an image-based cropping length measurement method to ensure the accurate matching of the osteotomy length dimensions between the specially designed tumor-type artificial joint prosthesis and the patient. However, since the length dimensions of the osteotomy object in the image are difficult to obtain directly, a reference object is needed to obtain a scale. Therefore, in the technical solution of this application, after the osteotomy object and the reference object are processed by an encoder and decoder with the same network structure to obtain decoded values ​​representing length features, the ratio between these decoded values ​​is used as the size ratio between the osteotomy dimension to be measured and the reference object. This allows for the construction of an image-based osteotomy dimension measurement scheme.

[0052] Specifically, in the technical solution of this application, firstly, detection images of the osteotomy to be measured and a reference object are acquired. Here, in order to obtain the actual length of the osteotomy to be measured in the detection image, the reference object has a known length. Accordingly, in a specific example, the reference object is a cuboid with a predetermined length.

[0053] Next, considering that various interfering features such as blood vessels, nerves, and ligaments around the osteotomy site exist within the surgical field, which are irrelevant to the determination of osteotomy length, this will affect the detection of osteotomy length in the detection image. Therefore, further noise reduction processing is needed to obtain a denoised detection image. Specifically, a noise reduction processor based on an automatic codec is used to denoise the detection image to obtain the denoised detection image.

[0054] Then, for the denoised detection image, it contains two feature information related to the bone resection length detection: the bone resection object and the reference object, as well as other useless interference feature information. Therefore, in order to accurately measure the length of the bone resection to be measured, a region of interest (ROI) is further applied to the denoised detection image to obtain the ROI of the bone resection object and the ROI of the reference object. Specifically, the denoised detection image is passed through a bone resection object detection network to obtain a first ROI, and the denoised detection image is passed through a reference image detection network to obtain a second ROI. Accordingly, in a specific example, the bone resection object detection network and the reference image detection network are anchor window-based target detection networks, such as Fast R-CNN, Faster R-CNN, or RetinaNet.

[0055] Furthermore, the first and second regions of interest (ROIs) are image encoded using a dual-encoding network model comprising a first image encoder and a second image encoder to obtain feature vectors for the osteotomy object and a reference object. Specifically, the first image encoder of the dual-encoding network model is used to extract features from the first ROI to extract the feature distribution information focused on the length dimension of the osteotomy object, thereby obtaining the osteotomy object feature vector. Similarly, the second image encoder of the dual-encoding network model is used to extract features from the second ROI to extract the feature distribution information focused on the length dimension of the reference object, thereby obtaining the reference object feature vector. In particular, the first and second image encoders have the same network structure. Using image encoders with the same network structure to image encode the two ROIs allows for the extraction of the length dimension feature differences between the osteotomy object and the reference object in the source image, which helps improve the accuracy of the size ratio between the two objects.

[0056] Specifically, in the technical solution of this application, the first image encoder and the second image encoder are deep convolutional neural network models containing multiple hybrid convolutional layers. That is, the first image encoder and the second image encoder use a deep convolutional neural network model containing multiple hybrid convolutional layers to encode the first region of interest and the second region of interest respectively, so as to extract multi-scale hidden feature distribution information regarding the measured osteotomy size and the reference object size in the first region of interest and the second region of interest respectively, thereby obtaining the osteotomy object feature vector and the reference object feature vector. This expands the receptive field while avoiding downsampling loss information, thereby improving the sufficiency of extracting the features of the measured osteotomy size and the reference object size, and helping to improve the accuracy of the size ratio between the two.

[0057] Then, the corrected osteotomy object feature vector and the corrected reference object feature vector are passed through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value. Next, based on the ratio between the first decoded value and the second decoded value and the known length of the reference object, the length of the osteotomy to be measured is generated. Specifically, after calculating the ratio between the first decoded value and the second decoded value to obtain the size ratio between the osteotomy to be measured and the reference object size, the ratio is further multiplied by the known length of the reference object to obtain the length of the osteotomy to be measured. Thus, after the osteotomy object and the reference object obtain decoded values ​​representing length features through an encoder and decoder with the same network structure, the ratio between these decoded values ​​is used as the size ratio between the osteotomy to be measured and the reference object size. This ratio is then multiplied by the known length of the reference object to measure the osteotomy length, thereby improving the accuracy of detecting the length of the osteotomy to be measured.

[0058] Specifically, in the technical solution of this application, in order to improve the accuracy of the ratio between the first decoded value and the second decoded value, it is desirable to improve the correlation between the first decoded value and the second decoded value obtained by the decoder from the osteotomy object feature vector and the reference object feature vector. That is, it is desirable to improve the regression correlation of the osteotomy object feature vector and the reference object feature vector as separate feature distributions in the predetermined decoding regression task. Therefore, it is desirable that the osteotomy object feature vector and the reference object feature vector, as separate feature distributions, have high distribution continuity in the predetermined decoding regression task.

[0059] This can be achieved by setting weights as hyperparameters for the feature vectors of the osteotomy object and the reference object, respectively, to improve their regression correlation in the decoding regression task. However, since these weights need to be obtained during model training, this increases the training burden on the model. Therefore, in the technical solution of this application, the multi-distribution decoding regression continuity factor of the feature vectors of the osteotomy object and the reference object is calculated separately, and expressed as:

[0060]

[0061]

[0062] Wherein, V1 is the feature vector of the osteotomy object, V2 is the feature vector of the reference object, and V rIt is a reference vector based on the feature vector of the osteotomy object and the feature vector of the reference object. For example, it can be set as the mean vector of the feature vector of the osteotomy object and the feature vector of the reference object. decodev(·) represents the decoded value obtained by the decoder of the feature vector.

[0063] Here, to avoid difficulties in converging towards the target regression domain due to excessive fragmentation of the decision boundaries corresponding to the individual feature distributions of the osteotomy object feature vector and the reference object feature vector under a predetermined decoding regression task, the regression continuity factor of each individual feature distribution relative to the global feature distribution is predicted by calculating the regression value offset distribution modulus information of the decoding regression of each individual feature distribution relative to the global average feature distribution. Thus, by using this information as weight to weight the osteotomy object feature vector and the reference object feature vector, the optimization of hyperparameters during training can be transformed from backpropagation into a regression problem based on the decoding regression of multiple distributions, and the accuracy of calculating the ratio between the first decoded value and the second decoded value is improved. This enhances the accuracy of measuring the length dimension of the osteotomy to be measured.

[0064] Based on this, this application proposes an osteotomy length measurement device, comprising: an imaging unit for acquiring detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length; an image denoising unit for denoising the detection images to obtain a denoised detection image; a first target detection unit for passing the denoised detection image through an osteotomy object detection network to obtain a first region of interest; a second target detection unit for passing the denoised detection image through a reference image detection network to obtain a second region of interest; and a dual encoding unit for passing the first region of interest and the second region of interest through a dual encoding network including a first image encoder and a second image encoder. The system includes a network model to obtain feature vectors of the osteotomy object and a reference object; a feature correction unit to perform feature distribution correction on the feature vectors of the osteotomy object and the reference object to obtain corrected feature vectors of the osteotomy object and the reference object; a dual decoding unit to pass the corrected feature vectors of the osteotomy object and the reference object through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value; and a length measurement result generation unit to generate the length dimension of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length dimension of the reference object.

[0065] Figure 1 This is an application scenario diagram of the osteotomy length measuring device according to an embodiment of this application. For example... Figure 1As shown, in this application scenario, the data is first acquired from the camera (e.g., such as...). Figure 1 The osteotomy to be measured (e.g., as shown in C) Figure 1 The O shown in the diagram and the reference object (e.g., such as O) ... Figure 1 The detected image of R (as shown in the figure), wherein the reference object has a known length. Then, the detected images of the osteotomy to be measured and the reference object are input to a server deployed with an osteotomy length measurement algorithm (e.g., such as...). Figure 1 In the S shown, the server is able to process the input detection images of the osteotomy to be measured and the reference object with the osteotomy length measurement algorithm to obtain the length dimension of the osteotomy to be measured.

[0066] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0067] Exemplary System

[0068] Figure 2 This is a block diagram of an osteotomy length measuring device according to an embodiment of this application. Figure 2 As shown, the osteotomy length measuring device 100 according to an embodiment of this application includes: an imaging unit 110, used to acquire detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length; an image denoising unit 120, used to denoise the detection images to obtain a denoised detection image; a first target detection unit 130, used to pass the denoised detection image through an osteotomy object detection network to obtain a first region of interest; a second target detection unit 140, used to pass the denoised detection image through a reference image detection network to obtain a second region of interest; and a dual encoding unit 150, used to encode the first region of interest and the second region of interest through a dual encoding unit including a first image encoder and a second image encoder. A recoding network model is used to obtain feature vectors of the osteotomy object and the reference object; a feature correction unit 160 is used to perform feature distribution correction on the feature vectors of the osteotomy object and the reference object to obtain corrected feature vectors of the osteotomy object and the reference object; a dual decoding unit 170 is used to pass the corrected feature vectors of the osteotomy object and the corrected feature vectors of the reference object through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value; and a length measurement result generation unit 180 is used to generate the length dimension of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length dimension of the reference object.

[0069] Figure 3 This is a schematic diagram of the architecture of an osteotomy length measuring device according to an embodiment of this application. Figure 3 As shown in the schematic diagram, in this architecture, detection images of the osteotomy to be measured and a reference object are first acquired, with the reference object having a known length. Next, the detection images are denoised to obtain denoised detection images. Then, the denoised detection images are passed through an osteotomy object detection network to obtain a first region of interest (ROI), and the denoised detection images are passed through a reference image detection network to obtain a second ROI. Furthermore, the first and second ROIs are passed through a dual-encoding network model containing a first image encoder and a second image encoder to obtain osteotomy object feature vectors and reference object feature vectors. Next, feature distribution correction is performed on the osteotomy object feature vectors and the reference object feature vectors to obtain corrected osteotomy object feature vectors and corrected reference object feature vectors. Then, the corrected osteotomy object feature vectors and the corrected reference object feature vectors are passed through a dual-decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value. Finally, based on the ratio between the first decoded value and the second decoded value and the known length of the reference object, the length of the osteotomy to be measured is generated.

[0070] In the aforementioned osteotomy length measurement device 100, the imaging unit 110 is used to acquire detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length. As mentioned above, with the advancements in imaging, chemotherapy, and surgery, limb-sparing surgery has gradually become the mainstream trend in the surgical treatment of malignant bone tumors. For long bone tumor resection, a specially designed tumor-type artificial joint prosthesis is used for replacement to restore the patient's motor function. Accurate measurement of the tumor bone osteotomy length during surgery is a crucial step. Due to limitations imposed by the surgical field, surrounding blood vessels, nerves, and ligaments, complete separation and measurement of the tumor bone is not always possible. Even when complete separation is achieved, measuring the length of the irregular three-dimensional structure of the tumor bone using a ruler during surgery often requires the assistance of temporary tools, inevitably leading to errors. Therefore, an optimized osteotomy length measurement device is desired.

[0071] Currently, deep learning and neural networks are widely used in fields such as computer vision, natural language processing, and speech signal processing. Furthermore, deep learning and neural networks have demonstrated near-human or even surpassed human-level performance in areas such as image classification, object detection, semantic segmentation, and text translation. In recent years, the development of deep learning and neural networks has provided new solutions and approaches for the intelligent measurement of osteotomy length.

[0072] Correspondingly, accurately measuring the length of the osteotomy in a tumor-sparing surgery is a crucial step in limb-sparing surgery. However, due to limitations imposed by the surgical field, surrounding blood vessels, nerves, and ligaments, complete separation and measurement of the tumor bone is not always possible. With the continuous maturation of deep learning-based artificial intelligence technology, this application aims to employ an image-based cropping length measurement method to ensure the accurate matching of the specially designed tumor-type artificial joint prosthesis with the patient's osteotomy length. However, since the length of the osteotomy object in an image is difficult to obtain directly, a reference object is needed to obtain a scale. Therefore, in this application's technical solution, after the osteotomy object and the reference object are processed by an encoder and decoder with the same network structure to obtain decoded values ​​representing length features, the ratio between these decoded values ​​is used as the size ratio between the osteotomy size to be measured and the reference object. This allows for the construction of an image-based osteotomy size measurement scheme. Specifically, in this application's technical solution, firstly, detection images of the osteotomy to be measured and the reference object are acquired. Here, in order to obtain the actual length of the osteotomy in the detection image, the reference object has a known length. Accordingly, in a specific example, the reference object is a cuboid with a predetermined length. Furthermore, a camera deployed on the osteotomy length measuring device can be used to acquire detection images of the osteotomy to be measured and the reference object.

[0073] In the aforementioned osteotomy length measurement device 100, the image denoising unit 120 is used to denoise the detection image to obtain a denoised detection image. Considering that various interference features unrelated to osteotomy length determination, such as blood vessels, nerves, and ligaments, exist around the osteotomy site within the surgical field, this can affect the osteotomy length detection in the detection image. Therefore, further denoising processing of the detection image is required to obtain a denoised detection image. Specifically, an automatic codec-based denoiser is used to denoise the detection image to obtain the denoised detection image.

[0074] Specifically, in this embodiment of the application, the image denoising unit 120 is further configured to input the detection image into a denoiser based on an automatic codec to obtain the denoised detection image.

[0075] More specifically, the image denoising unit 120 is further configured to: input the detected image into the encoder of the autoencoder-based denoiser through the image feature extraction subunit, wherein the encoder uses a convolutional layer to explicitly spatially encode the detected image to obtain detected image features; and input the detected image features into the decoder of the autoencoder-based denoiser through the image feature decoding subunit, wherein the decoder uses a deconvolutional layer to deconvolve the detected image features to obtain the denoised detected image.

[0076] In the aforementioned osteotomy length measurement device 100, the first target detection unit 130 is used to pass the denoised detection image through an osteotomy object detection network to obtain a first region of interest. The denoised detection image contains two feature information related to osteotomy length detection: the osteotomy object and a reference object, as well as other useless interference feature information. Therefore, in order to accurately measure the length of the osteotomy to be measured, a region of interest is further applied to the denoised detection image to obtain the region of interest of the osteotomy object. Specifically, the denoised detection image is passed through an osteotomy object detection network to obtain the first region of interest.

[0077] In the osteotomy length measuring device 100 described above, the second target detection unit 140 is used to pass the denoised detection image through a reference image detection network to obtain a second region of interest. Similarly, in order to improve the accuracy of measuring the length of the osteotomy to be measured, a region of interest is applied to the denoised detection image to obtain the region of interest of the reference object. That is, specifically, the denoised detection image is passed through a reference image detection network to obtain a second region of interest.

[0078] Accordingly, in a specific example, the osteotomy object detection network and the reference image detection network are anchor window-based object detection networks, such as Fast R-CNN, Faster R-CNN, or RetinaNet.

[0079] In the osteotomy length measurement device 100 described above, the dual encoding unit 150 is used to pass the first region of interest and the second region of interest through a dual encoding network model containing a first image encoder and a second image encoder to obtain osteotomy object feature vectors and reference object feature vectors. After obtaining the first region of interest and the second region of interest, they are image encoded through a dual encoding network model containing a first image encoder and a second image encoder to obtain osteotomy object feature vectors and reference object feature vectors. That is, the first image encoder of the dual encoding network model is used to extract features from the first region of interest to extract feature distribution information focused on the length dimension of the osteotomy object in the first region of interest, thereby obtaining the osteotomy object feature vector; and the second image encoder of the dual encoding network model is used to extract features from the second region of interest to extract feature distribution information focused on the length dimension of the reference object in the second region of interest, thereby obtaining the reference object feature vector.

[0080] Specifically, here, the first image encoder and the second image encoder have the same network structure. Using an image encoder with the same network structure to encode images of two regions of interest can extract the length and size differences of the osteotomy object and the reference object in the source domain image, which is beneficial to improving the accuracy of the size ratio between the two.

[0081] Specifically, in this embodiment, the first image encoder and the second image encoder are deep convolutional neural network models containing multiple hybrid convolutional layers. That is, the first image encoder and the second image encoder use a deep convolutional neural network model containing multiple hybrid convolutional layers to encode the first region of interest and the second region of interest, respectively, to extract multi-scale hidden feature distribution information of the first region of interest and the second region of interest regarding the size of the osteotomy to be measured and the size of the reference object, thereby obtaining the osteotomy object feature vector and the reference object feature vector.

[0082] Those skilled in the art will know that convolutional neural networks (CNNs) use convolutional kernels as feature filtering factors for feature extraction. While these kernels give CNNs powerful feature extraction capabilities, they also limit the scale and receptive field of feature extraction due to the parameters of the kernels. Therefore, in the technical solution of this application, the layers of the CNN model are replaced with hybrid convolutional layers instead of traditional layer structures. Specifically, the hybrid convolutional layer (MCL) is designed with four parallel branches, consisting of one ordinary convolutional layer with a kernel size of 3×3 and three dilated convolutional layers with a kernel size of 3×3. These layers operate on the input feature map, and the dilation rates of the three dilated convolutional branches are set to 2, 3, and 4, respectively. By setting different dilation rates, image information with different receptive fields can be obtained, resulting in feature maps of different scales. This expands the receptive field while avoiding downsampling loss of information. Then, the feature maps of the four branches are fused, resulting in denser sampling. This approach achieves high-level features without adding additional parameters. This improves the sufficiency of extracting the size features of the osteotomy to be measured and the size features of the reference object, which helps to improve the accuracy of the size ratio between the two.

[0083] More specifically, in the deep convolutional neural network model containing multiple hybrid convolutional layers, each hybrid convolutional layer processes the input data during the forward propagation of the layer as follows: convolution processing is performed on the input data based on a first convolutional kernel to obtain a first-scale feature map; convolution processing is performed on the input data based on a second convolutional kernel to obtain a second-scale feature map, wherein the second convolutional kernel is a dilated convolutional kernel with a first dilation rate; convolution processing is performed on the input data based on a third convolutional kernel to obtain a third-scale feature map, wherein the third convolutional kernel is a dilated convolutional kernel with a second dilation rate; and convolution processing is performed on the input data based on a fourth convolutional kernel. The input data is convolved to obtain a fourth-scale feature map, wherein the fourth convolution kernel is a dilated convolution kernel with a third dilation rate; the first-scale feature map, the second-scale feature map, the third-scale feature map, and the fourth-scale feature map are concatenated to obtain a multi-scale feature map; the multi-scale feature map is pooled to obtain a pooled feature map; and the pooled feature map is activated to obtain an activated feature map; wherein the output of the last hybrid convolutional layer of the deep convolutional neural network model is the feature vector of the osteotomy object or the feature vector of the reference object.

[0084] In the osteotomy length measurement device 100 described above, the feature correction unit 160 is used to perform feature distribution correction on the feature vector of the osteotomy object and the feature vector of the reference object to obtain corrected feature vectors of the osteotomy object and the reference object. Here, in order to improve the accuracy of the ratio between the first decoded value and the second decoded value in subsequent operations, it is desirable to improve the correlation between the first decoded value and the second decoded value obtained by the decoder from the feature vector of the osteotomy object and the feature vector of the reference object. That is, it is desirable to improve the regression correlation of the feature vector of the osteotomy object and the feature vector of the reference object as separate feature distributions in the predetermined decoding regression task. Therefore, it is desirable that the feature vector of the osteotomy object and the feature vector of the reference object, as separate feature distributions, have high distribution continuity in the predetermined decoding regression task.

[0085] This can be achieved by setting weights as hyperparameters for the feature vectors of the osteotomy object and the reference object, respectively, to improve their regression correlation in the decoding regression task. However, since these weights need to be obtained during model training, this increases the training burden on the model. Therefore, in the technical solution of this application, the multi-distribution decoding regression continuity factor of the feature vectors of the osteotomy object and the reference object is calculated separately, and expressed as:

[0086]

[0087]

[0088] Wherein, V1 is the feature vector of the osteotomy object, V2 is the feature vector of the reference object, and V r It is a reference vector based on the feature vector of the osteotomy object and the feature vector of the reference object. For example, it can be set as the mean vector of the feature vector of the osteotomy object and the feature vector of the reference object. decodev(·) represents the decoded value obtained by the decoder of the feature vector.

[0089] Here, to avoid difficulties in converging towards the target regression domain due to excessive fragmentation of the decision boundaries corresponding to the individual feature distributions of the osteotomy object feature vector and the reference object feature vector under a predetermined decoding regression task, the regression continuity factor of each individual feature distribution relative to the global feature distribution is predicted by calculating the regression value offset distribution modulus information of the decoding regression of each individual feature distribution relative to the global average feature distribution. Thus, by using this information as weight to weight the osteotomy object feature vector and the reference object feature vector, the optimization of hyperparameters during training can be transformed from backpropagation into a regression problem based on the decoding regression of multiple distributions, and the accuracy of calculating the ratio between the first decoded value and the second decoded value is improved. This enhances the accuracy of measuring the length dimension of the osteotomy to be measured.

[0090] Figure 4 This is a block diagram of the feature correction unit in the osteotomy length measuring device according to an embodiment of this application. Figure 4 As shown, the feature correction unit 160 includes: a first correction factor generation subunit 161, which calculates the multi-distribution decoding regression continuity factor of the osteotomy object feature vector as the first factor using the following formula; wherein, the formula is:

[0091]

[0092] Where V1 is the feature vector of the osteotomy object, V r It is a reference vector based on the feature vector of the osteotomy object, decodev(·) represents the decoded value obtained by the decoder of the feature vector, and log represents the logarithmic function value with base 2; the second correction factor generation subunit 162 calculates the multi-distribution decoding regression continuity factor of the feature vector of the reference object as the second factor using the following formula; wherein, the formula is:

[0093]

[0094] Where V2 is the feature vector of the reference object, V r It is a reference vector based on the feature vector of the reference object, decodev(·) represents the decoded value obtained by the decoder of the feature vector, and log represents the logarithmic function value with base 2; and, the correction factor action subunit 163 is used to perform weighted correction on the feature vector of the osteotomy object and the feature vector of the reference object with the first factor and the second factor respectively to obtain the corrected feature vector of the osteotomy object and the corrected feature vector of the reference object.

[0095] In the osteotomy length measuring device 100 described above, the dual decoding unit 170 is used to pass the corrected osteotomy object feature vector and the corrected reference object feature vector through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value.

[0096] Specifically, in this embodiment, the dual decoding unit 170 is further configured to: use multiple fully connected layers of the first decoder to perform decoding regression on the feature vector of the corrected osteotomy object using the following formula to obtain the first decoded value, wherein the formula is: Where X1 is the feature vector of the corrected osteotomy object, Y1 is the first decoded value, W1 is the weight matrix, and B1 is the bias vector. Let h(·) represent matrix multiplication, and h(·) be the activation function; and, the corrected reference object feature vector is decoded and regressed using multiple fully connected layers of the second decoder according to the following formula to obtain the second decoded value, wherein the formula is: Where X2 is the corrected reference object feature vector, Y2 is the second decoded value, W2 is the weight matrix, and B2 is the bias vector. denoted as matrix multiplication, and h(·) is the activation function.

[0097] In the osteotomy length measuring device 100 described above, the length measurement result generation unit 180 is used to generate the length dimension of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length dimension of the reference object. Specifically, after calculating the ratio between the first decoded value and the second decoded value to obtain the size ratio between the osteotomy dimension to be measured and the size of the reference object, the ratio is further multiplied by the known length dimension of the reference object to obtain the length dimension of the osteotomy to be measured.

[0098] Specifically, in this embodiment, the length measurement result generation unit 180 includes: a ratio calculation subunit for calculating the ratio between the first decoded value and the second decoded value; and a size calculation subunit for multiplying the ratio by the known length of the reference object to obtain the length of the osteotomy to be measured. Thus, based on the decoded values ​​representing length features obtained from the osteotomy object and the reference object via an encoder and decoder with the same network structure, the ratio between these decoded values ​​is used as the size ratio between the osteotomy size to be measured and the size of the reference object. This ratio is then multiplied by the known length of the reference object to measure the osteotomy length, thereby improving the accuracy of detecting the length of the osteotomy to be measured.

[0099] In summary, the osteotomy length measuring device 100 according to the embodiments of this application is explained, which uses an image-based screenshot length measurement method to measure the length of the osteotomy, ensuring the accuracy of the fit between the specially designed tumor-type artificial joint prosthesis and the patient's osteotomy length size. That is, after the osteotomy object and a reference object are processed by an encoder and decoder with the same network structure to obtain decoded values ​​representing length features, the ratio between the decoded values ​​is used as the size ratio between the osteotomy size to be measured and the reference object, thereby constructing an osteotomy size measurement scheme based on the image. This improves the accuracy of measuring the length of the osteotomy to be measured.

[0100] Exemplary methods

[0101] Figure 5 This is a flowchart of a method for measuring osteotomy length according to an embodiment of this application. Figure 5 As shown, the osteotomy length measurement method according to an embodiment of this application includes the following steps: S110, acquiring detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length; S120, performing noise reduction processing on the detection images to obtain a noise-reduced detection image; S130, passing the noise-reduced detection image through an osteotomy object detection network to obtain a first region of interest; S140, passing the noise-reduced detection image through a reference image detection network to obtain a second region of interest; S150, passing the first region of interest and the second region of interest through a dual encoding network including a first image encoder and a second image encoder. The model obtains the feature vector of the osteotomy object and the feature vector of the reference object; S160, the feature distribution of the osteotomy object feature vector and the reference object feature vector is corrected to obtain the corrected osteotomy object feature vector and the corrected reference object feature vector; S170, the corrected osteotomy object feature vector and the corrected reference object feature vector are passed through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value; and S180, the length dimension of the osteotomy to be measured is generated based on the ratio between the first decoded value and the second decoded value and the known length dimension of the reference object.

[0102] In one example, in the above osteotomy length measurement method, the step of denoising the detection image to obtain a denoised detection image includes: inputting the detection image into a denoiser based on an automatic codec to obtain the denoised detection image.

[0103] In one example, in the above osteotomy length measurement method, the step of denoising the detection image to obtain a denoised detection image includes: inputting the detection image into the encoder of the autoencoder-based denoiser, wherein the encoder uses a convolutional layer to explicitly spatially encode the detection image to obtain detection image features; and inputting the detection image features into the decoder of the autoencoder-based denoiser, wherein the decoder uses a deconvolutional layer to deconvolve the detection image features to obtain the denoised detection image.

[0104] In one example, in the osteotomy length measurement method described above, the osteotomy object detection network and the reference image detection network are anchor window-based target detection networks, and the anchor window-based target detection network is Fast R-CNN, Faster R-CNN, or RetinaNet.

[0105] In one example, in the osteotomy length measurement method described above, the first image encoder and the second image encoder have the same network structure.

[0106] In one example, in the osteotomy length measurement method described above, the first image encoder and the second image encoder are deep convolutional neural network models containing multiple hybrid convolutional layers.

[0107] In one example, in the above osteotomy length measurement method, the step of performing feature distribution correction on the feature vector of the osteotomy object and the feature vector of the reference object to obtain the corrected feature vector of the osteotomy object and the corrected feature vector of the reference object includes: calculating the multi-distribution decoding regression continuity factor of the feature vector of the osteotomy object as the first factor using the following formula; wherein, the formula is:

[0108]

[0109] Where V1 is the feature vector of the osteotomy object, V r It is a reference vector based on the feature vector of the osteotomy object, where decodev(·) represents the decoded value obtained by the decoder, and log represents the logarithmic function value to the base 2. The multi-distribution decoding regression continuity factor of the feature vector of the reference object is calculated as the second factor using the following formula: [Formula omitted for brevity]

[0110]

[0111] Where V2 is the feature vector of the reference object, V rIt is a reference vector based on the feature vector of the reference object, decodev(·) represents the decoded value obtained by the decoder of the feature vector, and log represents the logarithmic function value with base 2; and, the osteotomy object feature vector and the reference object feature vector are weighted and corrected by the first factor and the second factor respectively to obtain the corrected osteotomy object feature vector and the corrected reference object feature vector.

[0112] In one example, in the above osteotomy length measurement method, the step of passing the corrected osteotomy object feature vector and the corrected reference object feature vector through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value includes: using multiple fully connected layers of the first decoder to perform decoding regression on the corrected osteotomy object feature vector using the following formula to obtain the first decoded value, wherein the formula is: Where X1 is the feature vector of the corrected osteotomy object, Y1 is the first decoded value, W1 is the weight matrix, and B1 is the bias vector. Let h(·) represent matrix multiplication, and h(·) be the activation function; and, the corrected reference object feature vector is decoded and regressed using multiple fully connected layers of the second decoder according to the following formula to obtain the second decoded value, wherein the formula is: Where X2 is the corrected reference object feature vector, Y2 is the second decoded value, W2 is the weight matrix, and B2 is the bias vector. denoted as matrix multiplication, and h(·) is the activation function.

[0113] In one example, in the above osteotomy length measurement method, generating the length of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length of the reference object includes: calculating the ratio between the first decoded value and the second decoded value; and multiplying the ratio by the known length of the reference object to obtain the length of the osteotomy to be measured.

[0114] In summary, the osteotomy length measurement method according to the embodiments of this application is explained. It employs an image-based screenshot length measurement method to ensure the accuracy of the fit between the specially designed tumor-type artificial joint prosthesis and the patient's osteotomy length. Specifically, after the osteotomy object and a reference object are encoded using encoders and decoders with the same network structure to obtain decoded values ​​representing length features, the ratio between these decoded values ​​is used as the size ratio between the osteotomy size to be measured and the reference object. This allows for the construction of an image-based osteotomy size measurement scheme. This improves the accuracy of measuring the length of the osteotomy to be measured.

Claims

1. A device for measuring osteotomy length, characterized in that, include: An imaging unit is used to acquire detection images of the osteotomy to be measured and a reference object, wherein the reference object has a known length; An image denoising unit is used to perform denoising processing on the detection image to obtain a denoised detection image; The first target detection unit is used to pass the denoised detection image through the osteotomy object detection network to obtain the first region of interest; The second target detection unit is used to pass the denoised detection image through a reference image detection network to obtain a second region of interest. A dual encoding unit is used to pass the first region of interest and the second region of interest through a dual encoding network model containing a first image encoder and a second image encoder to obtain the osteotomy object feature vector and the reference object feature vector; A feature correction unit is used to perform feature distribution correction on the feature vector of the osteotomy object and the feature vector of the reference object to obtain the corrected feature vector of the osteotomy object and the corrected feature vector of the reference object. A dual decoding unit is used to pass the corrected osteotomy object feature vector and the corrected reference object feature vector through a dual decoding network model containing a first decoder and a second decoder to obtain a first decoded value and a second decoded value. as well as The length measurement result generation unit is used to generate the length dimension of the osteotomy to be measured based on the ratio between the first decoded value and the second decoded value and the known length dimension of the reference object; The feature correction unit includes: The first correction factor generation subunit calculates the multi-distribution decoding regression continuity factor of the osteotomy object feature vector as the first factor using the following formula; The formula is as follows: in It is the feature vector of the osteotomy object. It is a reference vector based on the feature vector of the osteotomy object. This represents the decoded value obtained by the decoder from the feature vector. This represents the value of a logarithmic function with base 2. The second correction factor generation subunit calculates the multi-distribution decoding regression continuity factor of the reference object feature vector as the second factor using the following formula; The formula is as follows: in It is a reference vector based on the feature vector of the reference object. This represents the decoded value obtained by the decoder from the feature vector. Represent the value of the logarithmic function base 2; and The correction factor action subunit is used to perform weighted correction on the osteotomy object feature vector and the reference object feature vector using the first factor and the second factor, respectively, to obtain the corrected osteotomy object feature vector and the corrected reference object feature vector.

2. The osteotomy length measuring device according to claim 1, characterized in that, The image denoising unit is further configured to input the detected image into an autocoder-based denoiser to obtain the denoised detected image.

3. The osteotomy length measuring device according to claim 2, characterized in that, The image noise reduction unit includes: An image feature extraction subunit is used to input the detected image into the encoder of the autoencoder-based denoising unit, wherein the encoder uses a convolutional layer to explicitly spatially encode the detected image to obtain detected image features; and An image feature decoding subunit is used to input the detected image features into the decoder in the autoencoder-based denoising unit, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the detected image features to obtain the denoised detected image.

4. The osteotomy length measuring device according to claim 3, characterized in that, The osteotomy object detection network and the reference image detection network are anchor window-based target detection networks, which are FastR-CNN, Faster R-CNN, or RetinaNet.

5. The osteotomy length measuring device according to claim 4, characterized in that, The first image encoder and the second image encoder have the same network structure.

6. The osteotomy length measuring device according to claim 5, characterized in that, The first image encoder and the second image encoder are deep convolutional neural network models containing multiple hybrid convolutional layers.

7. The osteotomy length measuring device according to claim 6, characterized in that, The dual decoding unit is further used for: The first decoded value is obtained by decoding and regressing the feature vector of the corrected osteotomy object using multiple fully connected layers of the first decoder according to the following formula: ,in It is the feature vector of the corrected osteotomy object. It is the first decoded value. It is a weight matrix. It is a bias vector. Represents matrix multiplication. For activation functions; and The corrected reference object feature vector is decoded and regressed using multiple fully connected layers of the second decoder according to the following formula to obtain the second decoded value, wherein the formula is: ,in The corrected reference object feature vector The second decoded value, , , Represents matrix multiplication. This is the activation function.

8. The osteotomy length measuring device according to claim 7, characterized in that, The length measurement result generation unit includes: A ratio calculation subunit is used to calculate the ratio between the first decoded value and the second decoded value; and The size calculation subunit is used to multiply the ratio by the known length dimension of the reference object to obtain the length dimension of the osteotomy to be measured.

Images