A method and apparatus for calibrating an image

Abstract

The application provides a method and device for calibrating an image, and relates to the technical field of image processing. The method comprises the following steps: obtaining a first diameter of a target object in a first image; adding an image of a reference object to the first image to obtain a second image; converting the second image into a third image; determining a fourth diameter of the target object in the third image according to a third diameter of the reference object in the third image in a polar coordinate system, the first diameter and the second diameter; determining a calibration value according to the third diameter and the fourth diameter; and determining a calibrated image according to the calibration value. The relative position relationship between the target object and the reference object in the polar coordinate system can be determined by using the relative position relationship between the target object and the reference object in the world coordinate system, and the calibration value in the polar coordinate system can be determined according to the relative position relationship between the target object and the reference object in the polar coordinate system, so that the image can be calibrated by using the calibration value, and the problem of low calibration accuracy caused by manual calibration can be avoided.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a method and device for calibrating an image. BACKGROUND

[0002] An image is a main carrier of information, and people can acquire or exchange information by using images. In order to analyze information in an image, people generally process an acquired image to obtain a required result, for example, performing gray processing on an acquired color image, or performing calibration processing on an acquired distorted image.

[0003] For example, in an optical coherence tomography (OCT) system, an image processing device in the system scans and images a biological tissue by using an interference principle of a light source, acquires an image of the biological tissue, and uses the image for auxiliary diagnosis of a disease, for example, an OCT device acquires a cross-sectional image of a blood vessel. However, due to a condition limitation of the image processing device and a limitation of an acquisition environment, an image acquired by using the image processing device has an error, thereby causing a misjudgment of a patient's condition by a calibration personnel. Therefore, it is often necessary to calibrate the image acquired by the image processing device.

[0004] In the related art, a manual calibration manner is generally used to calibrate an image. However, this manner completely depends on manual operation ability of a calibration personnel, is time-consuming and laborious, and has low calibration precision. SUMMARY

[0005] Embodiments of the present application provide a method and device for calibrating an image, which calibrate an image acquired by an OCT device, and avoid a problem of low calibration precision caused by manual calibration.

[0006] To achieve the above object, in a first aspect, embodiments of the present application provide a method for calibrating an image, the method is applied to an image processing device, and the method comprises the following steps.

[0007] acquiring a first diameter of a target object in a first image, the target object being circular or elliptical in shape;

[0008] adding an image of a reference object to the first image to obtain a second image, the second image comprising the target object and the reference object, the reference object being circular in shape, the reference object having a second diameter, and the first image and the second image corresponding to a world coordinate system;

[0009] converting the second image into a third image, the third image corresponding to a polar coordinate system;

[0010] determine a fourth radius of the target object in the third image according to a third radius of the reference object in the third image in a polar coordinate system, the first diameter and the second diameter;

[0011] determine the calibration value according to the third radius and the fourth radius;

[0012] determine the calibrated image according to the calibration value.

[0013] In the above scheme, the target object and the reference object have a relative position relationship in a world coordinate system; after the second image in the world coordinate system is converted into the third image in the polar coordinate system, the target object and the reference object also have a relative position relationship. The relative position relationship of the target object and the reference object in the polar coordinate system can be determined by using the relative position relationship of the target object and the reference object in the world coordinate system, and the calibration value in the polar coordinate system can be determined according to the relative position relationship of the target object and the reference object in the polar coordinate system, so that the image can be calibrated by using the calibration value, wherein the first diameter and the second diameter can represent the relative position relationship of the target object and the reference object in the world coordinate system, and the third radius and the fourth radius can represent the relative position relationship of the target object and the reference object in the polar coordinate system, thereby avoiding the problem of low calibration accuracy caused by manual calibration.

[0014] Optionally, the method further comprises:

[0015] obtaining an original image corresponding to the first image;

[0016] inputting the original image into a contour model to determine the first image, the first image comprising a contour image of the target object;

[0017] determining the first diameter of the target object according to the contour image of the target object.

[0018] Optionally, determining the first diameter of the target object according to the contour image of the target object comprises:

[0019] determining an area of the target object in the first image according to the contour image of the target object;

[0020] determining the first diameter of the target object according to the area of the target object.

[0021] Optionally, the method further comprises:

[0022] obtaining marking information of a first object in a fourth image, the marking information being used to mark the first object in the fourth image, the first object being circular or elliptical;

[0023] inputting the fourth image into an i-th detection model to obtain a predicted object;

[0024] determining a loss value of the first object according to the marked first object and the predicted object in the fourth image.

[0025] Determine whether the i-th detection model is a contour model based on the loss value of the first object;

[0026] Where i is a positive integer.

[0027] In the above scheme, the labeling information of the first object in the fourth image is obtained, and the fourth image is input into the i-th detection model to train the i-th detection model so that the model can predict the first object in the fourth image. The trained detection model is then identified as the contour model. The loss value between the labeled first object and the predicted first object is used as the criterion for whether the contour model has been successfully trained, which can improve the accuracy of the contour model in predicting the first object.

[0028] Optionally, determining the loss value of the first object based on the labeled first object and the predicted object in the fourth image includes:

[0029] The contour loss and diameter loss of the first object are determined based on the first object marked in the fourth image and the predicted object.

[0030] The loss value of the first object is determined based on the contour loss and diameter loss.

[0031] Optionally, determining whether the i-th detection model is a contour model based on the loss value of the first object includes:

[0032] If the loss value of the first object is less than or equal to the preset loss value, the i-th detection model is determined as the contour model.

[0033] Optionally, determining the fourth radius of the target object in the third image based on the third radius, first diameter, and second diameter of the reference object in the third image in polar coordinates includes:

[0034] Determine the ratio of the second diameter to the first diameter;

[0035] The fourth radius is determined based on the ratio of the second diameter to the first diameter and the third radius.

[0036] Optionally, the calibrated image is determined based on the calibration value, including:

[0037] The third image is calibrated based on the calibration values ​​to obtain the calibrated image in polar coordinates.

[0038] Convert the image calibrated in polar coordinates to the image calibrated in world coordinates.

[0039] In the above scheme, the third image is calibrated according to the calibration value. The transformation relationship between polar coordinates and the world coordinate system is used to convert the calibrated third image into a calibrated image in the world coordinate system. That is, under the determined calibration value, the calibrated image is obtained by using the coordinate transformation relationship corresponding to the image, without the need for manual image calibration, thus avoiding the problem of low calibration accuracy caused by manual image calibration.

[0040] Secondly, embodiments of this application provide an image calibration apparatus, the apparatus comprising:

[0041] The acquisition unit is used to acquire the first diameter of the target object in the first image, wherein the shape of the target object is a circle or an ellipse;

[0042] The processing unit is used to add the image of the reference object to the first image to obtain a second image. The second image includes the target object and the reference object. The reference object has a circular shape and a second diameter. The image coordinate system corresponding to the first image and the second image is the world coordinate system.

[0043] The transformation unit is used to convert the second image into a third image, where the image coordinate system corresponding to the third image is a polar coordinate system.

[0044] The determining unit is used to determine the fourth radius of the target object in the third image based on the third radius, first diameter and second diameter of the reference object in the third image in polar coordinates.

[0045] The determining unit is further configured to determine a calibration value based on the third radius and the fourth radius;

[0046] The determining unit is also used to determine the calibrated image based on the calibration value.

[0047] Optionally, the acquisition unit is further configured to acquire the original image corresponding to the first image.

[0048] Optionally, the determining unit is further configured to input the original image into the contour model and determine a first image, wherein the first image includes the contour image of the target object.

[0049] Optionally, the determining unit is further configured to determine a first diameter of the target object based on the contour image of the target object.

[0050] Optionally, the determining unit is specifically used to determine the area of ​​the target object in the first image based on the contour image of the target object;

[0051] The first diameter of the target object is determined based on its area.

[0052] Optionally, the acquisition unit is further configured to acquire the marking information of the first object in the fourth image, the marking information being used to mark the first object in the fourth image, the first object being circular or elliptical.

[0053] Optionally, the processing unit is further configured to input the fourth image into the i-th detection model to obtain the predicted object.

[0054] Optionally, the determining unit is further configured to determine the loss value of the first object based on the marked first object and the predicted object in the fourth image.

[0055] Optionally, the determining unit is further configured to determine whether the i-th detection model is a contour model based on the loss value of the first object.

[0056] Optionally, the determining unit is further configured to determine the contour loss and diameter loss of the first object based on the marked first object and the predicted object in the fourth image;

[0057] The loss value of the first object is determined based on the contour loss and diameter loss.

[0058] Optionally, the determining unit is further configured to determine the i-th detection model as a contour module if the loss value of the first object is less than or equal to a preset loss value, where i is a positive integer.

[0059] Optionally, the determining unit is further configured to determine the ratio of the second diameter to the first diameter;

[0060] The fourth radius is determined based on the ratio of the second diameter to the first diameter and the third radius.

[0061] Optionally, the processing unit is further configured to calibrate the third image according to the calibration value to obtain a calibrated image in polar coordinates.

[0062] Convert the image calibrated in polar coordinates to the image calibrated in world coordinates.

[0063] Thirdly, embodiments of this application provide an image processing apparatus, including a processor coupled to a memory, wherein the processor is used to execute a computer program or instructions stored in the memory to implement the method described in the first aspect or any embodiment of the first aspect.

[0064] The beneficial effects of this application embodiment compared with the prior art are: the relative positional relationship between the target object and the reference object in the world coordinate system can be determined by using the relative positional relationship between the target object and the reference object in the world coordinate system, and the calibration value in the polar coordinate system can be determined based on the relative positional relationship between the target object and the reference object in the polar coordinate system, so that the calibration value can be used to calibrate the image. The first diameter and the second diameter can represent the relative positional relationship between the target object and the reference object in the world coordinate system, and the third radius and the fourth radius can represent the relative positional relationship between the target object and the reference object in the polar coordinate system, avoiding the problem of low calibration accuracy caused by manual calibration. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram illustrating the principle of an OCT system for image acquisition according to an embodiment of this application;

[0067] Figure 2 This application provides an embodiment of an image acquired by an OCT device in an OCT system.

[0068] Figure 3 This is a schematic flowchart of an image calibration method provided in an embodiment of this application;

[0069] Figure 4 This is a diagram showing the relative positional relationship between the imaging conduit and the marker ring in polar coordinates, according to an embodiment of this application.

[0070] Figure 5 This is a diagram showing the relative positional relationship between the imaging conduit and the marker ring in a world coordinate system, provided by an embodiment of this application.

[0071] Figure 6 This application provides an embodiment of images of a third image before and after calibration in polar coordinates;

[0072] Figure 7 This application provides an embodiment of a method for converting an image calibrated in polar coordinates to an image calibrated in world coordinates.

[0073] Figure 8 This is a schematic diagram of the structure of an image calibration device provided in one embodiment of this application;

[0074] Figure 9This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0075] The technical solutions in the embodiments of this application will be described in detail below with reference to the embodiments of this application.

[0076] It should be understood that the methods, situations, categories, and classifications of embodiments in this application are only for the convenience of description and do not constitute any limitation on this application. Various methods, categories, situations, and features in the embodiments can be combined with each other without contradiction.

[0077] It should also be understood that the terms "first," "second," "third," and "fourth" in the embodiments of this application are for distinction only and do not constitute any limitation on this application. It should also be understood that in the various embodiments of this application, the sequence number of each process does not imply the execution order of the steps; the execution order of the steps is determined by their internal logic and does not constitute any limitation on the execution process of the embodiments of this application.

[0078] Images are a primary carrier of information, enabling the acquisition and exchange of information. To facilitate the analysis of information within images, people typically process the acquired images to obtain the desired results. For example, they may perform grayscale processing on acquired color images or calibration processing on acquired distorted images.

[0079] For example, Figure 1 This is a schematic diagram of the structure of an optical coherence tomography (OCT) system for acquiring images of biological tissue, provided in an embodiment of this application. The OCT system includes a catheter connection unit, an imaging catheter, and an image processing device. Figure 1 As shown, the light signal emitted by the image processing device is transmitted to the imaging catheter through the catheter connection unit to acquire images of biological tissues and use them for adjuvant treatment, such as using an OCT device to acquire cross-sectional images of blood vessels.

[0080] However, limitations in image processing equipment and acquisition environment can lead to errors in the images acquired, resulting in misdiagnosis of the patient's condition by calibrators. For example, OCT equipment may acquire images with errors such as... Figure 2 The image shown is a cross-sectional image of a blood vessel, where 'a' is the target object, which can be an imaging catheter, and 'b' is a reference object added artificially after the cross-sectional image is acquired, which can be a marker ring. The location of the marker ring is the reference position of the imaging catheter. When the imaging catheter and the marker ring are aligned, the cross-sectional area of ​​the blood vessel lumen derived from the size of the imaging catheter is accurate. However, when the imaging catheter and the marker ring are not aligned, the cross-sectional area of ​​the blood vessel lumen derived from the size of the imaging catheter is inaccurate. Figure 2If the cross-sectional area of ​​the blood vessel lumen derived from the size of the imaging catheter is smaller than the predetermined value, the calibrator may conclude that the patient has vascular stenosis. However, this could simply be due to imaging errors during image acquisition by the OCT device; in other words, the measurement values ​​of the blood vessel cross-section image obtained using the OCT system are inaccurate. Therefore, it is often necessary to calibrate the images acquired by the OCT device.

[0081] In related technologies, manual calibration is generally used, that is, by manually dragging... Figure 2 The marking ring is placed at the imaging catheter, and the marking ring is made to fit the imaging catheter as closely as possible to complete the calibration. However, this method relies entirely on the operator's manual operation skills, which is time-consuming and labor-intensive, and the accuracy of image calibration is not high under this method.

[0082] Based on the problems in related technologies, this application proposes a method and apparatus for calibrating images. First, a first diameter of a target object in a first image is obtained; the target object is circular or elliptical. An image of a reference object is added to the first image to obtain a second image. The second image includes the target object and the reference object; the reference object is circular, and its diameter is a second diameter. The image coordinate system corresponding to the first and second images is the world coordinate system. The second image is then converted to a third image, and the image coordinate system corresponding to the third image is polar coordinate system. Next, based on the third radius, first diameter, and second diameter of the reference object in the third image in polar coordinate system, a fourth radius of the target object in the third image is determined. A calibration value is determined based on the third and fourth radii. Finally, the calibrated image is determined based on the calibration value. In the world coordinate system, the target object and the reference object have a relative positional relationship; after converting the second image in the world coordinate system to the third image in the polar coordinate system, the target object and the reference object also have a relative positional relationship. The relative positional relationship between the target object and the reference object in the world coordinate system can be used to determine the relative positional relationship between the target object and the reference object in the polar coordinate system. Based on the relative positional relationship between the target object and the reference object in the polar coordinate system, the calibration value in the polar coordinate system can be determined. Thus, the calibration value can be used to calibrate the image. The first diameter and the second diameter can represent the relative positional relationship between the target object and the reference object in the world coordinate system, and the third radius and the fourth radius can represent the relative positional relationship between the target object and the reference object in the polar coordinate system, avoiding the problem of low calibration accuracy caused by manual calibration.

[0083] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0084] Since the implementation of the scheme in this application is based on deep learning networks, and deep learning network models need to be trained and tested before use, the process of obtaining the contour model is described here first:

[0085] Before obtaining the contour model, the model during the training process is called the detection model.

[0086] To better understand the solution in this application, an example of obtaining a contour model is given below:

[0087] Dataset: 4000 cross-sectional images of blood vessels containing a first object, and the labeling information of the first object in the image is known. The labeling information is used to label the first object in the cross-sectional image of the blood vessel. The shape of the first object is circular or elliptical. Among them, 3500 cross-sectional images of blood vessels are used as training images for training the detection model, and 500 cross-sectional images of blood vessels are used as test images for testing the detection model. The first object can be an imaging catheter.

[0088] Optionally, the 4,000 acquired cross-sectional images of blood vessels can be preprocessed, such as rotated, flipped, or have their contrast adjusted, to increase the dataset.

[0089] In the above scheme, preprocessing the acquired images can increase the dataset. This allows the contour model obtained by deep learning using a large number of training images to more accurately identify the first object. At the same time, the contour model can also identify some special images, such as blood residue near the imaging catheter or blood vessel cross-section images with unclear imaging catheter boundaries.

[0090] 1) Training the detection model: The first image from the training images is input into the convolutional neural network structure of the i-th detection model. The convolutional kernel extracts the conduit features. On the feature map of the last layer, the Region Proposal Network (RPN) in the i-th detection model is used. A 3*3 sliding window of the RPN traverses the feature map. At the center of each sliding window (pixel), 9 anchor boxes are generated according to the size values ​​[52, 32, 64] and the scale values ​​[0.5, 1, 2]. The 9 anchor boxes are input into the fully connected layer of the i-th detection model to obtain the binary classification value and bounding box regression (BB) of each anchor box. Specifically, the binary classification value is the probability that the anchor box is foreground and the probability that it is background. Finally, 300 regions of interest (ROIs) are output. The 300 ROIs are then input into the RPN network for binary classification and Black-Brow regression. ROIs whose anchor boxes are background are filtered out. The remaining ROIs are then aligned in the i-th detection model. The feature maps are interpolated through the ROI alignment network so that each ROI in the remaining ROIs generates a feature map of a fixed size. Finally, three output vectors are obtained: the binary classification result, Black-Brow regression, and binary mask for each ROI. The ROI is the object predicted by the i-th detection model. The binary classification result of the ROI is the probability that it is the first object and the probability that it is not the first object. The three losses obtained from the above three vectors are the binary classification loss, regression loss, and mask loss. The contour loss and diameter loss of the first object obtained from the labeled first object and the predicted first object are determined as the loss value of the first object.

[0091] If the loss value is less than or equal to the preset loss value, then the i-th detection model is determined as the contour model;

[0092] If the loss value is greater than the preset loss value, the i-th detection model is adjusted according to the loss value to determine the (i+1)-th detection model. The training process of the above detection model is repeated until the obtained loss value is less than or equal to the preset loss value.

[0093] The contour loss of the first object is: i represents the pixel value at the corresponding position in the image of the first object, and Y represents the first object being marked. The first object predicted by the model; the diameter loss of the first object is:

[0094] Loss Diameters =(D auto -D manual ) / D manual Among them, D autoD is the predicted diameter of the first object. manual The diameter of the first object is predicted.

[0095] In the above scheme, the labeling information of the first object in the fourth image is obtained, and the fourth image is input into the i-th detection model to train the i-th detection model so that the model can predict the first object in the fourth image. The trained detection model is then identified as the contour model. The loss value between the labeled first object and the predicted first object is used as the criterion for whether the contour model has been successfully trained, which can improve the accuracy of the contour model in recognizing the first object.

[0096] 2) Contour model testing: Each image in the test image is input into the contour model, which will predict a contour ROI segmentation map. The overlap between the predicted contour ROI map and the labeled ROI map is calculated. When the overlap is less than a first value, the contour model test is complete.

[0097] Figure 3 This is a flowchart illustrating a method for calibrating an image provided in an embodiment of this application, as shown below. Figure 3 As shown, this method is applied to an image processing device and includes the following steps:

[0098] S310, the image processing device acquires the first diameter of the target object in the first image, wherein the shape of the target object is circular or elliptical.

[0099] Optionally, the first image in step S310 may be a cross-sectional image of a blood vessel.

[0100] The target object in the first image may be an imaging conduit.

[0101] For example, an image processing device acquires the first diameter of an imaging catheter in a cross-sectional image of a blood vessel.

[0102] Optionally, when the target object in step S310 is elliptical, the major axis of the ellipse is used as the first diameter of the target object.

[0103] Optionally, when the target object in step S310 is elliptical, the minor axis of the ellipse is used as the first diameter of the target object.

[0104] Optionally, when the target object in step S310 is elliptical, the average length of the minor axis and major axis of the ellipse is taken as the first diameter of the target object.

[0105] Optionally, step S310 further includes: the image processing device acquiring the original image corresponding to the first image;

[0106] The image processing device inputs the original image into the contour model to determine the first image, which includes the contour image of the target object;

[0107] The image processing device determines the first diameter of the target object based on the outline image of the target object.

[0108] One feasible approach is to input the original image into a contour model, which determines a ROI segmentation map containing a second object, identifies the second object as the target object, and defines the image including the contour of the target object as the first image.

[0109] Optionally, the image processing device determines a first diameter of the target object based on the contour image of the target object, including:

[0110] The area of ​​the target object in the first image is determined based on the outline image of the target object;

[0111] The first diameter of the target object is determined based on its area.

[0112] In one possible approach, the image processing device uses a contour model to determine the number of pixels in the contour image of the target object, determines the area of ​​the target object based on the number of pixels and the square of the pixel pitch, and then determines the first diameter of the target object based on the relationship between area and diameter, where the square of the pixel pitch is used to represent the area represented by each pixel.

[0113] For example, if the number of pixels is 56600, the square of the pixel pitch is 0.01, and the diameter formula is... The area of ​​the target object is obtained from 56600 * 0.01 = 566. Then, based on... The first diameter of the target object is 27, where S is the area of ​​the target object.

[0114] Optionally, the image processing device obtains the number of pixels along the longest axis of the target object and determines this number as the first diameter of the target object.

[0115] For example, if the number of pixels along the longest axis of the target object is 26, then the first diameter of the target object is 26.

[0116] S320, the image processing device adds an image of a reference object to the first image to obtain a second image. The second image includes the target object and the reference object. The reference object is circular in shape and has a second diameter. The image coordinate system corresponding to the first image and the second image is the world coordinate system.

[0117] Optionally, in step S320, the first image may be a cross-sectional image of a blood vessel, and the first image includes an imaging catheter;

[0118] The reference object can be a marker ring.

[0119] For example, an image processing device adds an image of a marker ring to a cross-sectional image of a blood vessel including an imaging catheter to obtain a second image, which then includes the imaging catheter and the marker ring.

[0120] S330, the image processing device converts the second image into a third image, wherein the image coordinate system corresponding to the third image is a polar coordinate system.

[0121] S340, the image processing device determines the fourth radius of the target object in the third image based on the third radius, first diameter and second diameter of the reference object in the third image in polar coordinates.

[0122] Optionally, step S340 includes: the image processing device determining the ratio of the second diameter to the first diameter based on the first diameter and the second diameter;

[0123] The image processing device determines the fourth radius based on the ratio of the second diameter to the first diameter and the third radius.

[0124] For example, based on F2 = F1 ÷ (D1 / D2), which is the ratio of the third radius of the reference object to the fourth radius of the target object in polar coordinates, and the ratio of the second diameter of the reference object to the first diameter of the target object in world coordinates, the fourth radius is determined. Here, F1 is the third radius of the reference object. Figure 4 The distance between the marked ring and the θ axis is given by the formula F1 = R ÷ (7 / 2) * (0.89 / 2); R is the calibration range within the field of view of 7 mm; F2 is the fourth radius of the target object, i.e., as shown in the figure. Figure 4 The distance between the position of the imaging conduit and the θ axis is shown; D1 is as follows Figure 5 The reference object shown, namely the second diameter of the marking ring, D2, is as follows: Figure 5 The target object shown is the first diameter of the imaging conduit.

[0125] Optionally, step S340 includes: the image processing device determining a first ratio of the first diameter to the second diameter based on the first diameter and the second diameter;

[0126] The image processing device determines the fourth radius based on the product of the third radius and the first ratio.

[0127] For example, the fourth radius is determined by F2 = F1 × (D2 / D1), which is the ratio of the fourth radius of the target object to the third radius of the reference object in the polar coordinate system, and the ratio of the first diameter of the target object to the second diameter of the reference object in the world coordinate system. Here, F1 is the third radius of the reference object, specifically F1 = R ÷ (7 / 2) * (0.89 / 2); R is the calibration range under the field of view of 7mm; F2 is the fourth radius of the target object; D1 is the second diameter of the reference object, which can be the second diameter of the marking ring; and D2 is the first diameter of the imaging conduit, which can be the first diameter of the imaging conduit.

[0128] S350, the image processing device determines the calibration value based on the third and fourth radii.

[0129] Optionally, step S350 includes: the image processing device determining a calibration value based on the difference between the third radius and the fourth radius, and a first initial value, wherein the first initial value is an initial calibration value.

[0130] For example, based on F'=F2-F1=F1÷(D1 / D2)-F1=F1÷(D1 / D2-1) and the first initial value S, the calibration value is determined by S'=S+F1÷(D1 / D2-1), where S represents the initial calibration value, specifically 0, F1 is the third radius of the reference object, F2 is the fourth radius of the target object, D1 is the second diameter of the reference object, D2 is the first diameter of the target object, F′ is the difference between the third radius and the fourth radius, and S' is the calibration value.

[0131] Optionally, step S350 includes: the image processing device performs absolute value processing on the difference between the fourth radius and the third radius, and then determines the first difference;

[0132] The image processing device determines the calibration value based on the sum of the first difference and the first initial value, wherein the first initial value is the initial calibration value.

[0133] For example, based on F'=|F1-F2|=|F1×(D2 / D1)-F1|=F1×(D2 / D1-1) and the first initial value S, the calibration value is determined by S'=S+F1×(D2 / D1-1), where S represents the initial calibration value, specifically 0, F1 is the third radius of the reference object, F2 is the fourth radius of the target object, D1 is the second diameter of the reference object, D2 is the first diameter of the target object, F′ is the difference between the third radius and the fourth radius, and S' is the calibration value.

[0134] S360, the image processing device determines the calibrated image based on the calibration value.

[0135] Optionally, step S360 includes: the image processing device calibrates the third image according to the calibration value to determine the calibrated image in the polar coordinate system.

[0136] For example, such as Figure 6 The image shown is an example of a third image before and after calibration in polar coordinates, according to an embodiment of this application. Figure 6 (a) is the third image. Figure 6 (b) is the image after calibration of the third image. The specific process is as follows: In Figure 6 In the third image of (a), taking the starting point 0 on the vertical axis as a reference, the distance downwards from the calibration value F′ is determined. The third image is then horizontally cropped from F′. At this point, the width of the third image is less than 670 pixels, so zero-padding is performed on the third image, which results in the following: Figure 6 The image shown in (b), i.e. the calibrated image, has 670 pixels. Specifically, in the current environment, the height range of the image in the polar coordinate system corresponding to a field of view of 7 mm is 670 pixels.

[0137] Optionally, step S360 includes: the image processing device calibrates the third image according to the calibration value to obtain a calibrated image in polar coordinates;

[0138] Image processing equipment converts an image calibrated in polar coordinates to an image calibrated in world coordinates.

[0139] For example, such as Figure 7 As shown, the following is given: Figure 6 (b) The image calibrated in the polar coordinate system is converted to the image calibrated in the world coordinate system.

[0140] In the above scheme, the third image is calibrated according to the calibration value. The transformation relationship between polar coordinates and the world coordinate system is used to convert the calibrated third image into a calibrated image in the world coordinate system. That is, under the determined calibration value, the calibrated image is obtained by using the coordinate transformation relationship corresponding to the image, without the need for manual image calibration, thus avoiding the problem of low calibration accuracy caused by manual image calibration.

[0141] Figure 8 This is a schematic diagram of the structure of an image calibration device provided in an embodiment of this application, as shown below. Figure 8 As shown, the apparatus provided in this embodiment includes:

[0142] The acquisition unit 810 is used to acquire the first diameter of the target object in the first image, wherein the shape of the target object is a circle or an ellipse;

[0143] The processing unit 820 is used to add an image of a reference object to the first image to obtain a second image. The second image includes a target object and a reference object. The reference object has a circular shape and a second diameter. The image coordinate system corresponding to the first image and the second image is the world coordinate system.

[0144] The conversion unit 830 is used to convert the second image into a third image, wherein the image coordinate system corresponding to the third image is a polar coordinate system.

[0145] The determining unit 840 is used to determine the fourth radius of the target object in the third image based on the third radius, first diameter and second diameter of the reference object in the third image in polar coordinates.

[0146] The determining unit is further configured to determine a calibration value based on the third radius and the fourth radius;

[0147] The determining unit is also used to determine the calibrated image based on the calibration value.

[0148] Optionally, the acquisition unit is further configured to acquire the original image corresponding to the first image.

[0149] Optionally, the determining unit is further configured to input the original image into the contour model and determine a first image, wherein the first image includes the contour image of the target object.

[0150] Optionally, the determining unit is further configured to determine a first diameter of the target object based on the contour image of the target object.

[0151] Optionally, the determining unit is specifically used to determine the area of ​​the target object in the first image based on the contour image of the target object;

[0152] The first diameter of the target object is determined based on its area.

[0153] Optionally, the acquisition unit is further configured to acquire the marking information of the first object in the fourth image, the marking information being used to mark the first object in the fourth image, the first object being circular or elliptical.

[0154] Optionally, the processing unit is further configured to input the fourth image into the i-th detection model to obtain the predicted object.

[0155] Optionally, the determining unit is further configured to determine the loss value of the first object based on the marked first object and the predicted object in the fourth image.

[0156] Optionally, the determining unit is further configured to determine whether the i-th detection model is a contour model based on the loss value of the first object.

[0157] Optionally, the determining unit is further configured to determine the contour loss and diameter loss of the first object based on the marked first object and the predicted object in the fourth image;

[0158] The loss value of the first object is determined based on the contour loss and diameter loss.

[0159] Optionally, the determining unit is further configured to determine the i-th detection model as a contour module if the loss value of the first object is less than or equal to a preset loss value.

[0160] Optionally, the determining unit is further configured to determine the ratio of the second diameter to the first diameter;

[0161] The fourth radius is determined based on the ratio of the second diameter to the first diameter and the third radius.

[0162] Optionally, the processing unit is further configured to calibrate the third image according to the calibration value to obtain a calibrated image in polar coordinates.

[0163] Convert the image calibrated in polar coordinates to the image calibrated in world coordinates.

[0164] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0165] Based on the same inventive concept Figure 9 An image processing apparatus provided in this application includes a processor coupled to a memory. The processor is used to execute a computer program or instructions stored in the memory to implement the method of the first aspect or any embodiment of the first aspect described above.

[0166] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a device. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer chip, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0167] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0169] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0170] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0171] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for calibrating an image, characterized in that, The method is applied to an image processing device, including: Obtain the first diameter of the target object in the first image, wherein the shape of the target object is circular or elliptical; Adding an image of a reference object to the first image yields a second image. The second image includes the target object and the reference object. The reference object is circular in shape and has a second diameter. The image coordinate system corresponding to the first image and the second image is the world coordinate system. The second image is converted into a third image, and the image coordinate system corresponding to the third image is a polar coordinate system; Determine the ratio of the second diameter to the first diameter; Based on the ratio of the second diameter to the first diameter and the third radius of the reference object in the third image in the polar coordinate system, the fourth radius of the target object in the third image is determined. The calibration value is determined based on the third radius and the fourth radius; The third image is calibrated according to the calibration value to obtain a calibrated image in polar coordinates. The image calibrated in the polar coordinate system is converted into an image calibrated in the world coordinate system.

2. The method as described in claim 1, characterized in that, The method further includes: Obtain the original image corresponding to the first image; The original image is input into the contour model to determine the first image, which includes the contour image of the target object; The first diameter of the target object is determined based on the outline image of the target object.

3. The method as described in claim 2, characterized in that, Determining the first diameter of the target object based on its contour image includes: The area of ​​the target object in the first image is determined based on the outline image of the target object; The first diameter of the target object is determined based on the area of ​​the target object.

4. The method as described in claim 2, characterized in that, The method further includes: Obtain the marking information of the first object in the fourth image, wherein the marking information is used to mark the first object in the fourth image, and the first object is circular or elliptical; The fourth image is input into the i-th detection model to obtain the predicted object; The loss value of the first object is determined based on the first object marked in the fourth image and the predicted object; Based on the loss value of the first object, determine whether the i-th detection model is the contour model; Where i is a positive integer.

5. The method according to claim 4, characterized in that, Determining the loss value of the first object based on the marked first object in the fourth image and the predicted object includes: The contour loss and diameter loss of the first object are determined based on the first object marked in the fourth image and the predicted object. The loss value of the first object is determined based on the contour loss and the diameter loss.

6. The method according to claim 4, characterized in that, The step of determining whether the i-th detection model is the contour model based on the loss value of the first object includes: If the loss value of the first object is less than or equal to the preset loss value, the i-th detection model is determined as the contour model.

7. An apparatus for calibrating images, characterized in that, The device includes: The acquisition unit is used to acquire the first diameter of a target object in the first image, wherein the target object is circular or elliptical in shape; The determining unit is used to add reference object processing to the first image and determine a second image, the second image including the target object and the reference object, the reference object having a circular shape and a second diameter, and the coordinate system of the image data corresponding to the first image and the second image being the world coordinate system; A conversion unit is used to convert the second image into a third image, wherein the coordinate system of the image data corresponding to the second image is a polar coordinate system; The determining unit is further configured to determine the ratio of the second diameter to the first diameter; and to determine the fourth radius of the target object in the third image based on the ratio of the second diameter to the first diameter and the third radius of the reference object in the third image in the polar coordinate system. The determining unit is further configured to determine a calibration value based on the third radius and the fourth radius; The determining unit is further configured to calibrate the third image according to the calibration value to obtain a calibrated image in polar coordinates; and to convert the calibrated image in polar coordinates into a calibrated image in world coordinates.

8. An image processing apparatus, comprising a processor coupled to a memory, the processor being configured to execute a computer program or instructions stored in the memory to implement the method as claimed in any one of claims 1-6.

Images