Arthroscope image enhancement method and system based on artificial intelligence

Abstract

The present invention discloses an image enhancement method and system for an arthroscopy based on artificial intelligence. The joint denoising algorithm is used to remove the influence of the solution during the movement of the arthroscopy. Then, by inserting a 0 value into the denoised image, the number of pixels is increased, thereby increasing the pixel size, thereby increasing the clarity and the field of view. When using a triangle enhancement algorithm, the pixel value is modified by constructing a set of 9 adjacent pixels and obtaining 4 triangle areas, so that the transition between pixel values ​​is smoother. The edge of the target photographed by the arthroscopy is refined according to the contour enhancement algorithm. An improved convolutional neural network is used to perform image enhancement operations based on the clarity and field of view of the image, so that the clarity and field of view of the enhanced image meet the standards. This lays the foundation for the accuracy of subsequent operations. The technical effect of more accurately enhancing the clarity of images photographed by the arthroscopy and increasing the field of view is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an arthroscopic image enhancement method and system based on artificial intelligence. Background Art

[0002] Currently, a solution is injected into the joint to stretch the joint, and then an arthroscope is used for detection. The solution will vibrate when the arthroscope moves. And when the camera device captures the image, the image quality is poor due to motion blur or occlusion problems or the accuracy of the device itself. The same natural image often has different types of degradation in different areas. For example, noise mainly affects the image area, and blur mainly affects the imaging quality of the edge. Due to the complex structure of the joint, the image captured by the arthroscope when it moves in the joint may not be clear enough or the field of view is not wide enough. Summary of the invention

[0003] The purpose of the present invention is to provide an arthroscopic image enhancement method and system based on artificial intelligence to solve the above-mentioned problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides an arthroscopic image enhancement method based on artificial intelligence, comprising: acquiring joint images at multiple time points; the joint images are images taken by the arthroscopic device during the detection process;

[0005] Using a joint denoising algorithm, based on the joint part images at the plurality of time points, the influence of the solution during the arthroscopic movement is reduced to obtain a joint denoised image;

[0006] By using a triangular enhancement algorithm, the clarity is enhanced based on the joint denoising image to obtain an interpolated enhanced image; the interpolated enhanced images at multiple time points are obtained corresponding to the joint part images at multiple time points;

[0007] Through the contour enhancement algorithm, based on the interpolated enhanced images at multiple time points, the pixel values ​​are adjusted according to the contour of the target to obtain the first clear image;

[0008] Based on the first clear image, the image is projected into two spaces to acquire features, enhance the image, and obtain an arthroscopic enhanced image.

[0009] Optionally, the joint denoising algorithm is used to reduce the influence of the solution during the movement of the arthroscopy based on the joint part images at the multiple time points to obtain the joint denoised image, including:

[0010] Based on the joint images at multiple time points, water wave images at multiple time points are obtained; the water wave images are images that mark the water waves caused by the arthroscope during movement;

[0011] Convert the joint image from RGB space to HSV space to obtain an HSV joint image;

[0012] Extract the channels corresponding to the hue and saturation from the HSV joint image to obtain a hue-saturated joint image;

[0013] The hue and lightness of the position corresponding to the water wave in the water wave image in the hue-saturated joint part image are set to -1 to obtain an overlapping image;

[0014] The overlapping image and the hue-saturated joint part image are superimposed and input into the joint detection denoising network, and the hue and saturation of the water wave are adjusted to obtain a joint denoising image.

[0015] Optionally, the step of enhancing the clarity of the joint image by a triangular enhancement algorithm to obtain an interpolated enhanced image includes:

[0016] Based on the joint image, a plurality of triangular region sets are obtained; one triangular region set includes 4 triangular regions;

[0017] Find the average value of each edge in the triangular area as the gradient change value; a triangular area set contains 12 gradient change values;

[0018] According to the triangular region set, a midpoint of the triangular region and a boundary point of the triangular region are obtained; the midpoint of the triangular region includes 4 gradient change values; the boundary point of the triangular region includes 2 gradient change values;

[0019] The four gradient change values ​​are averaged to obtain the midpoint average value of the triangular area;

[0020] According to the average value of the midpoints of the triangular area, the values ​​of the boundary points of the triangular area are adjusted according to the following method:

[0021] std(a,b)>a1,c=c1; std(a,b)<=a1,c=(a+b) / 2;

[0022] Wherein, std() represents the standard deviation formula, a represents a gradient change value in the boundary point of the triangular area, b represents another gradient change value in the boundary point of the triangular area, c represents the adjusted value of the boundary point of the triangular area, c1 represents the average value of the midpoint of the triangular area, and a1 represents the difference threshold;

[0023] The midpoints of the triangle regions and the boundary points of the triangle regions of the plurality of triangle region sets are adjusted to obtain an interpolated enhanced image.

[0024] Optionally, the step of obtaining the first clear image by using a contour enhancement algorithm based on the interpolated enhanced images at multiple time points and adjusting the pixel values ​​according to the contour of the target includes:

[0025] Through the tracking algorithm, the same target is marked based on the interpolated enhanced images at multiple time points to obtain the same target area;

[0026] Perform edge detection according to the same target area to obtain the same edge contour;

[0027] According to the same edge contours of the interpolated enhanced images at two adjacent time points, position matching is performed through a matching network to obtain two matching images; the matching images are images that mark the positions of the same targets;

[0028] An average value is calculated for the matching positions in the two matching images to obtain a first clear image.

[0029] Optionally, projecting the first clear image into two spaces to enhance the image and obtain an arthroscopic enhanced image includes:

[0030] Stretch the first clear image from top to bottom and from left to right into a joint vector;

[0031] Performing Fourier transformation on the joint vector to obtain a first joint transformation feature map;

[0032] Inputting the first clear image into a first convolutional network, extracting image features, and obtaining a second joint feature map;

[0033] The first joint transformation feature map and the second joint feature map are input into the enhancement network to obtain an arthroscopic enhanced image.

[0034] Optionally, the obtaining of a plurality of triangular area sets based on the joint part image includes:

[0035] Inserting a column of all zeros between two adjacent columns of the joint image, and inserting a row of all zeros between two adjacent rows, to obtain a first intermediate expanded image;

[0036] The four adjacent values ​​of the joint part image are corresponding to the nine values ​​surrounded by the four values ​​in the first intermediate expanded image as a nine-number region, to obtain a plurality of nine-number regions;

[0037] Divide the nine-number region into regions containing three adjacent values, and obtain four triangular regions;

[0038] The four triangular regions are formed into a set to obtain a triangular region set; and multiple triangular region sets are obtained corresponding to multiple nine-number regions.

[0039] Optionally, the water wave images at multiple time points are obtained based on the joint part images at multiple time points, including:

[0040] Obtaining the solution type and the arthroscopic time difference; the arthroscopic time difference indicates the time length between the time when the lens of the arthroscopic lens takes pictures and the pre-shooting position reached by the front end; the solution type indicates the type of solution injected into the joint to support the joint;

[0041] According to the type of the solution, a water wave propagation frequency and a water wave propagation frequency amplitude are obtained;

[0042] A water wave image is obtained according to the arthroscopic time difference, the water wave propagation frequency and the water wave propagation frequency amplitude; the water wave image represents an image of the transverse wave generated in the water when the arthroscopic movement occurs;

[0043] The joint images at multiple time points correspond to the water wave images at multiple time points.

[0044] Optionally, the step of inputting the first joint transformation feature map and the second joint feature map into an enhancement network to obtain an arthroscopic enhanced image comprises:

[0045] Fusing the first joint transformation feature map and the second joint feature map to obtain a first fused feature;

[0046] The first fused feature is convolved with the inverse convolution kernel to expand the feature map and obtain an enhanced image.

[0047] Optionally, the training method of the matching network includes:

[0048] Obtain two training images; the two training images are images containing the same edge contours in the interpolated enhanced images at two adjacent time points;

[0049] According to the same edge contours of the two training images, a rectangular area containing the same edge contour is obtained as a first edge contour area and a second edge contour area;

[0050] Matching the center point of the first edge contour area with the center point of the second edge contour area to obtain a matching center point;

[0051] According to the matching center point, matching the first edge contour region with the position where two identical edge contours overlap in the second edge contour region to obtain a labeled image;

[0052] Inputting the two training images into a matching network to obtain a predicted image;

[0053] The loss is calculated based on the labeled image and the predicted image, and the matching network is trained by backpropagation.

[0054] In a second aspect, an embodiment of the present invention provides an arthroscopic image enhancement system based on artificial intelligence, comprising:

[0055] Acquisition module: acquires joint images at multiple time points; the joint images are images taken by the arthroscope during the detection process;

[0056] Joint denoising module: using a joint denoising algorithm, based on the joint part images at the plurality of time points, to reduce the influence of the solution during the arthroscopic movement, and obtain a joint denoising image;

[0057] Triangle enhancement module: using a triangle enhancement algorithm, based on the joint denoising image, the definition is enhanced to obtain an interpolated enhanced image; the joint part images at multiple time points correspond to the interpolated enhanced images at multiple time points;

[0058] Contour enhancement module: Through the contour enhancement algorithm, based on the interpolation enhancement image at multiple time points, the pixel value is adjusted according to the contour of the target to obtain the first clear image;

[0059] Clear enhancement module: Based on the first clear image, project it into two spaces, enhance the image, and obtain an arthroscopic enhanced image.

[0060] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0061] An embodiment of the present invention also provides an image enhancement method and system for an arthroscopic image based on artificial intelligence, the method comprising: acquiring images of joint parts at multiple time points. The images of joint parts are images taken by the arthroscopy during the detection process. Using a joint denoising algorithm, based on the images of joint parts at the multiple time points, the influence of the solution during the movement of the arthroscopy is reduced to obtain a joint denoised image. Using a triangular enhancement algorithm, based on the joint denoised image, the clarity is enhanced to obtain an interpolated enhanced image. The interpolated enhanced images of the multiple time points are obtained corresponding to the joint part images at the multiple time points. Using a contour enhancement algorithm, based on the interpolated enhanced images at the multiple time points, the pixel values ​​are adjusted according to the contour of the target to obtain a first clear image. Based on the first clear image, features are projected into two spaces to obtain features, enhance the image, and obtain an arthroscopic enhanced image.

[0062] In the present invention, the influence of the solution caused by the arthroscope during the movement is removed by a joint denoising algorithm. Then, by inserting a 0 value into the denoised image, the number of pixels is increased, thereby increasing the pixel, thereby increasing the clarity and the field of view. When using a triangle enhancement algorithm, the corresponding 0 value is modified by constructing a set of 9 adjacent pixels and obtaining 4 triangle areas, so that the transition between pixel values ​​is smoother. Multiple images are taken during operation; enhancement judgment is made based on the multiple images. The edge of the target photographed by the arthroscope is refined according to the contour enhancement algorithm. The refined image is projected into two spaces to obtain features. The features of the two spaces are compared to enhance the clarity. First, an image of the surgical site is collected through an arthroscope to determine whether the clarity and field of view of the image meet the standards. If not, an improved convolutional neural network is used to perform image enhancement operations based on the clarity and field of view of the image, so that the clarity and field of view of the enhanced image meet the standards. This lays the foundation for the accuracy of subsequent operations. The technical effect of more accurately enhancing the clarity of the image taken by the arthroscope and increasing the field of view is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a flow chart of an arthroscopic image enhancement method based on artificial intelligence provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0065] Example 1

[0066] like Figure 1 As shown, an embodiment of the present invention provides an arthroscopic image enhancement method based on artificial intelligence, the method comprising:

[0067] S101: Acquire joint images at multiple time points. The joint images are images taken by the arthroscope during the detection process.

[0068] The joint part image is obtained by injecting a solution to open the joint and inserting the arthroscope into the joint opened by the solution. When the arthroscope enters the joint, it continuously takes images including part of the joint to obtain joint part images at multiple time points.

[0069] S102: Using a joint denoising algorithm, based on the joint part images at the multiple time points, the influence of the solution during the movement of the arthroscopy is reduced to obtain a joint denoised image.

[0070] S103: enhancing the clarity of the first adjusted image based on the first joint transformation image by using a triangular enhancement algorithm to obtain an interpolated enhanced image.

[0071] S104: using a contour enhancement algorithm, based on the interpolated enhanced images at multiple time points, adjusting the pixel values ​​according to the contour of the target, to obtain a first clear image.

[0072] S105: Based on the first clear image, project it into two spaces to acquire features, enhance the image, and obtain an arthroscopic enhanced image.

[0073] Optionally, the joint denoising algorithm is used to reduce the influence of the solution during the movement of the arthroscopy based on the joint part images at the multiple time points to obtain the joint denoised image, including:

[0074] Based on the joint images at multiple time points, water wave images at multiple time points are obtained, wherein the water wave images are images that mark the water waves caused by the arthroscope during movement.

[0075] Among them, the joint part image at a time point corresponds to the water wave image at a time point.

[0076] The joint part image is converted from the RGB space to the HSV space to obtain an HSV joint part image.

[0077] Among them, in this embodiment, the BGR2HSV function in opencv can be directly used for conversion.

[0078] The pages corresponding to the hue and saturation are extracted from the HSV joint image to obtain a hue-saturated joint image.

[0079] Among them, in different media, the color of the image may change due to factors such as refraction and reflection of light. Generally speaking, in air and in water, the hue (Hue, H) and saturation (Saturation, S) parts of the HSV (Hue, Saturation, Value) color space of the image will change, while the value (Value, V) part is relatively stable. Specifically, in water, due to the different refractive indices, the light will refract, causing the hue of the image to change. At the same time, due to the absorption of water, the intensity of the light will weaken, causing the saturation of the image to decrease. In the air, due to factors such as the scattering of light, the hue and saturation of the image will also be affected. Therefore, only the H and S channels corresponding to the hue and saturation of the HSV joint image are extracted for extraction.

[0080] The number of channels of the hue-saturated joint image is 2.

[0081] The hue and lightness of the position corresponding to the water wave in the water wave image in the hue-saturated joint part image are set to -1 to obtain an overlapping image.

[0082] Among them, setting the hue and lightness of the position corresponding to the water wave in the water wave image in the hue-saturated joint part image to -1 means that the pixel value of this position is deleted.

[0083] The overlapping image and the hue-saturated joint part image are superimposed and input into the joint detection denoising network, and the hue and saturation of the water wave are adjusted to obtain a joint denoising image.

[0084] The joint detection denoising network includes a convolutional neural network (CNN) and an algorithm for converting HSV to RGB. In this embodiment, the algorithm is HSV2RGB in opencv.

[0085] Wherein, the joint denoising image is an RGB image.

[0086] Through the above method, because the joint needs to be stretched open with a solution when the arthroscopy is performed, the solution may fluctuate during the movement of the arthroscopy, resulting in inaccurate detection, so the pixel value at the water wave needs to be re-judged.

[0087] Optionally, the step of enhancing the clarity based on the joint denoised image by a triangular enhancement algorithm to obtain an interpolated enhanced image includes:

[0088] Based on the joint image, a triangular region set is obtained. A triangular region set includes 4 triangular regions.

[0089] The triangular area forms a right triangle, and the triangular area includes 6 pixel points.

[0090] The average value of each edge in the triangular region is found as the gradient change value. A triangular region set contains 12 gradient change values.

[0091] Wherein, the triangular area contains three sides.

[0092] According to the triangular region set, a midpoint of the triangular region and a boundary point of the triangular region are obtained. The midpoint of the triangular region includes 4 gradient change values. The boundary point of the triangular region includes 2 gradient change values.

[0093] The midpoint of the triangular area represents the midpoint of 9 adjacent pixel points in the triangular area set constituting the 3x3 arrangement data.

[0094] The four gradient change values ​​are averaged to obtain the midpoint average of the triangular area.

[0095] According to the average value of the midpoints of the triangular area, the values ​​of the boundary points of the triangular area are adjusted according to the following method:

[0096] std(a,b)>a1,c=c1; std(a,b)<=a1,c=(a+b) / 2.

[0097] Among them, the std represents the standard deviation formula, a represents a gradient change value in the boundary point of the triangular area, b represents another gradient change value in the boundary point of the triangular area, c represents the adjusted value of the boundary point of the triangular area, c1 represents the average value of the midpoint of the triangular area, and a1 represents the difference threshold.

[0098] In this embodiment, the phase difference threshold is 2.

[0099] The midpoints of the triangle regions and the boundary points of the triangle regions of the plurality of triangle region sets are adjusted to obtain an interpolated enhanced image.

[0100] The adjustment method is the same as the adjustment method of the midpoints and boundary points of the triangle regions of the above-mentioned triangle region set.

[0101] Through the above method, because the four triangular areas formed by a triangular area set all contain the middle point, the average value that satisfies the four triangular areas is found, and the difference in the value of the middle point on an edge in each triangular area is different, so as to adjust other positions in the triangular area. And consider the difference in the pixel difference of an edge that two triangular areas pass through, so as to find the most satisfactory added pixel value. Use three pixel points to predict an inserted pixel point for enhancement.

[0102] Optionally, the step of obtaining the first clear image by using a contour enhancement algorithm based on the interpolated enhanced images at multiple time points and adjusting the pixel values ​​according to the contour of the target includes:

[0103] Through the tracking algorithm, the same target is marked according to the interpolated enhanced images at multiple time points, and the same target area is obtained.

[0104] Among them, in this embodiment, the tracking algorithm is the multi-target tracking algorithm DeepSort.

[0105] Edge detection is performed based on the same target area to obtain the same edge contour.

[0106] In this embodiment, the Canny edge detection algorithm is used to perform edge detection.

[0107] According to the same edge contours of the interpolated enhanced images at two adjacent time points, position matching is performed through a matching network to obtain two matching images. The matching images are images that mark the positions of the same targets.

[0108] An average value is calculated for the matching positions in the two matching images to obtain a first clear image.

[0109] Optionally, projecting the first clear image into two spaces to enhance the image and obtain an arthroscopic enhanced image includes:

[0110] The first clear image is stretched from top to bottom and from left to right to form a joint vector.

[0111] Perform Fourier transform on the joint vector to obtain a first joint transformation feature map.

[0112] The first clear image is input into a first convolutional network to extract image features to obtain a second joint feature map.

[0113] Among them, in this embodiment, the first convolutional network is a convolutional neural network (Convolutional Neural Networks, CNN).

[0114] The first joint transformation feature map and the second joint feature map are input into the enhancement network to obtain an arthroscopic enhanced image.

[0115] Among them, in this embodiment, the enhanced network is a convolutional neural network (CNN), which includes a convolution kernel and a deconvolution kernel.

[0116] Optionally, the method of predicting the values ​​in the triangular area by a triangular enhancement algorithm to obtain an added image includes:

[0117] A column of all zeros is inserted between two adjacent columns of the joint image, and a row of all zeros is inserted between two adjacent rows to obtain a first intermediate expanded image.

[0118] Among them, in the program, we can compare the image to a matrix. The image has height and width, and the matrix has rows and columns. Conventionally, we regard the height of the image as rows, and the width of the image as columns. Because the image is an RGB image, the number of channels of the first adjustment image is 3. For example, if the size of the first adjustment image is 64*64*3, then 64*1*3 zeros are inserted between two adjacent columns of the first adjustment image. 1*64*3 zeros are inserted between two adjacent rows of the first adjustment image. The number of inserted rows is 64-1=63, and the number of inserted columns is 64-1=63. The number of rows of the first intermediate expansion image is 64+63=125, the number of columns is 64+63=125, and the number of channels is 3.

[0119] The nine values ​​surrounded by the four values ​​in the first intermediate expanded image corresponding to the four adjacent values ​​of the joint part image are taken as a nine-number region to obtain a plurality of nine-number regions.

[0120] Among them, the four adjacent values ​​of the joint image are The nine values ​​enclosed by the four values ​​expanded to the first intermediate expanded image are

[0121] The nine-number region is divided into regions containing three adjacent numerical values, thereby obtaining four triangular regions.

[0122] Among them, nine positions correspond to four triangular areas.

[0123] Among them, the four triangular areas are

[0124] The four triangular regions are formed into a set to obtain a triangular region set. Multiple nine-number regions correspond to multiple triangular region sets.

[0125] Optionally, the water wave images at multiple time points are obtained based on the joint part images at multiple time points, including:

[0126] The solution type and the arthroscopic time difference are obtained. The arthroscopic time difference indicates the time length between the time when the lens of the arthroscopic camera shoots and the pre-shooting position reached by the front end. The solution type indicates the type of solution injected into the joint to support the joint.

[0127] Usually, the camera device of the arthroscope is inside the arthroscope, and the front end is at a certain distance from the camera device of the arthroscope. Therefore, when shooting, ripples will be caused because the front end moves.

[0128] According to the solution type, the water wave propagation frequency and the water wave propagation frequency amplitude are obtained.

[0129] The propagation speed of water waves is related to the density of the solution, which is determined by the type of solution. For example, if the solution type is water, the solution density is 1g / cm 3 .

[0130] A water wave image is obtained according to the arthroscopic time difference, the water wave propagation frequency and the water wave propagation frequency amplitude. The water wave ripple image represents an image of transverse waves generated in water when the arthroscopic movement occurs.

[0131] The water wave image is a binary image, where 1 represents the location where the water wave exists, and 0 represents the location where the water wave does not exist.

[0132] The joint images at multiple time points correspond to the water wave images at multiple time points.

[0133] Optionally, inputting the first joint feature into an enhancement network to obtain an enhanced image includes:

[0134] The first joint transformation feature map and the second joint feature map are fused to obtain a first fused feature.

[0135] Among them, fusion is performed by superposition in the channel direction.

[0136] The first fused feature is convolved with the inverse convolution kernel to expand the feature map and obtain an enhanced image.

[0137] Among them, the length of the convolution kernel is greater than the length of the first fusion feature, and the width of the convolution kernel is greater than the width of the first fusion feature.

[0138] The length of the enhanced image is twice the length of the joint image minus 1. The width of the enhanced image is twice the width of the joint image minus 1.

[0139] Optionally, the training method of the matching network includes:

[0140] Two training images are obtained. The two training images are images containing the same edge contours in the interpolated enhanced images at two adjacent time points.

[0141] According to the same edge contours of the two training images, a rectangular area including the same edge contour is obtained as a first edge contour area and a second edge contour area.

[0142] The center point of the first edge contour area is matched with the center point of the second edge contour area to obtain a matching center point.

[0143] According to the matching center point, the first edge contour region is matched with the position where two identical edge contours overlap in the second edge contour region to obtain a labeled image.

[0144] The two training images are input into the matching network to obtain a predicted image.

[0145] The loss is calculated based on the labeled image and the predicted image, and the matching network is trained by backpropagation.

[0146] Among them, the loss value is calculated by cross entropy.

[0147] Among them, the boundaries of the joint images can be padded with 0s and the size and number of deconvolutions in the matching network can be adjusted so that the boundaries are also expanded to increase the field of view.

[0148] Example 2

[0149] Based on the above-mentioned artificial intelligence-based arthroscopic image enhancement method, an embodiment of the present invention also provides an artificial intelligence-based arthroscopic image enhancement system, which includes an acquisition module, a joint denoising module, a triangle enhancement module, a contour enhancement module and a clarity enhancement module.

[0150] The acquisition module is used to acquire joint images at multiple time points. The joint images are images taken by the arthroscopic method during the detection process.

[0151] The joint denoising module is used to reduce the influence of the solution during the movement of the arthroscopy based on the joint part images at the plurality of time points through a joint denoising algorithm to obtain a joint denoised image.

[0152] The triangular enhancement module is used to enhance the clarity based on the joint denoising image through a triangular enhancement algorithm to obtain an interpolated enhanced image. The joint part images at multiple time points correspond to the interpolated enhanced images at multiple time points.

[0153] The contour enhancement module is used to obtain a first clear image by adjusting pixel values ​​according to the contour of the target through a contour enhancement algorithm based on the interpolation enhancement images at multiple time points.

[0154] The clarity enhancement module is used to project the first clear image into two spaces to enhance the image and obtain an arthroscopic enhanced image.

Claims

1. An arthroscopic image enhancement method based on artificial intelligence, characterized in that: include: Acquire images of joint parts at multiple time points; The joint part image is an image taken by the arthroscopy during the detection process; Using a joint denoising algorithm, based on the joint part images at the plurality of time points, the influence of the solution during the arthroscopic movement is reduced to obtain a joint denoised image; By using a triangular enhancement algorithm, the clarity is enhanced based on the joint denoising image to obtain an interpolated enhanced image; the interpolated enhanced images at multiple time points are obtained corresponding to the joint part images at multiple time points; Through the contour enhancement algorithm, based on the interpolated enhanced images at multiple time points, the pixel values ​​are adjusted according to the contour of the target to obtain the first clear image; Based on the first clear image, project it into two spaces to acquire features, enhance the image, and obtain an arthroscopic enhanced image; The method of enhancing the clarity based on the joint denoised image by using a triangular enhancement algorithm to obtain an interpolated enhanced image includes: Based on the joint image, a plurality of triangular region sets are obtained; one triangular region set includes 4 triangular regions; Find the average value of each edge in the triangular area as the gradient change value; a triangular area set contains 12 gradient change values; According to the triangular region set, a midpoint of the triangular region and a boundary point of the triangular region are obtained; the midpoint of the triangular region includes 4 gradient change values; the boundary point of the triangular region includes 2 gradient change values; The four gradient change values ​​are averaged to obtain the midpoint average value of the triangular area; According to the average value of the midpoints of the triangular area, the values ​​of the boundary points of the triangular area are adjusted according to the following method: std(a,b)>a1,c=c1; std(a,b)<=a1,c=(a+b) / 2; Wherein, std() represents the standard deviation formula, a represents a gradient change value in the boundary point of the triangular area, b represents another gradient change value in the boundary point of the triangular area, c represents the adjusted value of the boundary point of the triangular area, c1 represents the average value of the midpoint of the triangular area, and a1 represents the difference threshold; The midpoints of the triangle regions and the boundary points of the triangle regions of the plurality of triangle region sets are adjusted to obtain an interpolated enhanced image.

2. The method for arthroscopic image enhancement based on artificial intelligence according to claim 1, characterized in that: The joint denoising algorithm is used to reduce the influence of the solution during the movement of the arthroscopy based on the joint part images at the multiple time points to obtain the joint denoising image, including: Based on the joint images at multiple time points, water wave images at multiple time points are obtained; the water wave images are images that mark the water waves caused by the arthroscope during movement; Convert the joint image from RGB space to HSV space to obtain an HSV joint image; Extract the channels corresponding to the hue and saturation from the HSV joint image to obtain a hue-saturated joint image; The hue and lightness of the position corresponding to the water wave in the water wave image in the hue-saturated joint part image are set to -1 to obtain an overlapping image; The overlapping image and the hue-saturated joint part image are superimposed and input into the joint detection denoising network, and the hue and saturation of the water wave are adjusted to obtain a joint denoising image.

3. The method for arthroscopic image enhancement based on artificial intelligence according to claim 1, characterized in that: The method of obtaining a first clear image by using a contour enhancement algorithm based on the interpolated enhanced images at multiple time points and adjusting the pixel values ​​according to the contour of the target includes: Through the tracking algorithm, the same target is marked based on the interpolated enhanced images at multiple time points to obtain the same target area; Perform edge detection according to the same target area to obtain the same edge contour; According to the same edge contours of the interpolated enhanced images at two adjacent time points, position matching is performed through a matching network to obtain two matching images; the matching images are images that mark the positions of the same targets; An average value is calculated for the matching positions in the two matching images to obtain a first clear image.

4. The method for arthroscopic image enhancement based on artificial intelligence according to claim 1, characterized in that: The method of projecting the first clear image into two spaces to enhance the image and obtain an arthroscopic enhanced image includes: Stretch the first clear image from top to bottom and from left to right into a joint vector; Performing Fourier transformation on the joint vector to obtain a first joint transformation feature map; Inputting the first clear image into a first convolutional network, extracting image features, and obtaining a second joint feature map; The first joint transformation feature map and the second joint feature map are input into the enhancement network to obtain an arthroscopic enhanced image.

5. The method for arthroscopic image enhancement based on artificial intelligence according to claim 1, characterized in that: Based on the joint part image, a plurality of triangular area sets are obtained, including: Inserting a column of all zeros between two adjacent columns of the joint image, and inserting a row of all zeros between two adjacent rows, to obtain a first intermediate expanded image; The four adjacent values ​​of the joint part image are corresponding to the nine values ​​surrounded by the four values ​​in the first intermediate expanded image as a nine-number region, to obtain a plurality of nine-number regions; Divide the nine-number region into regions containing three adjacent values, and obtain four triangular regions; The four triangular regions are formed into a set to obtain a triangular region set; and multiple triangular region sets are obtained corresponding to multiple nine-number regions.

6. The method for arthroscopic image enhancement based on artificial intelligence according to claim 2, characterized in that: The water wave images at multiple time points are obtained based on the joint part images at multiple time points, including: Obtaining the solution type and the arthroscopic time difference; the arthroscopic time difference indicates the time length between the time when the lens of the arthroscopic lens takes pictures and the pre-shooting position reached by the front end; the solution type indicates the type of solution injected into the joint to support the joint; According to the type of the solution, a water wave propagation frequency and a water wave propagation frequency amplitude are obtained; A water wave image is obtained according to the arthroscopic time difference, the water wave propagation frequency and the water wave propagation frequency amplitude; the water wave image represents an image of the transverse wave generated in the water when the arthroscopic movement occurs; The joint images at multiple time points correspond to the water wave images at multiple time points.

7. The method for arthroscopic image enhancement based on artificial intelligence according to claim 4, characterized in that: The step of inputting the first joint transformation feature map and the second joint feature map into an enhancement network to obtain an arthroscopic enhanced image comprises: Fusing the first joint transformation feature map and the second joint feature map to obtain a first fused feature; The first fused feature is convolved with the inverse convolution kernel to expand the feature map and obtain an enhanced image.

8. The method for arthroscopic image enhancement based on artificial intelligence according to claim 3, characterized in that: The training method of the matching network comprises: Obtain two training images; the two training images are images containing the same edge contours in the interpolated enhanced images at two adjacent time points; According to the same edge contours of the two training images, a rectangular area containing the same edge contour is obtained as a first edge contour area and a second edge contour area; Matching the center point of the first edge contour area with the center point of the second edge contour area to obtain a matching center point; According to the matching center point, matching the first edge contour region with the position where two identical edge contours overlap in the second edge contour region to obtain a labeled image; Inputting the two training images into a matching network to obtain a predicted image; The loss is calculated based on the labeled image and the predicted image, and the matching network is trained by backpropagation.

9. An arthroscopic image enhancement system based on artificial intelligence, characterized in that: include: Acquisition module: acquire joint images at multiple time points; The joint part image is an image taken by the arthroscopy during the detection process; Joint denoising module: using a joint denoising algorithm, based on the joint part images at the plurality of time points, to reduce the influence of the solution during the arthroscopic movement, and obtain a joint denoising image; Triangle enhancement module: using a triangle enhancement algorithm, based on the joint denoising image, the definition is enhanced to obtain an interpolated enhanced image; the joint part images at multiple time points correspond to the interpolated enhanced images at multiple time points; Contour enhancement module: Through the contour enhancement algorithm, based on the interpolation enhancement image at multiple time points, the pixel value is adjusted according to the contour of the target to obtain the first clear image; A clarity enhancement module: based on the first clear image, projecting it into two spaces, enhancing the image, and obtaining an arthroscopic enhanced image; The method of enhancing the clarity based on the joint denoised image by using a triangular enhancement algorithm to obtain an interpolated enhanced image includes: Based on the joint image, a plurality of triangular region sets are obtained; one triangular region set includes 4 triangular regions; Find the average value of each edge in the triangular area as the gradient change value; a triangular area set contains 12 gradient change values; According to the triangular region set, a midpoint of the triangular region and a boundary point of the triangular region are obtained; the midpoint of the triangular region includes 4 gradient change values; the boundary point of the triangular region includes 2 gradient change values; The four gradient change values ​​are averaged to obtain the midpoint average value of the triangular area; According to the average value of the midpoints of the triangular area, the values ​​of the boundary points of the triangular area are adjusted according to the following method: std(a,b)>a1,c=c1; std(a,b)<=a1,c=(a+b) / 2; Wherein, std() represents the standard deviation formula, a represents a gradient change value in the boundary point of the triangular area, b represents another gradient change value in the boundary point of the triangular area, c represents the adjusted value of the boundary point of the triangular area, c1 represents the average value of the midpoint of the triangular area, and a1 represents the difference threshold; The midpoints of the triangle regions and the boundary points of the triangle regions of the plurality of triangle region sets are adjusted to obtain an interpolated enhanced image.

Images