A perianal image target tracking method and system based on visual servoing

Abstract

The application discloses a perianal image target tracking method and system based on visual servoing, relates to the field of medical image processing, and solves the problem of insufficient tracking accuracy of the perianal image target. The method comprises the following steps: S1, acquiring a real-time perianal image; S2, acquiring a lesion confidence, processing the real-time perianal image to obtain a lesion mask image, counting the lesion distribution according to the lesion mask image, and recording lesion parameter information; S3, evaluating the sampling effect of the real-time perianal image, judging the center position of the lesion distribution, regulating the visual collection point according to the center position, and obtaining a regulated perianal image; acquiring the sampling evaluation score of the regulated perianal image, and optimizing the regulation of the visual collection point in combination with the real-time sampling evaluation score; and S4, storing the perianal image and the lesion parameter information. The application can effectively improve the accuracy of target tracking.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and more specifically relates to visual servoing technology, specifically a method and system for tracking perianal images based on visual servoing. Background Technology

[0002] Current methods for tracking targets in perianal images have the following specific drawbacks: 1. Existing perianal image target tracking algorithms are mostly designed for rigid bodies. They track targets by acquiring data in real time within a fixed range. However, this method is not accurate enough when dealing with highly non-rigid soft tissues such as perianal and rectal tissues. When the lesion undergoes large-scale compression, stretching, or other non-rigid deformations, the bounding box of traditional algorithms drifts or even the target is completely lost.

[0003] 2. Existing endoscopes mostly rely on doctors to manually adjust the viewing angle. The accuracy of this adjustment depends on the doctor's operating experience, making it difficult to achieve precise and effective tracking of the target. At the same time, this method increases the doctor's workload and is prone to causing missed diagnoses or accidental injury to patients.

[0004] 3. Existing target tracking algorithms mainly make corresponding changes based on changes in the position of visual images. They focus on ensuring accurate detection of targets but ignore external parameters, resulting in insufficient data utilization. Furthermore, in medical procedures, it is difficult to accurately judge the patient's condition based solely on visual image data.

[0005] To address this, we propose a visual servoing-based method and system for tracking perianal images. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a visual servoing-based method and system for target tracking in perianal images, thereby improving the accuracy of target tracking in perianal images.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method and system for tracking perianal images based on visual servoing, the specific working process of each step is as follows: Step S1: Obtain visual acquisition points and acquire perianal images in real time to obtain real-time perianal images; construct a coordinate system from the real-time perianal images to obtain the perianal reference coordinate system, and bind the real-world parameters to the perianal reference coordinate system; Step S2: Determine the background area and lesion area of ​​the real-time perianal image to obtain the lesion confidence score; based on the lesion confidence score, perform binarization masking on the real-time perianal image to obtain the lesion mask image; based on the lesion mask image, statistically analyze the lesion distribution to obtain lesion distribution information; combine the perianal reference coordinate system to record the actual mapping position and parameters of the lesion to obtain lesion parameter information. Step S3: Based on the lesion distribution information, evaluate the sampling effect of the real-time perianal image to obtain a real-time sampling evaluation score. At the same time, determine the center position of the lesion distribution, adjust the visual acquisition point according to the center position, and acquire the perianal image to obtain the adjusted perianal image. Obtain the sampling evaluation score of the adjusted perianal image, and optimize the adjustment of the visual acquisition point by combining the real-time sampling evaluation score. Step S4: Collect perianal images using the optimized visual sampling points and store them in conjunction with lesion parameter information.

[0008] Furthermore, the specific steps of step S1 are as follows: Step S11: Obtain the acquisition range corresponding to the visual acquisition point, record the actual boundary distance of the acquisition range, and record the actual boundary distance as wlX×wlY; The pixel resolution of the real-time perianal image is acquired and denoted as us × vs; the acquisition parameters of the image acquisition device are obtained, including the acquisition height Z and the equivalent focal length F. x With F y Based on the acquisition height, equivalent focal length, and pixel resolution, the image display range zsX×zsY is calculated. The scaling correction parameter xzc of the image is obtained by proportionally calculating the actual boundary distance wlX×wlY and the image display range zsX×zsY. Step S12: Based on the pixel resolution us×vs of the real-time perianal image, obtain the pixel center (uzx, vzx), where uzx = us / 2 and vzx = vs / 2; construct a polar coordinate system with the pixel center as the origin; traverse the pixels (u, v) in the real-time perianal image, and record the mapping position of the pixels in the real-time perianal image on the polar coordinate system through the acquisition parameters of the image acquisition device to obtain the coordinate points (zjl, zjd); at the same time, associate the image scaling correction parameter xzc with the polar coordinate system to obtain the perianal reference coordinate system.

[0009] Furthermore, the specific steps of step S2 are as follows: Step S21: Collect pixel judgment values ​​of real-time perianal images, normalize the pixel judgment values ​​to obtain the lesion confidence of each pixel, set an adaptive lesion judgment threshold based on the lesion confidence of each pixel, compare the adaptive lesion judgment threshold with the lesion confidence, binarize the real-time perianal images to obtain lesion mask images. Step S22: Divide the lesions according to the lesion mask image; traverse each lesion, iterate over the preset lesion center point, collect the distance from the lesion center point to the lesion boundary to obtain the lesion range value; record the distribution of lesions based on the lesion center point and the lesion range value; map the lesion center point to the perianal reference coordinate system according to the lesion distribution, combine with real-world parameters to record the actual mapped position of the lesion, extract the boundary pixels within the lesion range through the lesion range value, and record the actual parameters of the lesion; obtain the lesion parameter information from the actual mapped position and actual parameters of the lesion.

[0010] Furthermore, the specific steps of step S21 are as follows: Step S211: Transmit the real-time perianal image to the ultra-lightweight medical image segmentation model, process the real-time perianal image, and obtain the pixel judgment value of the real-time perianal image; The pixel judgment values ​​of real-time perianal images are statistically analyzed and denoted as xsp(u,v); the pixel judgment values ​​xsp(u,v) are normalized to obtain the lesion confidence of the pixel zxd(u,v); Step S212: Calculate based on lesion confidence zxd(u,v) and set an adaptive lesion judgment threshold pdy; The adaptive lesion judgment threshold pdy and the lesion confidence score zxd(u, v) are compared by iterative process: If pdy≥zxd(u,v), it indicates that the pixel (u,v) corresponding to the lesion confidence zxd(u,v) is a low-confidence background, and the pixel value xsz(u,v) at pixel (u,v) is assigned to 0; If pdy < zxd(u,v), it indicates that the pixel (u,v) corresponding to the lesion confidence zxd(u,v) is a high-confidence lesion, and the pixel value xsz(u,v) at pixel (u,v) is assigned the value of 255; After traversing all pixels and their corresponding pixel values, a binary image of the real-time perianal image is obtained, which is denoted as the lesion mask image.

[0011] Furthermore, the specific steps of step S22 are as follows: Step S221: Perform continuity judgment on the lesion mask image. If the pixel value of pixel (u, v) in the lesion mask image is 255, extract the pixel values ​​of its neighboring pixels, and merge pixels with the same pixel value to form a set of local lesions. The local lesion set is statistically analyzed, and the lesion mask image is divided into multiple lesions. Each local lesion set is traversed and iterated to analyze the lesion center point. The pixel positions of the elements in the local lesion set are integrated with the lesion center point, and the distance from the lesion center point to the lesion boundary is recorded to obtain the lesion range value. The local lesions are marked in combination with the lesion center point, and the distribution of the lesions is recorded. Step S222: Extract the center point of each lesion according to the distribution of lesions, map the center point of the lesion to the perianal reference coordinate system, and correct the distance value by combining real-world parameters to obtain the real-world mapped position of the lesion. The boundary pixels within the lesion range corresponding to the center point of each lesion are extracted, the lesion parameters are recorded, and the parameters are converted into real-world parameters and recorded in combination with real-world parameters.

[0012] Furthermore, the specific steps of step S221 are as follows: Obtain the boundary elements in the set of local lesions, and collect the distribution extreme values ​​based on the boundary elements; preset the lesion center point based on the distribution extreme values, and denote the distribution center point as fbz(u,v); The distances between the distribution center point and each boundary element are collected. The maximum distance between the distribution center point and each boundary element is used as the division radius. The distribution center point and the division radius are recorded as the initial iteration data. Based on the distribution center point and the division radius, the local lesion set is divided into two parts. The distribution center point and the division radius of each part are obtained, and the iteration continues. The number of distribution center points and the division radius of each part are recorded. The iteration results are evaluated to obtain the iteration evaluation value dpg. The iterative result with the smallest iterative evaluation value is collected, the center point of the lesion and its corresponding division radius are recorded, the lesion range value corresponding to the center point of the lesion is obtained from the division radius, and the center point of the lesion and its corresponding lesion range value are statistically stored to obtain the distribution of the lesions.

[0013] Furthermore, the specific steps of step S3 are as follows: Step S31: Obtain the center point and range value of the lesion from the lesion distribution information; map the center point of the lesion to the perianal reference coordinate system to obtain the mapped coordinates; judge the sampling deviation of the lesion based on the mapped coordinates; calculate the lesion sampling evaluation value by combining the lesion range value; traverse all lesions in the real-time perianal image and integrate their corresponding lesion sampling evaluation values ​​to obtain the real-time sampling evaluation score; traverse the mapped coordinates of all lesion center points and calculate the center position of the lesion distribution to obtain the distribution center. Step S32: Adjust the visual acquisition points according to the distribution center, acquire the perianal image of the adjusted visual acquisition points, and obtain the adjusted perianal image; acquire the lesion distribution information of the adjusted perianal image, analyze the sampling effect of the adjusted perianal image, obtain the adjustment sampling evaluation score, and judge the adjustment result of the visual acquisition points in combination with the real-time sampling evaluation score, and perform feedback optimization.

[0014] Furthermore, the specific steps of step S31 are as follows: Step S311: Map the center point of the lesion to the perianal reference coordinate system to obtain the mapped coordinates ysz(zjl, zjd); count the number of lesion center points and denote them as bs; map all lesion center points and record their corresponding mapped coordinates ysz. b (zjl, zjd); Collect the lesion range value corresponding to the center point of each lesion, denoted as bzf. b ; the mapping coordinates ysz corresponding to the center point of the lesion b (zjl, zjd) and the lesion extent value bzf b The sampling effect of real-time perianal images is evaluated to obtain the real-time sampling evaluation score cpg; Step S312: Obtain the mapped coordinates ysz1(zjl, zjd) and ysz2(zjl, zjd) of the center points of the two lesions, and their corresponding lesion range values ​​bzf1 and bzf2. Use the lesion range values ​​as weights to extract the midpoint coordinates of the mapped coordinates ysz1(zjl, zjd) and ysz2(zjl, zjd) to obtain the midpoint coordinates zzb(zjl, zjd). The mapped coordinates of all lesion center points are traversed, the mapped coordinates of the lesion center points and the midpoint coordinates are extracted and calculated, and the midpoint coordinates are modified and iterated to obtain the center position of the lesion distribution.

[0015] Furthermore, the specific steps of step S32 are as follows: Step S321: Based on the distribution center, acquire the coordinate data of the center position. Based on the coordinate data of the center position, obtain the adjustment direction and adjustment distance of the visual acquisition point. Based on the adjustment direction and adjustment distance of the visual acquisition point, move the image acquisition device and acquire the perianal image of the corresponding position after the movement to obtain the adjusted perianal image. Step S322: Based on the real-time perianal image lesion distribution information acquisition step, the lesion distribution information of the perianal image is acquired, and the sampling effect of the perianal image is evaluated and calculated by combining the calculation formula of the real-time sampling evaluation score to obtain the control sampling evaluation score; the control result of the visual acquisition point is judged by the control sampling evaluation score and the real-time sampling evaluation score. If the control sampling evaluation score is lower than the real-time sampling evaluation score, it indicates that the control is abnormal, and the abnormality is transmitted for manual verification; if the control sampling evaluation score is higher than the real-time sampling evaluation score, it indicates that the control is normal and the target tracking of the lesion is achieved.

[0016] A visual servoing-based perianal image target tracking system, the tracking system comprising: Data acquisition module: acquires visual acquisition points, and based on the visual acquisition points, acquires perianal images in real time to obtain real-time perianal images; constructs a coordinate system from the real-time perianal images to obtain the perianal reference coordinate system, and binds real-world parameters to the perianal reference coordinate system; The lesion parameter processing and analysis module: judges the background area and lesion area of ​​the real-time perianal image to obtain the lesion confidence score; based on the lesion confidence score, performs binarization masking processing on the real-time perianal image to obtain the lesion mask image; according to the lesion mask image, statistically analyzes the lesion distribution to obtain lesion distribution information; and records the actual mapping position and parameters of the lesion in combination with the perianal reference coordinate system to obtain lesion parameter information. Effect evaluation and optimization module: Based on the lesion distribution information, the sampling effect of real-time perianal images is evaluated to obtain a real-time sampling evaluation score. At the same time, the center position of the lesion distribution is determined, and the visual acquisition point is adjusted according to the center position. Perianal images are also acquired to obtain adjusted perianal images. The sampling evaluation score of the adjusted perianal images is obtained, and the adjustment of the visual acquisition point is optimized by combining the real-time sampling evaluation score. Storage module: Collects perianal images from optimized visual sampling points, identifies and records lesions, stores them in conjunction with lesion parameter information, and transmits them to medical staff as data reference.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention uses an ultra-lightweight medical image segmentation model to accurately extract lesion pixels from perianal images, ensuring the effectiveness of target tracking. Based on lesion pixels, the lesions are iteratively divided to ensure the accuracy of lesion division. At the same time, the lesions are evaluated based on the lesion center point and range indicators to ensure the comprehensiveness and accuracy of lesion extraction from the overall perianal image. Furthermore, the range is analyzed in detail based on the pixels to avoid bounding box drift caused by large-scale non-rigid deformations such as compression and stretching of lesions.

[0018] 2. This invention is based on traversing the lesions and analyzing their relative positions on the overall image. Based on the relative positions, the optimal central target is determined. Using polar coordinates, the displacement distance and direction of the optimal central coordinates are accurately judged, which improves the accuracy and timeliness of target tracking, reduces the workload of doctors, and reduces the occurrence of missed diagnoses or misdiagnoses of patients.

[0019] 3. This invention associates images with real-world parameters, accurately reflects the image's performance in reality, accurately records the specific real-world parameters of lesions, provides accurate data references for medical staff, and ensures the safety and stability of medical procedures. Attached Figure Description

[0020] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the image processing of the present invention; Figure 3 This is a functional block diagram of the present invention; Detailed Implementation It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0022] This application provides a visual servoing-based method for tracking targets in perianal images. The execution entity of this visual servoing-based perianal image target tracking method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the visual servoing-based perianal image target tracking method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0023] Reference Figure 1The diagram shown is a flowchart illustrating a visual servoing-based perianal image target tracking method according to an embodiment of the present invention. In this embodiment, the visual servoing-based perianal image target tracking method includes: Step S1: Obtain visual acquisition points. Based on the visual acquisition points, acquire perianal images in real time to obtain real-time perianal images. Construct a coordinate system from the real-time perianal images to obtain the perianal reference coordinate system, and bind the real-world parameters to the perianal reference coordinate system. It should be noted that: the visual acquisition point refers to the relative position of the endoscope camera when acquiring images of the perianal area; the endoscope camera is moved by correcting the visual acquisition point; real-world parameters refer to the actual distance of the perianal images, etc.

[0024] Step S11: Obtain the acquisition range corresponding to the visual acquisition point, and record the actual boundary distance of the acquisition range as wlX×wlY; wlX represents the boundary distance in the horizontal direction of the acquisition range, and wlY represents the boundary distance in the vertical direction of the acquisition range. It should be noted that the acquisition range corresponding to the visual acquisition point refers to the range of the actual object acquired by acquisition devices such as endoscope cameras based on the acquisition point.

[0025] The pixel resolution of the real-time perianal image is acquired and denoted as us × vs; the acquisition parameters of the image acquisition device are obtained, including the acquisition height Z and the equivalent focal length F. x With F y Based on the acquisition height, equivalent focal length, and pixel resolution, the image display range zsX×zsY is calculated.

[0026] The scaling correction parameter xzc of the image is obtained by proportionally calculating the actual boundary distance wlX×wlY and the image display range zsX×zsY. ; It should be noted that the range of the image displayed is calculated by combining the pixel distance between the pixel boundary and the pixel center with camera parameters. The displayed range is then combined with the actual boundary distance to improve the accuracy of image distance judgment and prevent calculation errors caused by image scaling.

[0027] Step S12: Based on the pixel resolution us×vs of the real-time perianal image, obtain the pixel center (uzx, vzx), where uzx = us / 2 and vzx = vs / 2; construct a polar coordinate system with the pixel center as the origin; traverse the pixels (u, v) in the real-time perianal image, and record the mapping position of the pixels in the real-time perianal image on the polar coordinate system through the acquisition parameters of the image acquisition device to obtain the coordinate points (zjl, zjd); at the same time, associate the image scaling correction parameter xzc with the polar coordinate system to obtain the perianal reference coordinate system; It should be noted that polar coordinates provide a better representation of the range, and when adjusting the acquisition position of the image acquisition device, the angle and distance relationship shown by the polar coordinates reduces the computational complexity of the device adjustment.

[0028] Step S2: Determine the background area and lesion area of ​​the real-time perianal image to obtain the lesion confidence score; based on the lesion confidence score, perform binarization masking on the real-time perianal image to obtain the lesion mask image; based on the lesion mask image, statistically analyze the lesion distribution to obtain lesion distribution information; combine the perianal reference coordinate system to record the actual mapping position and parameters of the lesion to obtain lesion parameter information. Step S21: The pixel judgment values ​​of the real-time perianal image are collected using the ultra-lightweight medical image segmentation model MK-UNet. The pixel judgment values ​​are normalized to obtain the lesion confidence of each pixel. Based on the lesion confidence of each pixel, an adaptive lesion judgment threshold is set. The adaptive lesion judgment threshold is compared with the lesion confidence. The real-time perianal image is binarized to obtain the lesion mask image. It should be noted that MK-UNet is a pure CNN ultra-lightweight medical segmentation network, specifically designed for real-time segmentation scenarios in embedded medical devices.

[0029] Step S211: Transmit the real-time perianal image to the ultra-lightweight medical image segmentation model MK-UNet. The ultra-lightweight medical image segmentation model MK-UNet processes the real-time perianal image to obtain the pixel judgment value of the real-time perianal image. It should be noted that the above processing refers to color correction, bilateral filtering for noise reduction, and normalization to [0,1] of real-time perianal images based on MK-UNet, which is then converted into tensors [B,3,H,W] and input into the network. The encoder compresses the resolution and increases the number of channels through MaxPool stepwise, outputting multi-scale features, with shallow layers storing lesion edges and deep layers storing global lesion semantics. The features of each encoder layer are filtered for background noise by GAG and then fed into the corresponding decoder layer for fusion. The resolution is restored by bilinear interpolation, and the MKIRA module purifies lesion features and fills feature holes caused by occlusion.

[0030] The pixel judgment values ​​of real-time perianal images are statistically analyzed and denoted as xsp(u,v); xsp(u,v) represents the pixel judgment value at pixel (u,v), where (u,v) is an actual pixel in us×vs; the pixel judgment value xsp(u,v) is normalized to obtain the lesion confidence of the pixel zxd(u,v); ; It should be noted that normalizing pixel judgment values ​​through exponents enhances the contrast between different pixel judgment values ​​and improves the data accuracy of lesion confidence.

[0031] Step S212: Calculate based on lesion confidence zxd(u,v) and set an adaptive lesion judgment threshold pdy; ; It should be noted that by setting an adaptive lesion judgment threshold, the limitations of fixed thresholds in making comprehensive and effective judgments about the differences in clinical manifestations of lesions under different diseases can be prevented, thereby improving the accuracy and comprehensiveness of lesion judgment. The adaptive lesion judgment threshold pdy and the lesion confidence score zxd(u, v) are compared by iterative process: If pdy≥zxd(u,v), it indicates that the pixel (u,v) corresponding to the lesion confidence zxd(u,v) is a low-confidence background, and the pixel value xsz(u,v) at pixel (u,v) is assigned to 0; If pdy < zxd(u,v), it indicates that the pixel (u,v) corresponding to the lesion confidence zxd(u,v) is a high-confidence lesion, and the pixel value xsz(u,v) at pixel (u,v) is assigned the value of 255; After traversing all pixels and their corresponding pixel values, a binary image of the real-time perianal image is obtained, which is denoted as the lesion mask image.

[0032] Please see Figure 2 Step S22: Divide the lesions according to the lesion mask image; traverse each lesion, iterate over the preset lesion center point, collect the distance from the lesion center point to the lesion boundary, and obtain the lesion range value; record the distribution of lesions based on the lesion center point and the lesion range value; map the lesion center point to the perianal reference coordinate system according to the lesion distribution, combine with real-world parameters, record the actual mapped position of the lesion, extract the boundary pixels within the lesion range through the lesion range value, and record the actual parameters of the lesion; obtain the lesion parameter information from the actual mapped position and actual parameters of the lesion. Step S221: Perform continuity judgment on the lesion mask image. If the pixel value of pixel (u, v) in the lesion mask image is 255, extract the pixel values ​​of its neighboring pixels, and merge pixels with the same pixel value to form a set of local lesions. It should be noted that: neighboring pixels refer to pixels that are adjacent to each other in different directions. For example, the neighboring pixels of (u, v) are (u+1, v), (u-1, v), (u, v+1), and (u, v-1). The local lesion set is statistically analyzed, and the lesion mask image is divided into multiple lesions. Each local lesion set is traversed and iterated to analyze the lesion center point. The pixel positions of the elements in the local lesion set are integrated with the lesion center point, and the distance from the lesion center point to the lesion boundary is recorded to obtain the lesion range value. The local lesions are marked in combination with the lesion center point, and the distribution of the lesions is recorded. Step S2211: Obtain the boundary elements in the local lesion set, and collect the distribution extreme values ​​based on the boundary elements. The distribution extreme values ​​include maxu, maxv, minu, and minv. The distribution extreme values ​​are used to preset the lesion center point, and the distribution center point is denoted as fbz(u,v), fbz(u,v) = ((maxu + minu) / 2, (maxv + minv) / 2). It should be noted that: boundary elements refer to the elements corresponding to the maximum and minimum values ​​of u and v in the set of local lesions (u, v); Step S2212: Collect the distances between the distribution center point and each boundary element. Use the maximum distance between the distribution center point and each boundary element as the division radius. Record the distribution center point and the division radius as the initial iteration data. Based on the distribution center point and the division radius, divide the local lesion set into two parts. Obtain the distribution center point and division radius of each part and continue iterating. Record the number of distribution center points and the division radius of each part. Evaluate the iteration results and obtain the iteration evaluation value dpg. ; It should be noted that q1 and q2 are weights, and q1 + q2 = 1. The data values ​​are adjusted through the weights. fsl is the number of distribution center points, and hfb is the division radius. The formula is used to sum the areas of all parts by radius to effectively judge the redundancy and irrelevant regions. The larger the redundancy and irrelevant regions, the larger the evaluation value, indicating a worse iteration effect. At the same time, the more numerous they are, the more limited the iteration region is, reflecting poor iteration effect.

[0033] The iterative result with the smallest iterative evaluation value is collected, the center point of the lesion and its corresponding division radius are recorded, the lesion range value corresponding to the center point of the lesion is obtained from the division radius, and the center point of the lesion and its corresponding lesion range value are statistically stored to obtain the distribution of the lesions.

[0034] Step S222: Extract the center point of each lesion according to the distribution of lesions, map the center point of the lesion to the perianal reference coordinate system, and correct the distance value by combining real-world parameters to obtain the real-world mapped position of the lesion. The boundary pixels within the lesion range corresponding to the center point of each lesion are extracted, and the lesion parameters (including length, width, side length, area, etc.) are recorded. These parameters are then converted into real-world parameters and recorded.

[0035] Step S3: Based on the lesion distribution information, evaluate the sampling effect of the real-time perianal image to obtain a real-time sampling evaluation score. At the same time, determine the center position of the lesion distribution, adjust the visual acquisition point according to the center position, and acquire the perianal image to obtain the adjusted perianal image. Obtain the sampling evaluation score of the adjusted perianal image, and optimize the adjustment of the visual acquisition point by combining the real-time sampling evaluation score. Step S31: Obtain the center point and range value of the lesion from the lesion distribution information; map the center point of the lesion to the perianal reference coordinate system to obtain the mapped coordinates; judge the sampling deviation of the lesion based on the mapped coordinates; calculate the lesion sampling evaluation value by combining the lesion range value; traverse all lesions in the real-time perianal image and integrate their corresponding lesion sampling evaluation values ​​to obtain the real-time sampling evaluation score; traverse the mapped coordinates of all lesion center points and calculate the center position of the lesion distribution to obtain the distribution center. Step S311: Map the center point of the lesion to the perianal reference coordinate system to obtain the mapped coordinates ysz(zjl, zjd); count the number of lesion center points and denote them as bs; map all lesion center points and record their corresponding mapped coordinates ysz. b (zjl, zjd), ysz b (zjl, zjd) represents the mapped coordinates corresponding to the center point of the b-th lesion; the lesion range value corresponding to the center point of each lesion is collected and denoted as bzf. b ; the mapping coordinates ysz corresponding to the center point of the lesion b (zjl, zjd) and the lesion extent value bzf b The sampling effect of real-time perianal images is evaluated to obtain the real-time sampling evaluation score cpg; ; Among them ysz b(zjl) represents the coordinates mapped from ysz. b (zjl, zjd) represents the distance between the center point of the lesion and the origin. The greater the distance between the center point of the lesion and the origin, the greater the deviation, and the smaller the score. The lesion range value is bzf. b This is equivalent to weighting; the larger the lesion range value, the greater the impact of the deviation of its center point. Step S312: Obtain the mapped coordinates ysz1(zjl, zjd) and ysz2(zjl, zjd) of the center points of the two lesions, and their corresponding lesion range values ​​bzf1 and bzf2. Use the lesion range values ​​as weights to extract the midpoint coordinates of the mapped coordinates ysz1(zjl, zjd) and ysz2(zjl, zjd) to obtain the midpoint coordinates zzb(zjl, zjd). The mapping coordinates of all lesion center points are traversed, the mapping coordinates of the lesion center points and the midpoint coordinates are extracted and calculated, and the midpoint coordinates are modified and iterated to obtain the center position of the lesion distribution. Step S32: Adjust the visual acquisition points according to the distribution center, acquire the perianal image of the adjusted visual acquisition points, and obtain the adjusted perianal image; acquire the lesion distribution information of the adjusted perianal image, analyze the sampling effect of the adjusted perianal image, obtain the adjustment sampling evaluation score, and judge the adjustment result of the visual acquisition points in combination with the real-time sampling evaluation score, and perform feedback optimization. Step S321: Based on the distribution center, acquire the coordinate data of the center position. Based on the coordinate data of the center position, obtain the adjustment direction and adjustment distance of the visual acquisition point. Based on the adjustment direction and adjustment distance of the visual acquisition point, move the image acquisition device and acquire the perianal image of the corresponding position after the movement to obtain the adjusted perianal image. Step S322: Based on the real-time perianal image lesion distribution information acquisition step, the lesion distribution information of the perianal image is acquired, and the sampling effect of the perianal image is evaluated and calculated by combining the calculation formula of the real-time sampling evaluation score to obtain the control sampling evaluation score; the control result of the visual acquisition point is judged by the control sampling evaluation score and the real-time sampling evaluation score. If the control sampling evaluation score is lower than the real-time sampling evaluation score, it indicates that the control is abnormal, and the abnormality is transmitted for manual verification; if the control sampling evaluation score is higher than the real-time sampling evaluation score, it indicates that the control is normal and the target tracking of the lesion is achieved.

[0036] Step S4: Collect perianal images using optimized visual sampling points, identify and record lesions, store them in conjunction with lesion parameter information, and transmit them to medical staff as data reference.

[0037] Compared to the problems described in the background technology, this invention accurately extracts lesion pixels from perianal images using an ultra-lightweight medical image segmentation model, ensuring the effectiveness of target tracking. Based on lesion pixels, iterative segmentation of lesions ensures accurate segmentation. Simultaneously, lesions are evaluated based on their center point and range indicators, ensuring comprehensive and accurate extraction of lesions from the overall perianal image. Detailed analysis of the range based on pixel points avoids bounding box drift caused by large-scale compression, stretching, or other non-rigid deformations of the lesions. Furthermore, this invention traverses the lesions, analyzes their relative positions on the overall image, determines the optimal center target based on these relative positions, and accurately judges the displacement distance and direction of the optimal center coordinates using polar coordinates, improving the accuracy and timeliness of target tracking, reducing the workload of doctors, and minimizing missed diagnoses or misdiagnoses. Finally, this invention correlates images with real-world parameters, accurately reflecting the image's performance in reality and accurately recording the specific real-world parameters of the lesions, providing accurate data references for medical personnel and ensuring the safety and stability of medical procedures. Therefore, the visual servoing-based target tracking method for perianal images provided in this embodiment of the invention can improve the accuracy of target tracking in perianal images.

[0038] like Figure 3 The diagram shown is a functional block diagram of a visual servoing-based perianal image target tracking system according to the present invention.

[0039] The visual servoing-based perianal image target tracking system described in this invention can be installed in an electronic device. Depending on the functions implemented, the visual servoing-based perianal image target tracking system may include a data acquisition module, a lesion parameter processing and analysis module, an effect evaluation and optimization module, and a storage module. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0040] In this embodiment of the invention, the functions of each module / unit are as follows: The data acquisition module is used to acquire visual acquisition points, and based on the visual acquisition points, to acquire perianal images in real time to obtain real-time perianal images; a coordinate system is constructed from the real-time perianal images to obtain a perianal reference coordinate system, and real-world parameters are bound to the perianal reference coordinate system; The lesion parameter processing and analysis module is used to determine the background area and lesion area of ​​the real-time perianal image to obtain the lesion confidence score; based on the lesion confidence score, the real-time perianal image is subjected to binarization masking to obtain the lesion mask image; based on the lesion mask image, the lesion distribution is statistically analyzed to obtain the lesion distribution information; and combined with the perianal reference coordinate system, the actual mapping position and parameters of the lesion are recorded to obtain the lesion parameter information. The effect evaluation and optimization module is used to evaluate the sampling effect of real-time perianal images based on lesion distribution information, obtain a real-time sampling evaluation score, determine the center position of lesion distribution, adjust the visual acquisition point based on the center position, and acquire perianal images to obtain adjusted perianal images; obtain the sampling evaluation score of the adjusted perianal images, and optimize the adjustment of visual acquisition points based on the real-time sampling evaluation score; The storage module is used to collect perianal images from optimized visual sampling points, identify and record lesions, and store them in conjunction with lesion parameter information, transmitting them to medical staff as data references.

[0041] In detail, the modules in the visual servoing-based perianal image target tracking system described in this embodiment of the invention employ the same methods as described above. Figure 1 The method uses the same technique as the visual servoing-based perianal image target tracking method described above, and can produce the same technical effect, so it will not be repeated here.

[0042] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0043] Finally, it should be noted that deleting any one of the above embodiments does not affect the technical solutions of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for tracking perianal images based on visual servoing, characterized in that, include: Step S1: Obtain visual acquisition points and acquire perianal images in real time to obtain real-time perianal images; A coordinate system is constructed from real-time perianal images to obtain a perianal reference coordinate system, and real-world parameters are bound to the perianal reference coordinate system. Step S2: Determine the background area and lesion area of ​​the real-time perianal image to obtain the lesion confidence score; based on the lesion confidence score, perform binarization masking on the real-time perianal image to obtain the lesion mask image; Based on the lesion mask image, the distribution of lesions is statistically analyzed to obtain lesion distribution information. Combined with the perianal reference coordinate system, the actual mapping position and parameters of the lesions are recorded to obtain lesion parameter information. Step S3: Based on the lesion distribution information, evaluate the sampling effect of the real-time perianal image to obtain a real-time sampling evaluation score. At the same time, determine the center position of the lesion distribution, adjust the visual acquisition point according to the center position, and acquire the perianal image to obtain the adjusted perianal image. Obtain the sampling evaluation score of the adjusted perianal image, and optimize the adjustment of the visual acquisition point by combining the real-time sampling evaluation score. Step S4: Collect perianal images using the optimized visual sampling points and store them in conjunction with lesion parameter information.

2. The visual servoing-based perianal image target tracking method according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S11: Obtain the acquisition range corresponding to the visual acquisition point, record the actual boundary distance of the acquisition range, and record the actual boundary distance as wlX×wlY; The pixel resolution of the real-time perianal image is acquired and denoted as us × vs; the acquisition parameters of the image acquisition device are obtained, including the acquisition height Z and the equivalent focal length F. x With F y Based on the acquisition height, equivalent focal length, and pixel resolution, the image display range zsX×zsY is calculated. The scaling correction parameter xzc of the image is obtained by proportionally calculating the actual boundary distance wlX×wlY and the image display range zsX×zsY. Step S12: Based on the pixel resolution us×vs of the real-time perianal image, obtain the pixel center (uzx, vzx), where uzx = us / 2 and vzx = vs / 2; construct a polar coordinate system with the pixel center as the origin; traverse the pixels (u, v) in the real-time perianal image, and record the mapping position of the pixels in the real-time perianal image on the polar coordinate system through the acquisition parameters of the image acquisition device to obtain the coordinate points (zjl, zjd); at the same time, associate the image scaling correction parameter xzc with the polar coordinate system to obtain the perianal reference coordinate system.

3. The method for tracking perianal images based on visual servoing according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S21: Collect pixel judgment values ​​of real-time perianal images, normalize the pixel judgment values ​​to obtain the lesion confidence of each pixel, set an adaptive lesion judgment threshold based on the lesion confidence of each pixel, compare the adaptive lesion judgment threshold with the lesion confidence, binarize the real-time perianal images to obtain lesion mask images. Step S22: Divide the lesions according to the lesion mask image; traverse each lesion, iterate over the preset lesion center point, collect the distance from the lesion center point to the lesion boundary to obtain the lesion range value; record the distribution of lesions based on the lesion center point and the lesion range value; map the lesion center point to the perianal reference coordinate system according to the lesion distribution, combine with real-world parameters to record the actual mapped position of the lesion, extract the boundary pixels within the lesion range through the lesion range value, and record the actual parameters of the lesion; obtain the lesion parameter information from the actual mapped position and actual parameters of the lesion.

4. The visual servoing-based perianal image target tracking method according to claim 3, characterized in that, The specific steps of step S21 are as follows: Step S211: Transmit the real-time perianal image to the ultra-lightweight medical image segmentation model, process the real-time perianal image, and obtain the pixel judgment value of the real-time perianal image; The pixel judgment values ​​of real-time perianal images are statistically analyzed and denoted as xsp(u,v); the pixel judgment values ​​xsp(u,v) are normalized to obtain the lesion confidence of the pixel zxd(u,v); Step S212: Calculate based on lesion confidence zxd(u,v) and set an adaptive lesion judgment threshold pdy; The adaptive lesion judgment threshold pdy and the lesion confidence score zxd(u, v) are compared by iterative process: If pdy≥zxd(u,v), it indicates that the pixel (u,v) corresponding to the lesion confidence zxd(u,v) is a low-confidence background, and the pixel value xsz(u,v) at pixel (u,v) is assigned to 0; If pdy < zxd(u,v), it indicates that the pixel (u,v) corresponding to the lesion confidence zxd(u,v) is a high-confidence lesion, and the pixel value xsz(u,v) at pixel (u,v) is assigned the value of 255; After traversing all pixels and their corresponding pixel values, a binary image of the real-time perianal image is obtained, which is denoted as the lesion mask image.

5. The visual servoing-based perianal image target tracking method according to claim 3, characterized in that, The specific steps of step S22 are as follows: Step S221: Perform continuity judgment on the lesion mask image. If the pixel value of pixel (u, v) in the lesion mask image is 255, extract the pixel values ​​of its neighboring pixels, and merge pixels with the same pixel value to form a set of local lesions. The local lesion set is statistically analyzed, and the lesion mask image is divided into multiple lesions. Each local lesion set is traversed and iterated to analyze the lesion center point. The pixel positions of the elements in the local lesion set are integrated with the lesion center point, and the distance from the lesion center point to the lesion boundary is recorded to obtain the lesion range value. The local lesions are marked in combination with the lesion center point, and the distribution of the lesions is recorded. Step S222: Extract the center point of each lesion according to the distribution of lesions, map the center point of the lesion to the perianal reference coordinate system, and correct the distance value by combining real-world parameters to obtain the real-world mapped position of the lesion. The boundary pixels within the lesion range corresponding to the center point of each lesion are extracted, the lesion parameters are recorded, and the real-world parameters are combined with the real-world parameters to convert them into real-world parameters for recording.

6. The visual servoing-based perianal image target tracking method according to claim 5, characterized in that, The specific steps of step S221 are as follows: Obtain the boundary elements in the set of local lesions, and collect the distribution extreme values ​​based on the boundary elements; preset the lesion center point based on the distribution extreme values, and denote the distribution center point as fbz(u,v); The distances between the distribution center point and each boundary element are collected. The maximum distance between the distribution center point and each boundary element is used as the division radius. The distribution center point and the division radius are recorded as the initial iteration data. Based on the distribution center point and the division radius, the local lesion set is divided into two parts. The distribution center point and the division radius of each part are obtained, and the iteration continues. The number of distribution center points and the division radius of each part are recorded. The iteration results are evaluated to obtain the iteration evaluation value dpg. The iterative result with the smallest iterative evaluation value is collected, the center point of the lesion and its corresponding division radius are recorded, the lesion range value corresponding to the center point of the lesion is obtained from the division radius, and the center point of the lesion and its corresponding lesion range value are statistically stored to obtain the distribution of the lesions.

7. The visual servoing-based perianal image target tracking method according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S31: Obtain the center point and range value of the lesion from the lesion distribution information, map the center point of the lesion to the perianal reference coordinate system to obtain the mapped coordinates, judge the sampling deviation of the lesion from the mapped coordinates, and calculate the lesion sampling evaluation value in combination with the lesion range value. All lesions in the real-time perianal images are traversed, and their corresponding lesion sampling evaluation values ​​are integrated to obtain the real-time sampling evaluation score; the mapping coordinates of the center points of all lesions are traversed, and the center position of the lesion distribution is calculated to obtain the distribution center; Step S32: Adjust the visual acquisition points according to the distribution center, and acquire the perianal image of the adjusted visual acquisition points to obtain the adjusted perianal image; Information on the distribution of lesions in the perianal images was acquired, the sampling effect of the perianal images was analyzed, a sampling evaluation score was obtained, and the results of the control of the visual acquisition points were judged and optimized by combining the real-time sampling evaluation score.

8. The visual servoing-based perianal image target tracking method according to claim 7, characterized in that, The specific steps of step S31 are as follows: Step S311: Map the center point of the lesion to the perianal reference coordinate system to obtain the mapped coordinates ysz(zjl, zjd); count the number of lesion center points and denote them as bs; map all lesion center points and record their corresponding mapped coordinates ysz. b (zjl, zjd); Collect the lesion range value corresponding to the center point of each lesion, denoted as bzf. b ; the mapping coordinates ysz corresponding to the center point of the lesion b (zjl, zjd) and the lesion extent value bzf b The sampling effect of real-time perianal images is evaluated to obtain the real-time sampling evaluation score cpg; Step S312: Obtain the mapped coordinates ysz1(zjl, zjd) and ysz2(zjl, zjd) of the center points of the two lesions, and their corresponding lesion range values ​​bzf1 and bzf2. Use the lesion range values ​​as weights to extract the midpoint coordinates of the mapped coordinates ysz1(zjl, zjd) and ysz2(zjl, zjd) to obtain the midpoint coordinates zzb(zjl, zjd). The mapped coordinates of all lesion center points are traversed, the mapped coordinates of the lesion center points and the midpoint coordinates are extracted and calculated, and the midpoint coordinates are modified and iterated to obtain the center position of the lesion distribution.

9. A visual servoing-based perianal image target tracking method according to claim 7, characterized in that, The specific steps of step S32 are as follows: Step S321: Based on the distribution center, acquire the coordinate data of the center position. Based on the coordinate data of the center position, obtain the adjustment direction and adjustment distance of the visual acquisition point. Based on the adjustment direction and adjustment distance of the visual acquisition point, move the image acquisition device and acquire the perianal image of the corresponding position after the movement to obtain the adjusted perianal image. Step S322: Based on the lesion distribution information acquisition step of real-time perianal images, the lesion distribution information of the perianal images is acquired, and the sampling effect of the perianal images is evaluated and calculated by combining the calculation formula of real-time sampling evaluation score to obtain the control sampling evaluation score; the control result of the visual acquisition point is judged by the control sampling evaluation score and the real-time sampling evaluation score. If the control sampling evaluation score is lower than the real-time sampling evaluation score, it indicates that the control is abnormal and manual verification is performed. If the controlled sampling assessment score is higher than the real-time sampling assessment score, it indicates that the control is normal and the target tracking of the lesion has been achieved.

10. A visual servoing-based perianal image target tracking system, applicable to the visual servoing-based perianal image target tracking method described in any one of claims 1-9, characterized in that, include: Data acquisition module: acquires visual acquisition points, and based on the visual acquisition points, performs real-time acquisition of perianal images to obtain real-time perianal images; A coordinate system is constructed from real-time perianal images to obtain a perianal reference coordinate system, and real-world parameters are bound to the perianal reference coordinate system. Lesion parameter processing and analysis module: It judges the background area and lesion area of ​​the real-time perianal image to obtain the lesion confidence score; based on the lesion confidence score, it performs binarization masking processing on the real-time perianal image to obtain the lesion mask image; Based on the lesion mask image, the distribution of lesions is statistically analyzed to obtain lesion distribution information. Combined with the perianal reference coordinate system, the actual mapping position and parameters of the lesions are recorded to obtain lesion parameter information. Effect evaluation and optimization module: Based on the lesion distribution information, the sampling effect of real-time perianal images is evaluated to obtain a real-time sampling evaluation score. At the same time, the center position of the lesion distribution is determined, and the visual acquisition point is adjusted according to the center position. Perianal images are also acquired to obtain adjusted perianal images. The sampling evaluation score of the adjusted perianal images is obtained, and the adjustment of the visual acquisition point is optimized by combining the real-time sampling evaluation score. Storage module: Collects perianal images from optimized visual sampling points, identifies and records lesions, stores them in conjunction with lesion parameter information, and transmits them to medical staff as data reference.

Images