Visual tracking line identification positioning method, device, equipment and medium

Abstract

The application relates to the field of medical robots, in particular to a visual tracking line identification and positioning method, device, equipment and medium. The method identifies an original image containing a tracking line, obtains a pretreatment image containing a tracking line contour, performs morphological operation on the pretreatment image, determines abnormal pixel points in the pretreatment image, reverses the pixel value of the abnormal pixel points, determines the pretreatment image after the reversal as a target image, determines the coordinates of the leftmost and rightmost pixel points of the tracking line contour on each horizontal pixel row of the target image, takes the mean value of the coordinates of the leftmost and rightmost pixel points as a positioning point corresponding to the horizontal pixel row of the tracking line contour, performs straight line fitting on all the positioning points, obtains a fitted tracking line, indicates the center point of the medical robot to move along the fitted tracking line, and accurately gives the tracking line and the tracking line result through the above process, so that the medical robot moves along the tracking line, and the control difficulty of the movement of the medical robot is reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and medium for visual tracking line recognition and positioning. Background Technology

[0002] With the development of digital services, medical robots are being deployed in hospitals to guide patients through medical procedures and even help them pick up medications and queue. This requires medical robots to have accurate identification and positioning capabilities.

[0003] Currently, for medical robot tracking tasks, designers need to consider different sensors to detect the position of the tracking line based on the working environment, and then control the movement of the medical robot. Existing common detection methods are infrared detection and visual detection. Infrared detection is simple and sensitive, but it is greatly affected by the color of the light, and it cannot be used underwater, which imposes many limitations. Visual detection requires algorithm design to obtain relevant information about the tracking line from the image. Existing methods fit the obtained tracking line to a straight line in the image determined by the slope and intercept. When using PID algorithms to control the movement, this method also requires separate position correction for the slope and intercept, and the two interfere with each other, causing instability. Therefore, how to improve the accuracy of tracking line recognition to reduce the difficulty of controlling the movement of medical robots has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, device, and medium for visual tracking line recognition and positioning to solve the problem of how to improve the accuracy of tracking line recognition and reduce the difficulty of controlling the movement of medical robots.

[0005] In a first aspect, embodiments of this application provide a method for identifying and locating visual tracking lines, the method comprising:

[0006] The original image containing the tracking line is subjected to tracking line contour recognition to obtain a preprocessed image containing the tracking line contour.

[0007] Perform morphological operations on the preprocessed image to identify abnormal pixels in the preprocessed image, invert the pixel values ​​of the abnormal pixels, and determine the inverted preprocessed image as the target image.

[0008] In the coordinate system of the target image, determine the coordinates of the leftmost pixel and the rightmost pixel of the tracking line contour on each horizontal pixel row of the target image;

[0009] For any horizontal pixel row, the average of the coordinates of the leftmost pixel and the rightmost pixel is used as the positioning point of the tracking line contour in the horizontal pixel row, thus obtaining all the positioning points of the tracking line contour.

[0010] A straight line is fitted to all positioning points to obtain a fitted tracking line, which is used to indicate the movement of the center point of the medical robot along the fitted tracking line.

[0011] In one embodiment, after performing straight-line fitting on all positioning points to obtain the fitted tracking line, the method further includes:

[0012] Based on the installation information of the acquisition device that acquired the original image on the medical robot, the positional relationship between the center point of the medical robot and the fitted tracking line is determined.

[0013] Based on the positional relationship between the center point and the midpoint of the front wheel drive shaft, and the positional relationship between the center point and the midpoint of the rear wheel drive shaft, determine the positional relationship between the midpoint of the front wheel drive shaft and the midpoint of the rear wheel drive shaft and the fitted tracking line, respectively.

[0014] With the midpoints of the front and rear drive shafts simultaneously on the fitted tracking line as the target, the steering angles of the front and rear wheels of the medical robot are controlled when it moves forward.

[0015] In one embodiment, performing line contour recognition on the acquired original image containing the line to obtain a preprocessed image containing the line contour includes:

[0016] The original image containing the tracking line is filtered to obtain the filtered original image;

[0017] The filtered original image is binarized to obtain a preprocessed image containing the contour of the tracking line.

[0018] In one embodiment, filtering the acquired original image containing the tracking line to obtain a filtered original image includes:

[0019] The original image containing the tracking line is extracted using a preset channel to obtain a single-channel original image;

[0020] The original image of the single channel is subjected to median filtering to obtain the filtered original image.

[0021] In one embodiment, performing morphological operations on the preprocessed image to determine abnormal pixels in the preprocessed image includes:

[0022] The preprocessed image is dilated to obtain a dilated image, and the dilated image is eroded to obtain an eroded image;

[0023] Extract the pixel value of each pixel in the eroded image, and determine the pixels with pixel values ​​greater than a threshold as abnormal pixels in the preprocessed image.

[0024] In one embodiment, inverting the pixel values ​​of the abnormal pixels and determining the inverted preprocessed image as the target image includes:

[0025] In the preprocessed image, the pixel values ​​of the abnormal pixels are set to zero, and the preprocessed image after being set to zero is determined to be the target image.

[0026] In one embodiment, a straight line is fitted to all positioning points to obtain the fitted tracking line, including:

[0027] Based on the forward direction of the medical robot, the fitting weights of all positioning points are assigned. Among them, all positioning points are perpendicular to the forward direction, and the positioning point corresponding to the farthest perpendicular point in the forward direction has the highest fitting weight.

[0028] Using the corresponding fitting weights, perform linear weighted fitting on all positioning points to determine the fitted line as the fitted tracking line.

[0029] Secondly, embodiments of this application provide a visual tracking line recognition and positioning device, the recognition and positioning device comprising:

[0030] The preprocessing module is used to perform line contour recognition on the acquired original image containing the line, and obtain a preprocessed image containing the line contour.

[0031] The target image determination module is used to perform morphological operations on the preprocessed image, determine abnormal pixels in the preprocessed image, invert the pixel values ​​of the abnormal pixels, and determine the inverted preprocessed image as the target image.

[0032] The coordinate determination module is used to determine the coordinates of the leftmost pixel and the rightmost pixel of the tracking line contour on each horizontal pixel row of the target image in the coordinate system of the target image.

[0033] The positioning point determination module is used to, for any horizontal pixel row, take the average of the coordinates of the leftmost pixel and the rightmost pixel as the positioning point of the tracking line contour in the horizontal pixel row, and obtain all the positioning points of the tracking line contour.

[0034] The tracking line determination module is used to perform straight line fitting on all positioning points to obtain the fitted tracking line, which is used to indicate the movement of the center point of the medical robot along the fitted tracking line.

[0035] In one embodiment, the identification and positioning device further includes:

[0036] The first positional relationship determination module is used to determine the positional relationship between the center point of the medical robot and the fitted tracking line after performing straight line fitting on all positioning points to obtain the fitted tracking line, based on the installation information of the acquisition device that acquired the original image on the medical robot.

[0037] The second position relationship determination module is used to determine the position relationship between the midpoint of the front wheel drive shaft and the midpoint of the rear wheel drive shaft and the fitted tracking line, respectively, based on the position relationship between the center point and the midpoint of the front wheel drive shaft and the position relationship between the center point and the midpoint of the rear wheel drive shaft.

[0038] The control module is used to control the steering angle of the front and rear wheels of the medical robot when it moves forward, with the midpoints of the front and rear drive shafts simultaneously on the fitted tracking line as the target.

[0039] In one embodiment, the preprocessing module includes:

[0040] The filtering unit is used to filter the acquired original image containing the tracking line to obtain the filtered original image.

[0041] The binarization processing unit is used to perform binarization processing on the filtered original image to obtain a preprocessed image containing the contour of the tracking line.

[0042] In one embodiment, the filtering unit includes:

[0043] The channel filtering subunit is used to extract the acquired original image containing the tracking line using a preset channel to obtain the original image of a single channel.

[0044] The median filtering subunit is used to perform median filtering on the original image of the single channel to obtain the filtered original image.

[0045] In one embodiment, the target image determination module includes:

[0046] A morphological operation unit is used to dilate the preprocessed image to obtain a dilated image, and to erode the dilated image to obtain an eroded image;

[0047] An anomaly point determination unit is used to extract the pixel value of each pixel in the eroded image and determine the pixel points with pixel values ​​greater than a threshold as anomaly pixels in the preprocessed image.

[0048] In one embodiment, the target image determination module includes:

[0049] The target image determination unit is used to set the pixel value of the abnormal pixel point to zero in the preprocessed image, and determine the preprocessed image after setting the pixel value to zero as the target image.

[0050] In one embodiment, the tracking line determination module includes:

[0051] The weight assignment unit is used to assign fitting weights to all positioning points according to the forward direction of the medical robot. The positioning point with the highest fitting weight is the one that is perpendicular to the forward direction when all positioning points are perpendicular to the forward direction.

[0052] The tracking line determination unit is used to perform linear weighted fitting on all positioning points using the corresponding fitting weights, and determine the fitted line as the fitted tracking line.

[0053] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the identification and positioning method as described in the first aspect.

[0054] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the identification and positioning method as described in the first aspect.

[0055] The beneficial effects of this application embodiment compared with the prior art are as follows: This application performs tracking line contour recognition on the acquired original image containing tracking lines to obtain a preprocessed image containing tracking line contours. Morphological operations are performed on the preprocessed image to identify abnormal pixels in the preprocessed image. The pixel values ​​of the abnormal pixels are inverted, and the inverted preprocessed image is determined as the target image. In the coordinate system of the target image, the coordinates of the leftmost and rightmost pixels of the tracking line contour on each horizontal pixel row of the target image are determined. For any horizontal pixel row, the average of the coordinates of the leftmost and rightmost pixels is used as the positioning point of the tracking line contour in the horizontal pixel row, thus obtaining all positioning points of the tracking line contour. Straight line fitting is performed on all positioning points to obtain the fitted tracking line. The fitted tracking line is used to indicate the movement of the center point of the medical robot along the fitted tracking line. Through the above process, the medical robot accurately identifies and locates the tracking line and obtains an accurate tracking line result, thereby controlling the medical robot to move along the tracking line and reducing the difficulty of controlling the movement of the medical robot. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of an application environment for a visual tracking line recognition and positioning method provided in Embodiment 1 of this application;

[0058] Figure 2 This is a flowchart illustrating a visual tracking line recognition and positioning method provided in Embodiment 2 of this application;

[0059] Figure 3 This is a flowchart illustrating a visual tracking line recognition and positioning method provided in Embodiment 3 of this application;

[0060] Figure 4 This is a schematic diagram of the structure of a visual tracking line recognition and positioning device provided in Embodiment 4 of this application;

[0061] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 5 of this application. Detailed Implementation

[0062] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0063] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0064] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0065] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0066] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0067] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0068] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0069] Foundational artificial intelligence technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, medical robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0070] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0071] To illustrate the technical solution of this application, specific embodiments are described below.

[0072] The visual tracking line recognition and positioning method provided in Embodiment 1 of this application can be applied to, for example, Figure 1 In this application environment, the client and server communicate to form a cloud-edge joint system. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, server-side computer equipment, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0073] See Figure 2 This is a flowchart illustrating a visual tracking line recognition and localization method provided in Embodiment 2 of this application. The aforementioned visual tracking line recognition and localization method is applied to... Figure 1 The client in this context can be a computer device that connects to the medical robot to control its movement. This computer device can be mounted on the medical robot or set up independently and communicate wirelessly or via a wired connection. The medical robot is equipped with corresponding image acquisition devices to obtain images of its surroundings. For example... Figure 2 As shown, the visual tracking line recognition and localization method may include the following steps:

[0074] Step S201: Perform line contour recognition on the acquired original image containing the line to obtain a preprocessed image containing the line contour.

[0075] In this application, the tracking line is a line that guides the movement of the medical robot. For example, if the medical robot moves on a road, the tracking line can be a line drawn on the road with a certain color or shape. Or, if the medical robot moves underwater, the tracking line can be a light-emitting cable laid in the water.

[0076] The original image can be acquired by an image acquisition device on the medical robot. This device can be installed at any position on the medical robot, but the positional relationship between the image acquisition device and the center of motion of the medical robot must be known. This information is then used to map the position of the center of motion of the medical robot to the position in the original image through coordinate transformation or other means. For example, for a medical robot moving on a road, lines are laid on the road surface, and an image acquisition device is placed in the direction of the medical robot's movement to acquire an image of the tracking lines ahead.

[0077] The contour of a tracking line can refer to the outline of the line corresponding to the tracking line in the original image. The purpose of contour recognition is to remove other irrelevant content in the original image and retain only the tracking line as much as possible, in order to prepare for accurate identification and localization of the tracking line in the future.

[0078] Optionally, the acquired original image containing the tracking lines is subjected to tracking line contour recognition to obtain a preprocessed image containing the tracking line contours, including:

[0079] The original image containing the tracking line is filtered to obtain the filtered original image;

[0080] The filtered original image is binarized to obtain a preprocessed image containing the contour of the tracking line.

[0081] The process begins with filtering the original image. Filtering techniques, such as mean filtering, Gaussian filtering, and median filtering, are used to filter the image. Other methods include box filtering, bilateral filtering, and separable filtering. Mean filtering uses the average of multiple measurements, replacing the value of each pixel with the average of its surrounding pixels. The specific value can be specified. It can repair pixels affected by noise by using the values ​​of surrounding pixels, but this reduces the differences between pixels, resulting in a blurred image. Gaussian filtering is mainly used to remove Gaussian noise. Median filtering sorts the surrounding pixels of the pixel to be processed from smallest to largest and replaces the pixel with its median value. This effectively removes salt-and-pepper noise, but when the convolution kernel is large, it still blurs the image and is computationally intensive.

[0082] Image binarization is the process of converting the grayscale values ​​of points in an image to 0 or 255, effectively presenting the entire image in a black and white manner. It involves obtaining a binarized image from a grayscale image with 256 brightness levels by applying an appropriate threshold, while still retaining the overall and local features of the image. Binary images play a crucial role in digital image processing, especially in practical applications. The set properties of an image depend only on the location of pixels with a value of 0 or 255, eliminating the need for multi-level pixel values, simplifying processing and reducing data processing and compression. To obtain an ideal binary image, closed, connected boundaries are typically used to define non-overlapping regions. All pixels with a grayscale value greater than or equal to the threshold are considered to belong to a specific object and are represented by a grayscale value of 255; otherwise, these pixels are excluded from the object region and have a grayscale value of 0, representing the background or an exception to the object region.

[0083] Binarization typically involves setting a threshold, comparing the pixel value of each pixel with the threshold, setting the pixel value of pixels above the threshold to 255, and setting the pixel value of pixels below the threshold to 0.

[0084] Optionally, the acquired original image containing the tracking line is filtered to obtain the filtered original image, including:

[0085] The original image containing the tracking line is extracted using a preset channel to obtain a single-channel original image;

[0086] Median filtering is applied to the original image of a single channel to obtain the filtered original image.

[0087] After acquiring the image, a single channel is extracted based on the environment; for example, the red channel is extracted underwater to reduce the influence of the blue channel. Median filtering and binarization are then used to convert the image into a black and white image, thus initially obtaining the contour of the tracking line.

[0088] Step S202: Perform morphological operations on the preprocessed image to identify abnormal pixels in the preprocessed image, invert the pixel values ​​of the abnormal pixels, and determine the inverted preprocessed image as the target image.

[0089] In this application, morphological operations, also known as morphological transformations, are simple operations based on image shapes, usually performed on binary images. Erosion and dilation are two basic morphological operators, with variants such as opening, closing, top hat, and black hat operations.

[0090] Morphological operations can identify anomalous pixels in the preprocessed image. These anomalous pixels are often bright spots caused by non-tracking lines during image acquisition. By inverting the pixel value of the anomalous pixel, it is transformed into a non-tracking pixel, so that the image contains only pixels corresponding to the tracking line contour as much as possible.

[0091] Optionally, morphological operations can be performed on the preprocessed image to identify anomalous pixels, including:

[0092] The preprocessed image is dilated to obtain the dilated image, and the dilated image is eroded to obtain the eroded image.

[0093] Extract the pixel value of each pixel in the eroded image, and identify pixels with pixel values ​​greater than a threshold as abnormal pixels in the preprocessed image.

[0094] Dilation expands the highlighted areas in an image, resulting in a larger highlighted area in the final image. Erosion, on the other hand, erodes the highlighted areas in the original image, resulting in a smaller highlighted area in the final image. Dilation finds a local maximum, while erosion finds a local minimum.

[0095] Optionally, the pixel values ​​of abnormal pixels are inverted, and the preprocessed image after inversion is determined to be the target image, including:

[0096] In the preprocessed image, the pixel values ​​of abnormal pixels are set to zero, and the preprocessed image after the values ​​are set to zero is determined to be the target image.

[0097] After binarization, the pixel values ​​of the pixels in the image are only 255 and 0. The pixel value of the pixel corresponding to the tracking line contour is determined to be 255, and the pixel value of the corresponding abnormal pixel is also 255. Therefore, inverting the pixel value means setting the pixel value of the abnormal pixel 255 to 0, thereby identifying a more accurate tracking line contour.

[0098] Step S203: In the coordinate system of the target image, determine the coordinates of the leftmost and rightmost pixels of the tracking line contour on each horizontal pixel row of the target image.

[0099] In this application, since there is no coordinate transformation process in steps S201 to S202, the coordinate system of the target image is the same as the coordinate system of the original image. For example, the lower left corner of the target image is taken as the origin, the horizontal axis is taken as the X-axis, and the vertical axis is taken as the Y-axis. Each horizontal pixel row is the vertical axis corresponding to each value on the Y-axis.

[0100] The tracking line contour is the outline of the line in the original image. At least two pixels may exist in the same horizontal pixel row to represent the tracking line contour. The leftmost pixel is the one closest to the origin on the X-axis, and the rightmost pixel is the one farthest from the origin on the X-axis. The coordinates of the pixels are in the coordinate system of the target image.

[0101] If there is only one pixel in a horizontal pixel row that represents the outline of the tracking line, then that pixel is both the leftmost pixel and a pixel on the left or right.

[0102] Step S204: For any horizontal pixel row, take the average of the coordinates of the leftmost pixel and the rightmost pixel as the positioning point of the tracking line contour in the horizontal pixel row, and obtain all the positioning points of the tracking line contour.

[0103] In this application, the tracking line contour in the target image may correspond to a line with a certain width. In order to control the movement of the medical robot, the tracking line needs to be drawn as a line without width. For any horizontal pixel row, the midpoint between the leftmost pixel and the rightmost pixel of the tracking line contour is used as the positioning point that constitutes the tracking line. This positioning point is used to guide the movement of the medical robot.

[0104] Step S205: Perform straight line fitting on all positioning points to obtain the fitted tracking line.

[0105] In this application, the fitted tracking line is used to indicate the movement of the center point of the medical robot along the fitted tracking line.

[0106] Linear fitting refers to fitting all points into a straight line, which is called the tracking line. If the coordinates of each point in the tracking line are known, and the coordinates of the center point of the medical robot are also known, then the difference in coordinates between the two points can control the turning, forward movement, and other movements of the medical robot, so that the center point of the medical robot can move along the fitted tracking line.

[0107] Optionally, a straight line is fitted to all the positioning points to obtain the fitted tracking line, including:

[0108] Based on the direction of movement of the medical robot, the fitting weights of all positioning points are assigned. Among them, all positioning points are perpendicular to the direction of movement, and the positioning point corresponding to the farthest perpendicular point in the direction of movement has the highest fitting weight.

[0109] Using the corresponding fitting weights, perform linear weighted fitting on all positioning points to determine the fitted line as the fitted tracking line.

[0110] Specifically, the weight of the forward motion direction is set to a high weight to reduce the influence of points already passed during the turning process. Traditional visual fitting methods require fitting different types of lines, which is complex and not conducive to motion control. Fitting various types of lines as straight lines reduces the difficulty of curve fitting and facilitates visual turning.

[0111] This application embodiment performs tracking line contour recognition on the acquired original image containing the tracking line to obtain a preprocessed image containing the tracking line contour. Morphological operations are performed on the preprocessed image to identify abnormal pixels. The pixel values ​​of the abnormal pixels are inverted, and the inverted preprocessed image is determined as the target image. In the coordinate system of the target image, the coordinates of the leftmost and rightmost pixels of the tracking line contour on each horizontal pixel row of the target image are determined. For any horizontal pixel row, the average of the coordinates of the leftmost and rightmost pixels is used as the positioning point of the tracking line contour in the horizontal pixel row, obtaining all positioning points of the tracking line contour. Line fitting is performed on all positioning points to obtain the fitted tracking line. The fitted tracking line is used to indicate the movement of the center point of the medical robot along the fitted tracking line. Through the above process, the medical robot accurately identifies and locates the tracking line and obtains an accurate tracking line result, thereby controlling the medical robot to move along the tracking line and reducing the difficulty of controlling the movement of the medical robot.

[0112] See Figure 3 This is a flowchart illustrating a visual tracking line recognition and positioning method provided in Embodiment 3 of this application. Figure 3 As shown, the identification and positioning method includes:

[0113] Step S301: Perform line contour recognition on the acquired original image containing the line to obtain a preprocessed image containing the line contour.

[0114] Step S302: Perform morphological operations on the preprocessed image to identify abnormal pixels in the preprocessed image, invert the pixel values ​​of the abnormal pixels, and determine the inverted preprocessed image as the target image.

[0115] Step S303: In the coordinate system of the target image, determine the coordinates of the leftmost and rightmost pixels of the tracking line contour on each horizontal pixel row of the target image.

[0116] Step S304: For any horizontal pixel row, take the average of the coordinates of the leftmost pixel and the rightmost pixel as the positioning point of the tracking line contour in the horizontal pixel row, and obtain all the positioning points of the tracking line contour.

[0117] Step S305: Perform straight line fitting on all positioning points to obtain the fitted tracking line.

[0118] Steps S301 to S305 are the same as steps S201 to S205 above. For details, please refer to the description of steps S201 to S205. They will not be repeated here.

[0119] Step S306: Based on the installation information of the acquisition device for acquiring the original image on the medical robot, determine the positional relationship between the center point of the medical robot and the fitted tracking line.

[0120] In this application, the installation information may include the positional relationship between the center point of the acquisition device and the medical robot. By converting the pixels in the image acquired by the acquisition device into the world coordinate system, the expression of the tracking line in the world coordinate system can be obtained.

[0121] Since the position of the center point in the world coordinate system is known, and the position of the acquisition device in the world coordinate system is also known, the positional relationship between the center point and the fitted tracking line in the world coordinate system, i.e., the difference in coordinates, can be determined.

[0122] Step S307: Based on the positional relationship between the center point and the midpoint of the front wheel drive shaft, and the positional relationship between the center point and the midpoint of the rear wheel drive shaft, determine the positional relationship between the midpoint of the front wheel drive shaft and the midpoint of the rear wheel drive shaft and the fitted tracking line, respectively.

[0123] In this application, the positional relationship between the midpoint of the front drive shaft, the midpoint of the rear drive shaft, and the center point of the medical robot is fixed and known. The positional relationship between the midpoints of the two shafts and the fitted tracking line can be determined through relational calculation.

[0124] For example, a medical robot can have four drive wheels, with two drive wheels in the front and two drive wheels in the rear. The front drive axle is the axle between the two front drive wheels, and the rear drive axle is the axle between the two rear drive wheels.

[0125] Step S308: With the midpoints of the front and rear drive shafts simultaneously on the fitted tracking line as the target, control the steering angle of the front and rear wheels when the medical robot moves forward.

[0126] In this application, the two midpoints of the front and rear axles can determine a line segment. Controlling the movement of the medical robot means aligning this line segment with the fitted tracking line, that is, both midpoints are on the fitted tracking line, thereby controlling the medical robot's forward movement and turning.

[0127] In controlling the motion of the medical robot, PID control can be employed. Unlike traditional methods that use slope and intercept as control parameters, this application uses two midpoints as PID correction points. The positions of these two midpoints in the fitted tracking line image are used as the expected values ​​of the PID controller. Two independent PID controllers are set up. The actual values ​​of the two midpoints are calculated using the fitted tracking line obtained earlier. The deviation between the two values ​​is input into the PID controller to obtain the motion control setpoint. Using the two midpoints as PID control parameters separates the preceding and following motions, avoiding mutual interference and making the motion control more stable.

[0128] This application embodiment performs tracking line contour recognition on the acquired original image containing tracking lines to obtain a preprocessed image containing tracking line contours. Morphological operations are performed on the preprocessed image to identify abnormal pixels. The pixel values ​​of these abnormal pixels are inverted, and the inverted preprocessed image is determined as the target image. In the coordinate system of the target image, the coordinates of the leftmost and rightmost pixels of the tracking line contour on each horizontal pixel row are determined. For any horizontal pixel row, the average of the coordinates of the leftmost and rightmost pixels is used as the location point of the tracking line contour in the horizontal pixel row, obtaining all location points of the tracking line contour. Line fitting is performed on all location points to obtain the fitted tracking line. Based on the installation information of the acquisition device on the medical robot, the center point of the medical robot and the fitted tracking line are determined. Positional relationships are determined based on the positional relationships between the center point and the midpoint of the front and rear drive shafts, as well as the positional relationships between the center point and the midpoint of the rear drive shafts and the fitted tracking line. With the goal of both the midpoints of the front and rear drive shafts simultaneously lying on the fitted tracking line, the steering angles of the front and rear wheels are controlled when the medical robot moves forward. Through this process, the medical robot accurately identifies and locates the tracking line and obtains an accurate tracking line result, thus enabling control of the medical robot along the tracking line and reducing the difficulty of controlling the robot's movement. When performing PID control, two sets of controllers are set up, with the drive midpoint used as the expected value for both controllers. The position of the drive midpoint in the image is relatively fixed, avoiding interference between different drives during control. Simultaneously, it balances steering and propulsion functions, making the system more efficient.

[0129] Corresponding to the visual tracking line recognition and localization method in the above embodiments, Figure 4 This diagram shows a structural block diagram of a visual tracking line recognition and positioning device provided in Embodiment 4 of this application. The above-mentioned recognition and positioning device is applied to... Figure 1The client in this application can be a computer device connected to the medical robot to control its movement. This computer device can be mounted on the medical robot or set up independently and communicate with it wirelessly or via a wired connection. The medical robot is equipped with corresponding image acquisition devices to acquire surrounding images. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0130] See Figure 4 The identification and positioning device includes:

[0131] Preprocessing module 41 is used to perform tracking line contour recognition on the acquired original image containing tracking lines to obtain a preprocessed image containing tracking line contours.

[0132] The target image determination module 42 is used to perform morphological operations on the preprocessed image, determine abnormal pixels in the preprocessed image, invert the pixel values ​​of the abnormal pixels, and determine the inverted preprocessed image as the target image.

[0133] The coordinate determination module 43 is used to determine the coordinates of the leftmost and rightmost pixels of the tracking line contour on each horizontal pixel row of the target image in the coordinate system of the target image.

[0134] The positioning point determination module 44 is used to take the average of the coordinates of the leftmost pixel and the rightmost pixel as the positioning point of the tracking line contour in the horizontal pixel row for any horizontal pixel row, so as to obtain all the positioning points of the tracking line contour.

[0135] The tracking line determination module 45 is used to perform straight line fitting on all positioning points to obtain the fitted tracking line. The fitted tracking line is used to indicate the movement of the center point of the medical robot along the fitted tracking line.

[0136] Optionally, the identification and positioning device also includes:

[0137] The first positional relationship determination module is used to determine the positional relationship between the center point of the medical robot and the fitted tracking line after performing straight line fitting on all positioning points to obtain the fitted tracking line, based on the installation information of the acquisition device that acquired the original image on the medical robot.

[0138] The second position relationship determination module is used to determine the position relationship between the midpoint of the front wheel drive shaft and the midpoint of the rear wheel drive shaft and the fitted tracking line, based on the position relationship between the center point and the midpoint of the front wheel drive shaft, and the position relationship between the center point and the midpoint of the rear wheel drive shaft.

[0139] The control module is used to control the steering angle of the front and rear wheels of the medical robot when it moves forward, with the midpoints of the front and rear drive shafts simultaneously lying on the fitted tracking line.

[0140] Optionally, the preprocessing module 41 includes:

[0141] The filtering unit is used to filter the acquired original image containing the tracking line to obtain the filtered original image.

[0142] The binarization processing unit is used to binarize the filtered original image to obtain a preprocessed image containing the contour of the tracking line.

[0143] Optionally, the filtering unit includes:

[0144] The channel filtering subunit is used to extract the acquired original image containing the tracking line using a preset channel to obtain the original image of a single channel.

[0145] The median filtering subunit is used to perform median filtering on the original image of a single channel to obtain the filtered original image.

[0146] Optionally, the target image determination module 42 includes:

[0147] The morphological operation unit is used to dilate the preprocessed image to obtain the dilated image, and then to erode the dilated image to obtain the eroded image.

[0148] The anomaly point determination unit is used to extract the pixel value of each pixel in the eroded image and determine the pixel values ​​of pixels with values ​​greater than a threshold as anomaly pixels in the preprocessed image.

[0149] Optionally, the target image determination module 42 includes:

[0150] The target image determination unit is used to set the pixel values ​​of abnormal pixels to zero in the preprocessed image, and determine the preprocessed image after setting the pixel values ​​to zero as the target image.

[0151] Optionally, the tracking line determination module 45 includes:

[0152] The weight assignment unit is used to assign fitting weights to all positioning points according to the forward direction of the medical robot. Among them, all positioning points are perpendicular to the forward direction, and the positioning point corresponding to the farthest perpendicular point in the forward direction has the highest fitting weight.

[0153] The tracking line determination unit is used to perform linear weighted fitting on all positioning points using the corresponding fitting weights, and determine the fitted line as the fitted tracking line.

[0154] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0155] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 5 of this application. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above-described embodiments of the visual tracking line recognition and localization methods.

[0156] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0157] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0158] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0160] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0161] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0162] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0163] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0164] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying and positioning a visual tracking line, characterized in that, The recognition positioning method comprises: contour recognition of the acquired original image containing the tracking line to obtain a pretreatment image containing a tracking line contour; performing morphological operation on the pretreatment image to determine abnormal pixel points in the pretreatment image, inverting the pixel value of the abnormal pixel points, and determining the pretreatment image after inversion as a target image; in the coordinate system of the target image, determining the coordinates of the leftmost pixel point and the coordinates of the rightmost pixel point of the tracking line contour on each horizontal pixel line of the target image; for any horizontal pixel line, taking the mean value of the coordinates of the leftmost pixel point and the coordinates of the rightmost pixel point as the positioning point of the tracking line contour corresponding to the horizontal pixel line to obtain all positioning points of the tracking line contour; performing linear fitting on all positioning points to obtain a fitted tracking line, wherein the fitted tracking line is used to indicate the movement of the center point of the medical robot along the fitted tracking line; after performing linear fitting on all positioning points to obtain a fitted tracking line, further comprising: determining the positional relationship between the center point of the medical robot and the fitted tracking line according to the installation information of the acquisition device on the medical robot; determining the positional relationship between the midpoint of the front wheel drive shaft and the fitted tracking line and the positional relationship between the midpoint of the rear wheel drive shaft and the fitted tracking line according to the positional relationship between the center point and the midpoint of the front wheel drive shaft and the positional relationship between the center point and the midpoint of the rear wheel drive shaft; controlling the steering angle of the front and rear wheels of the medical robot when moving forward, setting two independent PID controllers to control the front wheel drive shaft and the rear wheel drive shaft respectively, so that the midpoints of the two drive shafts are on the fitted tracking line at the same time; performing linear fitting on all positioning points to obtain a fitted tracking line comprises: assigning fitting weights to all positioning points according to the forward direction of the medical robot, wherein the fitting weight of the positioning point corresponding to the vertical point farthest in the forward direction is the highest; performing linear weighted fitting on all positioning points using the corresponding fitting weights to determine the fitting straight line as the fitted tracking line.

2. The method of claim 1, wherein, contour recognition of the acquired original image containing the tracking line to obtain a pretreatment image containing a tracking line contour comprises: filtering the acquired original image containing the tracking line to obtain a filtered original image; performing binaryzation processing on the filtered original image to obtain a pretreatment image containing a tracking line contour.

3. The method of claim 2, wherein, filtering the acquired original image containing the tracking line to obtain a filtered original image comprises: extracting the acquired original image containing the tracking line using a preset channel to obtain a single-channel original image; performing median filtering on the single-channel original image to obtain a filtered original image.

4. The method of claim 1 to 3, wherein, performing morphological operation on the pretreatment image to determine abnormal pixel points in the pretreatment image comprises: The pre-processed image is dilated to obtain a dilated image, and the dilated image is eroded to obtain an eroded image; The pixel value of each pixel point in the eroded image is extracted, and a pixel point with a pixel value greater than a threshold value is determined as an abnormal pixel point in the pre-processed image.

5. The method of claim 4, wherein, The pixel value of the abnormal pixel point is inverted, and the pre-processed image after inversion is determined as a target image, including: In the pre-processed image, the pixel value of the abnormal pixel point is set to zero, and the pre-processed image after setting to zero is determined as a target image.

6. A visual tracking line recognition positioning device, characterized by, The identification and positioning device comprises: A pre-processing module for performing contour recognition on the obtained original image containing a tracking line to obtain a pre-processed image containing a tracking line contour; A target image determination module for performing morphological operations on the pre-processed image, determining abnormal pixel points in the pre-processed image, inverting the pixel value of the abnormal pixel points, and determining the pre-processed image after inversion as a target image; A coordinate determination module for determining the coordinates of the leftmost pixel point and the coordinates of the rightmost pixel point of the tracking line contour on each horizontal pixel row of the target image in the coordinate system of the target image; A positioning point determination module for determining the mean value of the coordinates of the leftmost pixel point and the coordinates of the rightmost pixel point as the positioning point of the tracking line contour corresponding to the horizontal pixel row for any horizontal pixel row, and obtaining all positioning points of the tracking line contour; A tracking line determination module for performing linear fitting on all positioning points to obtain a fitted tracking line, wherein the fitted tracking line is used to indicate the movement of the center point of the medical robot along the fitted tracking line; After performing linear fitting on all positioning points to obtain a fitted tracking line, the method further comprises: According to the installation information of the acquisition device for acquiring the original image on the medical robot, the positional relationship between the center point of the medical robot and the fitted tracking line is determined; According to the positional relationship between the center point and the midpoint of the front wheel drive shaft, and the positional relationship between the center point and the midpoint of the rear wheel drive shaft, the positional relationship between the midpoints of the front wheel drive shaft and the rear wheel drive shaft and the fitted tracking line is determined; The steering angles of the front and rear wheels of the medical robot are controlled when the medical robot moves forward, with the midpoints of the front wheel drive shaft and the rear wheel drive shaft as targets on the fitted tracking line, wherein two independent PID controllers are set to control the front wheel drive shaft and the rear wheel drive shaft respectively, so that the midpoints of the two drive shafts are on the fitted tracking line at the same time; The method of performing linear fitting on all positioning points to obtain a fitted tracking line comprises: According to the forward direction of the medical robot, the fitting weights of all positioning points are assigned, wherein the fitting weight of the positioning point corresponding to the vertical point farthest in the forward direction is the highest; The linear weighted fitting of all positioning points is performed using the corresponding fitting weights, and the fitting straight line is determined as the fitted tracking line.

7. A computer device, comprising: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the identification positioning method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program is executable on the processor to implement the identification positioning method according to any one of claims 1 to 5.

Images