Biopsy sampling method and device, biopsy forceps and storage medium

Abstract

The invention provides a biopsy sampling method and device, biopsy forceps and a storage medium, and the method comprises the steps: obtaining a first endoscopic image of a gastrointestinal tract, and determining a target lesion region in the first endoscopic image of the gastrointestinal tract; determining a first position of preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; predicting a moving path of the biopsy forceps moving from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the moving path; after the biopsy forceps move to the target lesion area, biopsy sampling is conducted on the target lesion area through the biopsy forceps. The moving path of the biopsy forceps is planned in combination with the endoscopic image of the gastrointestinal tract, repeated positioning and repeated sampling in the sampling process of the biopsy forceps are avoided, and the sampling precision and the sampling efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of biopsy sampling, and in particular to a biopsy sampling method, device, biopsy forceps and storage medium. Background Art

[0002] Biopsy forceps are a medical device used to remove tissue samples from the patient's body for pathological examination. They are usually composed of a head, a rod and a handle. The head has a pair of blades with beaks for clamping tissue samples. Biopsy forceps are usually used in conjunction with endoscopes and other equipment to accurately locate and remove tissues that need to be biopsied. At present, when biopsy forceps are used for multiple tissue sampling, repeated sampling and repeated positioning are required, and the sampling efficiency is low. Summary of the invention

[0003] The embodiments of the present application provide a biopsy sampling method, device, biopsy forceps and storage medium, which can plan the route of the biopsy forceps in combination with endoscopic images, avoid repeated positioning and repeated sampling, and improve sampling accuracy and efficiency.

[0004] In a first aspect, the present application provides a biopsy sampling method, comprising:

[0005] Acquiring a first endoscopic image of the gastrointestinal tract, and determining a target lesion area in the first endoscopic image of the gastrointestinal tract;

[0006] Determining a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract;

[0007] Predicting a movement path of the biopsy forceps from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the movement path;

[0008] When the biopsy forceps is moved to the target lesion area, the biopsy forceps is used to perform biopsy sampling on the target lesion area.

[0009] In some possible embodiments, acquiring a first endoscopic image of the gastrointestinal tract and determining a target lesion area in the first endoscopic image of the gastrointestinal tract includes:

[0010] Acquiring the first endoscopic image of the gastrointestinal tract, and performing edge detection on the first endoscopic image of the gastrointestinal tract to determine an initial lesion area of ​​the first endoscopic image of the gastrointestinal tract;

[0011] Performing shape positioning analysis on the initial lesion area to determine the shape positioning lesion area of ​​the gastrointestinal endoscopic image; growing the shape positioning lesion area using a preset region growing algorithm to obtain a growing lesion area;

[0012] An initial contour line of the growing lesion area is determined, and a target contour line of the growing lesion area is determined based on a preset contour line evolution strategy and the initial contour line, wherein the area enclosed by the target contour line is the target lesion area.

[0013] In some possible embodiments, determining a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract includes:

[0014] Determining whether a preset biopsy forceps exists in the first endoscopic image of the gastrointestinal tract;

[0015] If the biopsy forceps exists in the first gastrointestinal endoscopic image, obtaining a preset biopsy forceps template image;

[0016] The first endoscopic image of the gastrointestinal tract and the biopsy forceps template image are matched to determine a first position of the biopsy forceps in the first endoscopic image of the gastrointestinal tract.

[0017] In some possible embodiments, the first position is used as a starting point node, and the center of the target lesion area is used as an ending point node;

[0018] The predicting a moving path of the biopsy forceps from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the moving path, comprises:

[0019] Acquire the three-dimensional model of the gastrointestinal tract, and determine other nodes in the three-dimensional model except the starting node and the end node;

[0020] Taking the starting point node as the current node, determining the adjacent nodes around the current node;

[0021] Calculating the heuristic function value between the current node and the adjacent node to determine the target adjacent node corresponding to the minimum heuristic function value;

[0022] Taking the target adjacent node as a new current node, determining a new target adjacent node until the target adjacent node is the end node or traversing the other nodes;

[0023] The moving path is generated according to the starting point node, the end point node and a plurality of target adjacent nodes.

[0024] In some possible embodiments, after the biopsy forceps is moved to the target lesion area, the biopsy forceps is used to perform biopsy sampling on the target lesion area, including:

[0025] When the biopsy forceps moves to the target lesion area, the biopsy forceps is controlled to perform a first sampling in the target lesion area, and a first sampling position of the first sampling is marked after the sampling is completed;

[0026] Controlling the biopsy forceps to move a preset distance in a preset direction to a second sampling position, where the second sampling position is located in the target lesion area;

[0027] The biopsy forceps is controlled to rotate at the second sampling position, and a second sampling is performed after the rotation.

[0028] In some possible embodiments, the method further includes:

[0029] In the process of the biopsy forceps moving along the moving path, new gastrointestinal endoscopic images are continuously acquired;

[0030] Tracking the positions of the target lesion area and the biopsy forceps in the new gastrointestinal endoscopic image;

[0031] Dynamically adjusting the moving path according to the positions of the target lesion area and the biopsy forceps in the new gastrointestinal endoscopic image to obtain an adjusted new moving path;

[0032] The biopsy forceps is controlled to move from the first position to the target lesion area according to a new moving path.

[0033] In some possible embodiments, before acquiring the first endoscopic image of the gastrointestinal tract, the method further includes:

[0034] Acquire an initial endoscopic image of the gastrointestinal tract, and convert the initial endoscopic image of the gastrointestinal tract into a grayscale endoscopic image;

[0035] Noise in the grayscale endoscopic image is removed to obtain the first endoscopic image of the gastrointestinal tract.

[0036] In a second aspect, an embodiment of the present application provides a biopsy sampling device, the device comprising:

[0037] A lesion region determination module, used to obtain a first endoscopic image of the gastrointestinal tract and determine a target lesion region in the first endoscopic image of the gastrointestinal tract;

[0038] A positioning module, used to determine a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract;

[0039] a path prediction module, used to predict a moving path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area according to the moving path;

[0040] The sampling module is used to perform biopsy sampling on the target lesion area using the biopsy forceps after the biopsy forceps is moved to the target lesion area.

[0041] In a third aspect, an embodiment of the present application further provides a biopsy forceps, the biopsy forceps comprising:

[0042] A handle operating device and a main pipe, one end of the main pipe is connected to the handle operating device, and the other end of the main pipe is connected to the auxiliary pipe;

[0043] A sliding rod is fixed on the side of the auxiliary pipe away from the main pipe, and a first clamp and a second clamp oppositely arranged are arranged on the outer side of the auxiliary pipe away from the bending pipe. The first clamp and the second clamp are connected to the sliding rod.

[0044] In a fourth aspect, the electronic device provided in the present application includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the biopsy sampling method provided in the present application.

[0045] In a fifth aspect, the computer-readable storage medium provided in the present application stores a plurality of instructions, which are suitable for loading by a processor to implement the steps in the biopsy sampling method provided in the present application.

[0046] The present application provides a biopsy sampling method, device, biopsy forceps and storage medium, which obtain a first endoscopic image of the gastrointestinal tract, and determine the target lesion area in the first endoscopic image of the gastrointestinal tract; determine the first position of the preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; predict the movement path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area according to the movement path; when the biopsy forceps moves to the target lesion area, use the biopsy forceps to perform biopsy sampling on the target lesion area. The present application plans the movement path of the biopsy forceps in combination with the endoscopic image of the gastrointestinal tract, avoids repeated positioning and repeated sampling during the sampling process of the biopsy forceps, and improves the sampling accuracy and sampling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 is a schematic diagram of a biopsy sampling system provided in an embodiment of the present application;

[0049] Figure 2It is a flow chart of an embodiment of the biopsy sampling method provided in the embodiments of the present application;

[0050] Figure 3 This is a flow chart of an embodiment of determining a target lesion area provided in an embodiment of the present application;

[0051] Figure 4 This is a flow chart of an embodiment of predicting a moving path provided by an embodiment of the present application;

[0052] Figure 5 It is a structural schematic diagram of an embodiment of the biopsy forceps provided in the embodiment of the present application;

[0053] Figure 6 It is a structural schematic diagram of the hinge joint of the biopsy forceps provided in an embodiment of the present application;

[0054] Figure 7 is a schematic diagram of the structure of the chuck provided in an embodiment of the present application;

[0055] Figure 8 It is a schematic structural diagram of a limiting ring of a biopsy forceps provided in an embodiment of the present application;

[0056] Fig. 9 The embodiment of this application provides Figure 6 An enlarged schematic diagram of point B in FIG.

[0057] Fig.10 It is a schematic diagram of the structure of the square plate in the biopsy forceps provided in the embodiment of the present application;

[0058] Fig.11 is a schematic diagram of the structure of a biopsy sampling device provided in an embodiment of the present application;

[0059] Fig.12 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0060] Fig.13 is a schematic diagram of the structure of an embodiment of a terminal device provided in an embodiment of the present application;

[0061] Fig.14 It is a schematic diagram of the server structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] It should be noted that the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application and should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

[0063] In the following description of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0064] In the following description of the present application, the terms "first\second\third" involved are merely to distinguish similar objects and do not represent a specific order for the objects. It is understandable that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0065] In order to improve the effect of biopsy sampling, the embodiments of the present application provide a biopsy sampling method, device, biopsy forceps and storage medium. Among them, the biopsy sampling method can be performed by a biopsy sampling device, or by an electronic device integrated with the biopsy sampling device. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0066] Please refer to Figure 1 , the present application also provides a biopsy sampling system, such as Figure 1 As shown, the biopsy sampling system includes an electronic device 100, in which the biopsy sampling device provided by the present application is integrated.

[0067] Among them, the electronic device 100 can be any device equipped with a processor and has processing capabilities, such as mobile electronic devices with processors such as smart phones, tablet computers, PDAs, laptops, smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, industrial equipment, etc.

[0068] In addition, if Figure 1 As shown, the biopsy sampling system may further include a memory 200 for storing endoscopic images.

[0069] In the embodiment of the present application, the memory 200 can be a cloud storage. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0070] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0071] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0072] It should be noted that Figure 1 The scenario diagram of the biopsy sampling system shown is merely an example. The biopsy sampling system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art will appreciate that with the evolution of the biopsy sampling system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0073] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence (AI), computer vision (CV), and machine learning (ML), which are specifically described by the following embodiments:

[0074] Among them, AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0075] AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0076] CV is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify, track and measure targets, and further performs graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes adversarial disturbance generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0077] The biopsy sampling method, device, biopsy forceps and storage medium provided by the present application are described in detail below. It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0078] Please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of the biopsy sampling method provided in the present application. Figure 2 As shown, the process of the biopsy sampling method provided in this application is as follows:

[0079] 201. Obtain a first endoscopic image of the gastrointestinal tract, and determine a target lesion area in the first endoscopic image of the gastrointestinal tract.

[0080] The biopsy sampling method provided in the present application is used for biopsy sampling of the gastrointestinal tract and mainly for biopsy sampling of the lesion area in the gastrointestinal tract, so it is necessary to first determine the lesion area in the gastrointestinal tract. In the embodiment of the present application, an endoscopic device can be used to collect a first endoscopic image of the gastrointestinal tract, and then the target lesion area in the gastrointestinal tract can be determined based on the first endoscopic image of the gastrointestinal tract, and finally a biopsy forceps can be used to perform biopsy sampling on the target lesion area.

[0081] Specifically, an initial endoscopic image of the gastrointestinal tract may be acquired by using an endoscopic device; the initial endoscopic image of the gastrointestinal tract is then preprocessed, such as converting the color initial endoscopic image acquired by the endoscopic device into a grayscale endoscopic image, thereby reducing the amount of subsequent calculations. In some embodiments, the following formula may be used to convert the color initial endoscopic image into a grayscale image:

[0082] Gray(i,j)=0.299*R(i,j)+0.587*G(i,j)+0.114*B(i,j)

[0083] Among them, R(i, j), G(i, j), B(i, j) are the red component, green component and blue component of the initial endoscopic image at point (i, j), respectively. After converting the initial endoscopic image into a grayscale endoscopic image, it is also necessary to remove the noise in the grayscale endoscopic image to obtain the final first endoscopic image of the gastrointestinal tract. In some embodiments, Gaussian filtering can be used to remove the noise in the grayscale endoscopic image. The specific process of denoising can refer to the prior art and is not limited in this application.

[0084] After obtaining the first endoscopic image of the gastrointestinal tract, it is also necessary to determine the target lesion area in the first endoscopic image of the gastrointestinal tract, that is, to determine the target lesion area in the gastrointestinal tract. Figure 3 As shown, a schematic diagram of a flow chart of an embodiment of determining a target lesion area provided in an embodiment of the present application may include the following steps:

[0085] 301. Obtain a first gastrointestinal endoscopic image, and perform edge detection on the gastrointestinal endoscopic image to determine an initial lesion area of ​​the gastrointestinal endoscopic image.

[0086] In this application, a variety of different methods are used to gradually determine the scope of the target lesion area: first, edge detection can be performed on the gastrointestinal endoscopic image to determine an initial lesion area with general accuracy, and then the initial lesion area can be gradually refined to obtain a target lesion area with higher accuracy. In some embodiments, the Canny algorithm can be used to perform edge detection on the gastrointestinal endoscopic image to obtain an initial lesion area. Among them, the Canny algorithm for edge detection includes but is not limited to Gaussian filtering, gradient calculation, non-maximum suppression and double threshold processing; the specific process of edge detection using the Canny algorithm can refer to the prior art and will not be repeated here.

[0087] 302. Perform shape analysis and positioning on the initial lesion area to determine the shape positioning lesion area of ​​the gastrointestinal endoscopic image.

[0088] After obtaining the initial lesion area, the initial lesion area can be further refined to obtain the target lesion area. In an embodiment of the present application, the initial lesion area can be firstly analyzed and located by shape analysis, for example, using shapes such as circles and straight lines to identify the initial lesion area to assist in locating the lesion area; for example, if the initial lesion area is close to a straight line shape, a straight line is used to assist in identification; if the initial lesion area is similar to a circle, a circle is used to assist in identification. Specifically, the corresponding parameter space can be set according to the shape to be detected, such as a circle or a straight line. For example, for a circle, the parameter space includes parameters such as the xy coordinates of the center of the circle and the radius r; if a straight line is used to assist in identification, the parameter space will include parameters such as the slope and intercept of the straight line.

[0089] A circle is defined in parameter space by three parameters: the x and y coordinates of the center of the circle and the radius r. For each edge point (x, y) in the image, it can be assumed to be a point on a circle, and possible radii r are traversed to draw a corresponding circle family including multiple different circles in parameter space. The intersection of these circles in parameter space is the possible center position, and the corresponding radius is the radius of the circle.

[0090] The equation of a circle can be expressed as (xa)2+(yb)2=r2, where (a, b) are the coordinates of the center of the circle and r is the radius. In Hough space, each edge point (x, y) is mapped to a series of possible circles (i.e., a series of (a, b, r) combinations). You can use the cv2.HoughCircles() function in OpenCV to perform Hough circle transform, and set appropriate parameters (such as dp, minDist, param1, param2, minRadius, maxRadius) to optimize the detection results. In the parameter space, through the accumulation process, find the points with higher cumulative values ​​(for example, higher than the threshold) at the intersection of the curve or surface drawn in the parameter space. These points are the candidate locations of the lesion area. According to the set threshold, select the points with cumulative values ​​higher than the threshold as the final lesion area detection points. According to the detected lesion area detection points, draw the corresponding shapes in the original image, extract the position and size information of these shapes, and determine the shape to locate the lesion area.

[0091] 303. Growing the shape-located lesion region using a preset region growing algorithm to obtain a grown lesion region.

[0092] After obtaining the shape-located lesion area, the region generation algorithm can be further used to further finely separate the lesion area to obtain the generated lesion area; this is because the actual lesion area is constantly spreading, and the original non-lesion area may also become a lesion area, so this part of the area needs to be considered, and the lesion condition can also be determined by biopsy of this part of the area to assist in diagnosis. Specifically, the lesion area detection point detected by the Hough transform can be used as the initial point for region generation, and then the adjacent pixels are added to the corresponding area according to the similarity between pixels; when there are no more qualified pixels to be added, the region stops generating and the growing lesion area is obtained. In some embodiments, pixels with a similarity greater than a preset similarity threshold can be determined based on the similarity of the grayscale values ​​or texture similarity between pixels, and then a plurality of qualified pixels are combined to obtain a growing lesion area.

[0093] 304. Determine an initial contour line of the growing lesion region, and determine a target contour line of the growing lesion region based on a preset contour line evolution strategy and the initial contour line. The region enclosed by the target contour line is the target lesion region.

[0094] The growing lesion area determined in the aforementioned embodiment is a fuzzy area range with low precision. Therefore, after obtaining the growing lesion area, the initial contour line of the growing lesion area can be further determined to obtain an accurate lesion area range. Specifically, by solving the contour line evolution equation, the contour line gradually approaches the real boundary of the lesion area, thereby obtaining the target contour line corresponding to the target lesion area, and then determining the final target lesion area. In the embodiment of the present application, the solution can be based on the gradient or the energy function of the area. When the change of the contour line is less than the preset contour line change threshold, the calculation is stopped. The contour line obtained at this time is the target contour line; and the area surrounded by the target contour line is the target lesion area. In the above embodiments, the specific methods for edge detection, shape positioning analysis, growth algorithm and contour line calculation can refer to the prior art and are not limited here.

[0095] 202. Determine a first position of a preset biopsy forceps in a first endoscopic image of the gastrointestinal tract.

[0096] In an actual biopsy sampling scenario, the biopsy forceps needs to go deep into the gastrointestinal tract until it reaches the vicinity of the target lesion area in order to perform biopsy sampling on the target lesion area; at this time, the endoscopic image collected by the endoscopic device usually includes the biopsy forceps. Therefore, it is first necessary to determine whether there is a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; if there is a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract, it is necessary to determine the first position of the preset biopsy forceps in the first endoscopic image of the gastrointestinal tract. In an embodiment of the present application, if it is determined that there is a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract, a predefined biopsy forceps template image can be obtained, and then the first endoscopic image of the gastrointestinal tract and the biopsy forceps template image are matched, that is, an image matching the biopsy forceps template image is determined in the first endoscopic image of the gastrointestinal tract, thereby determining the first position of the biopsy forceps in the first endoscopic image of the gastrointestinal tract.

[0097] Specifically, in the embodiment of the present application, the first gastrointestinal endoscopic image and the biopsy forceps template image can be compared to determine whether there is an image similar to or matching the biopsy forceps template image in the first gastrointestinal endoscopic image, so as to determine whether there is a biopsy forceps in the first gastrointestinal endoscopic image. If there is a biopsy forceps in the first gastrointestinal endoscopic image, the first position of the biopsy forceps in the first gastrointestinal endoscopic image can also be determined at the same time.

[0098] In some embodiments, the following formula may be used to determine whether a biopsy forceps is present in the first endoscopic image of the gastrointestinal tract. The formula is as follows:

[0099] R(x,y)=∑ x',y' (T(x',y')-I(x+x',y+y')) 2

[0100] Where T is the template image of the biopsy forceps, I is the first endoscopic image of the gastrointestinal tract to be matched, and R(x, y) is the residual sum of squares, that is, the degree of match between the two. T(x′, y′) is the value of the template image at the position (x′, y′). I(x+x′, y+y′) is the pixel value of the endoscopic image at the position (x+x′, y+y′). ∑x′, y′ represents the sum of all possible x′ and y′. According to the above formula, it can be determined whether there is a biopsy forceps in the first endoscopic image of the gastrointestinal tract.

[0101] 203. Predict a movement path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area according to the movement path.

[0102] Since the biopsy forceps need to be moved to the vicinity of the target lesion area before biopsy sampling, it is necessary to predict the movement path of the biopsy forceps from the current first position to the target lesion area, and control the biopsy forceps to move according to the predicted movement path until it reaches the target lesion area. The specific method of predicting the movement path of the biopsy forceps is described in detail in the subsequent embodiments and is not limited here.

[0103] 204. When the biopsy forceps is moved to the target lesion area, the biopsy forceps is used to take biopsy samples from the target lesion area.

[0104] When the biopsy forceps is moved to the target lesion area, biopsy sampling can be performed on the target lesion area. The specific method of sampling is described in detail in the subsequent embodiments and is not limited here.

[0105] The present application provides a biopsy sampling method, including obtaining a first endoscopic image of the gastrointestinal tract, and determining a target lesion area in the first endoscopic image of the gastrointestinal tract; determining a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; predicting a moving path of the biopsy forceps from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the moving path; after the biopsy forceps moves to the target lesion area, using the biopsy forceps to perform biopsy sampling on the target lesion area. The present application plans the moving path of the biopsy forceps in combination with the endoscopic image of the gastrointestinal tract, avoids repeated positioning and repeated sampling during the sampling process of the biopsy forceps, and improves the sampling accuracy and sampling efficiency.

[0106] like Figure 4 , is a flow chart of an embodiment of the predicted moving path provided in the embodiment of the present application. In the present application, the first position of the biopsy forceps currently located can be taken as the starting node, and the center of the target lesion area can be taken as the ending node. The ending node can be determined in advance. Predicting the moving path of the biopsy forceps from the starting node to the ending node can specifically include:

[0107] 401. Obtain a three-dimensional model of the gastrointestinal tract, and determine other nodes in the three-dimensional model except the starting node and the ending node.

[0108] 402. Taking the starting point node as the current node, determine the adjacent nodes around the current node.

[0109] 403. Calculate the heuristic function value between the current node and the adjacent node to determine the target adjacent node corresponding to the minimum heuristic function value.

[0110] 404. The target adjacent node is used as a new current node, and a new target adjacent node is determined until the target adjacent node is the end node or other nodes are traversed.

[0111] 405. Generate a moving path according to the starting node, the ending node, and a plurality of target adjacent nodes.

[0112] Specifically, since the biopsy forceps are used to sample the diseased area in the gastrointestinal tract, the moving position of the biopsy forceps cannot exceed the range of the gastrointestinal tract, so it is necessary to determine the range of the entire gastrointestinal tract. In some embodiments, the regional range of the gastrointestinal tract can be determined by constructing a three-dimensional model of the gastrointestinal tract. Specifically, a medical image of the gastrointestinal tract, such as CT, MRI or endoscopic images, can be first obtained, and these data include detailed information about the gastrointestinal tract; then, the medical image is segmented using image processing techniques including but not limited to threshold segmentation, region growing, level set and other methods to extract the gastrointestinal tract contour and the internal structure of the gastrointestinal tract. Then, the gastrointestinal tract is reconstructed in three dimensions based on the segmented structure to produce a three-dimensional model of the gastrointestinal tract.

[0113] When performing path planning, the three-dimensional model of the gastrointestinal tract can be discretized into a series of nodes or grid points, which will serve as potential exploration points in the path planning algorithm; the distribution and density of the nodes need to be adjusted according to the specific shape and size of the gastrointestinal tract. Since the biopsy forceps and the target lesion area are both located in the gastrointestinal tract, the starting node and the end node are also located in the three-dimensional model, that is, the starting node and the end node are two of the multiple nodes after the three-dimensional model is discretized; at the same time, the three-dimensional model also includes other nodes besides the starting node and the end node.

[0114] Since the biopsy forceps starts from the first position currently located, the starting point node corresponding to the first position can be used as the current node, and the path planning can be performed with the current node as the starting point. In this application, an open list and a closed list can be constructed respectively, the open list is used to store the nodes to be explored, and the closed list is used to store the explored nodes; before the path planning starts, the open list only includes the current node, and as the path planning proceeds, new nodes will be added to the open list for subsequent exploration; once the node is taken out of the open list and processed, the node will be added to the closed list to avoid subsequent repeated exploration.

[0115] In the path planning process, the current node is taken as the starting point, and then the adjacent nodes around the current node are determined and added to the open list for subsequent exploration. Then, the heuristic function values ​​between the current node and the adjacent nodes around it need to be calculated respectively, such as calculating the Euclidean distance between the current node and the adjacent nodes as the heuristic function value between the two; after calculating multiple distances, the target adjacent node with the smallest distance (or the smallest heuristic function value) needs to be selected from the multiple distances as the new current node.

[0116] The other nodes among the adjacent nodes except the target adjacent node will be removed from the development list and moved into the closed list; then it is necessary to further determine the adjacent nodes around the new current node, and recalculate the heuristic function value between the adjacent nodes around the new current node and the new current node. Similarly, the target adjacent node corresponding to the smallest heuristic function value is selected as the new current node again; repeat the above steps of determining the current node-calculating the heuristic function value-determining the new target adjacent node-taking the new target adjacent node as the new current node, until the target adjacent node is the terminal node, or all other nodes except the starting node and the terminal node are traversed. It should be noted that when determining the adjacent nodes above, it is necessary to ensure that the adjacent nodes are nodes located in the gastrointestinal tract, so as to ensure that the moving path is the path moving in the gastrointestinal tract. When the target adjacent node is the terminal node or after traversing all other nodes, the starting node-multiple target adjacent nodes and the terminal node can be connected in sequence to generate the moving path of the biopsy forceps.

[0117] On the basis of the above-mentioned embodiment, when controlling the biopsy forceps to move along the predicted moving path, it is necessary to obtain a new gastrointestinal endoscopic image in real time, so as to track the position of the target lesion area and the biopsy forceps in the new gastrointestinal endoscopic image, so as to track the movement of the biopsy forceps. According to the position of the target lesion area and the biopsy forceps in the new gastrointestinal endoscopic image, the moving path can be adjusted dynamically; and then the biopsy forceps is controlled to move from the first position to the target lesion area according to the new moving path. That is, in the embodiment of the present application, the moving path can be adjusted in real time according to the movement of the biopsy forceps to ensure that the biopsy forceps can be accurately moved to the target lesion area.

[0118] When the biopsy forceps moves to the target lesion area, a biopsy sample can be taken from the target lesion area. Specifically, when the biopsy forceps moves to the target lesion area, the image displayed on the display screen can prompt the biopsy forceps to take a sample, so as to control the biopsy forceps to take a first biopsy sample in the target lesion area, and mark the first sampling position of the first sampling after the sampling is completed. After the first biopsy sample is completed, the biopsy forceps can be controlled to move a preset distance in a preset direction to a second sampling position; the second sampling position needs to be located in the target lesion area. When the biopsy forceps moves to the second sampling position, the image displayed on the display screen can prompt the operator to rotate the biopsy forceps 180°, and after the rotation, it can be prompted again that a biopsy sample can be taken, and the biopsy forceps can be controlled to take a second sample.

[0119] Based on this, the present application also provides a biopsy forceps, which is used for biopsy sampling, and combined with endoscopic images, it can penetrate the gastrointestinal tract at one time, and can clamp two tissue samples at the same time, saving sampling inspection time, improving biopsy sampling clamping efficiency, and simplifying the biopsy sampling process. Figure 5 As shown, it is a schematic structural diagram of an embodiment of the biopsy forceps provided in the embodiment of the present application; Figure 6 As shown, it is a schematic diagram of the structure of the hinge joint of the biopsy forceps provided in the embodiment of the present application; Figure 7 The following is a schematic diagram of the chuck structure provided in the embodiment of the present application; Figure 5-7 The structure of the biopsy forceps in the present application is described in detail.

[0120] like Figure 5-7As shown, the biopsy forceps comprises: a fixed handle 1, and a ring finger ring 2 is provided inside the fixed handle to facilitate the operator's finger to be firmly inserted, so as to better operate the biopsy forceps. A hinge head 14 is fixedly connected to the top side of the fixed handle 1 to control the partial structure in the biopsy forceps to rotate around the hinge head 14 as the center, so as to facilitate sampling. A fixed shaft 3 is fixedly connected to the inner wall of the top end of the hinge head 14, and a rotating block 4 is rotatably connected to the outside of the fixed shaft 3, and the rotating block 4 is the center point of rotation. The side of the rotating block 4 away from the main pipe is also fixedly connected to the rotating handle 5, and the movement of the biopsy forceps can be controlled by the rotating block 4 and the fixed shaft 3. A hand-held support sheet 6 is fixedly connected to the right side of the rotating handle 5, and a secondary spring 7 is fixedly connected to the right side of the fixed handle 1, and the left side of the rotating handle 5 is fixedly connected to the main spring 8; the rotating handle 5 can be reset by alternating the main spring 8 and the secondary spring 7, and the top side of the rotating handle 5 is also fixedly connected to the auxiliary seat 9, and the second guide wheel 10 is fixedly connected to the inside of the auxiliary seat 9.

[0121] The left side of the fixed handle 1 is fixedly connected with two arc-shaped plates 12 arranged opposite to each other, and the interior of the two arc-shaped plates 12 is fixedly connected with a fixed rod 13, which is used to fix one end of a steel rope 16. The bottom side of the main pipe 17 is fixedly connected with a turning pipe 18, so that the biopsy forceps are suitable for different anatomical structures. The bottom side of the turning pipe 18 is fixedly connected with a sub-tube 19, and the outer side of the sub-tube 19 away from the turning pipe 18 is provided with two main clamps 20 arranged opposite to each other, which are the first clamp and the second clamp, respectively, and are used to hold the tissue when clamping the tissue. The inner wall of the sub-tube 19 is slidably connected with a steel rope 16, which is used to transmit the control action.

[0122] like Figure 8 As shown, it is a schematic diagram of the structure of the limiting ring of the biopsy forceps provided in the embodiment of the present application; Fig. 9 As shown, the embodiment of the present application provides Figure 6 The enlarged schematic diagram of B in FIG. Fig.10 FIG. 1 is a schematic diagram of the structure of the square plate in the biopsy forceps provided in the embodiment of the present application. Figure 5-Figure 10 The structure of the biopsy forceps in this application is described in detail. Figure 5-Figure 10The bottom side of the steel rope 16 is fixedly connected with a sliding rod 21, and the movement of the sliding rod 21 is controlled by the steel rope 16. The bottom end of the inner wall of the auxiliary tube 19 is fixedly connected with a limiting ring 22 for limiting the sliding rod 21. The outer sleeve of the sliding rod 21 is provided with a spring 23 to limit the spring 23 so that the spring 23 can be evenly stressed. The inner wall of the bottom end of the auxiliary tube 19 is fixedly connected with a circular plate 24, thereby driving the circular plate 24 to slide to squeeze the spring 23. The limiting ring 22 and the near side of the circular plate 24 are respectively fixedly connected to the far side of the spring 23. The outer side of the sliding rod 21 is slidably connected to the inner wall of the limiting ring 22, so that the sliding rod 21 can slide vertically. The bottom end of the sliding rod 21 is fixedly connected to the inner wall of the circular plate 24. The sliding rod 21 is provided with a sliding assembly, which includes a strip plate 25 for providing a supporting force. The top side of the strip plate 25 is fixedly connected to the bottom side of the sliding rod 21 to guide the movement of the sliding assembly.

[0123] The left and right sides of the strip plate 25 are fixedly connected with sliding rods 26, and the left and right ends of the sliding assembly are slidably connected with slave clamps 29. The outer side of the sliding rod 26 is slidably connected to the inner wall of the slave clamp 29 to ensure the smooth rotation of the slave clamp 29. The adjacent side of the two main clamps 20 is in contact with the distant side of the two slave clamps 29 for clamping tissues. The distant ends of the strip plate 25 are respectively slidably connected to the inner walls of the two slave clamps 29. The interior of the slave clamp 29 is provided with a T-shaped opening 30 for limiting the strip plate 25. The sliding assembly is used to push the slave chuck 29 to rotate, and the top end of the slave chuck 29 is rotatably connected to the front and rear ends of the fixed rod 28, so that the slave chuck 29 can rotate stably. The adjacent sides of the two main chucks 20 are fixedly connected with a connecting block 27, and the outside of the connecting block 27 is fixedly connected with a fixed rod 28. The adjacent side of the top of the hinged head 14 is fixedly connected with the main seat 15, and the inner wall of the main seat 15 is fixedly connected with the first guide wheel 11 for guiding the sliding of the steel rope 16.

[0124] The left end of the bottom side of the main seat 15 is fixedly connected to a connecting plate 31, and the left side of the connecting plate 31 is fixedly connected to a side block 32 to provide support force for the side block 32. The inner wall of the side block 32 is rotatably connected to a telescopic rod 33 to adapt to different angles. The top of the telescopic rod 33 is rotatably connected to a concave plate 34 to provide support force for the telescopic rod 33. The top side of the concave plate 34 is fixedly connected to an opening block 35. The front and rear ends of the opening block 35 are both provided with pushing components. A plurality of springs 36 are fixedly connected to the far sides of the two pushing components. The springs 36 are fixed so that the springs 36 can be evenly stressed. The pushing components include a clamping block 38, and the two The outsides of the two clamping blocks 38 are respectively slidably connected to the inner walls at the front and rear ends of the opening block 35. The clamping blocks 38 are restricted by the opening block 35 so that they can slide horizontally. The far sides of the two clamping blocks 38 are fixedly connected with square plates 37 for restricting the clamping blocks 38 and preventing them from slipping. The outsides of the two square plates 37 are respectively slidably connected to the inner walls at the front and rear ends of the opening block 35. The square plates 37 are restricted by the opening block 35 so that they can slide horizontally. The adjacent sides of the two clamping blocks 38 are in contact with the outside of the main pipe 17 to fix and restrict the main pipe 17. The pushing assembly is used to squeeze the spring 2 36 for compression.

[0125] When using the biopsy forceps for biopsy sampling, first insert the pipeline into the trachea, then guide the main pipe 17 into the pipeline, and push the main clamp 20 to the position where the tissue is required to be clamped. At this time, by holding the annular finger ring 2, the rotating handle 5 is pushed to drive the rotating block 4 to rotate with the fixed axis 3 as the origin. During the rotation of the rotating handle 5, the main shrapnel 8 is driven to press against the auxiliary shrapnel 7 for compression, and the main shrapnel 8 will be reset after being free from pressure, and the rotating handle 5 will drive the auxiliary seat 9 to slide during the rotation, thereby driving the second guide wheel 10 to rotate. At this time, the second guide wheel 10 will push the steel rope 16 to slide, and one end of the steel rope 16 is fixed to the fixed rod 13 to , and the other is fixed to the sliding rod 21, so that the second guide wheel 10 can pull the steel rope 16 to slide, and then the hand holding support piece 6 will drive the sliding rod 21 to slide and drive the circular plate 24 to slide. At this time, the circular plate 24 will squeeze the spring 1 23 so that the spring 1 23 can store elastic potential energy. At the same time, the sliding rod 21 will also drive the strip plate 25 to slide, thereby driving the sliding rod 26 to slide, and then push the two slave clamps 29 to slide to both sides with the fixed rod 28 as the center point, and then the two slave clamps 29 will rotate respectively to fit with the two main clamps 20 to complete the clamping of tissues at two positions, thereby avoiding insufficient clamping of tissues and the need to clamp again, thereby improving the clamping efficiency.

[0126] And there will be patients of different ages, and the lengths of their own tracheas are different. At this time, the main tube 17 will have an extra section in a suspended state. At this time, the medical staff needs to form an uncomfortable posture to keep the extra section of the main tube 17 in a horizontal state. At this time, the main tube 17 is clamped on the adjacent side of the two clamping blocks 38. At this time, the clamping block 38 will slide under the force, and then push the square plate 37 to slide and squeeze the spring 2 36, so that the spring 2 36 can be stored, and then give the clamping block 38 a reverse force to fix the main tube 17. According to the different lengths of the extra main tube 17, by pushing the opening block 35 to slide, the concave plate 34 is driven to slide, and the telescopic rod 33 is driven to telescope and rotate, so that the opening block 35 can always support the middle part of the extra section of the main tube 17, so that the staff does not need to maintain a posture so that the main tube 17 can always be in a horizontal state, thereby improving the working comfort.

[0127] In the present application, by combining the biopsy forceps with the endoscopic image, the lesion area and the position where the biopsy clamp needs to be taken can be better located, and repeated positioning and repeated sampling can be avoided, thereby improving the accuracy and efficiency of the biopsy. At the same time, after the biopsy forceps are controlled to move to the target lesion area, the biopsy forceps can be controlled to take the first sample in the target lesion area; and since the biopsy forceps provided in the present application have a first chuck and a second chuck, the first chuck in the biopsy forceps can be controlled to take the first sample in the target lesion area. When the biopsy forceps is controlled to move a preset distance in a preset direction to the second sampling position, the biopsy forceps can be controlled to rotate at the second sampling position, so that the second sampling is performed using the rotated second chuck. That is, in the present application, the biopsy forceps can be used to take samples twice at the same time, and there is no need to take out the biopsy forceps after sampling once and then go deep into the lesion area for a second clamping, which can further improve the clamping efficiency.

[0128] In order to facilitate better implementation of the biopsy sampling method provided in the embodiment of the present application, the embodiment of the present application also provides a biopsy sampling device based on the above biopsy sampling method. The meanings of the terms are the same as those in the above biopsy sampling method. For specific implementation details, please refer to the description in the above method embodiment.

[0129] Please refer to Fig.11 , Fig.11 A schematic diagram of the structure of a biopsy sampling device provided in an embodiment of the present application, the biopsy sampling device may include:

[0130] The lesion region determination module 1101 is used to obtain a first endoscopic image of the gastrointestinal tract and determine a target lesion region in the first endoscopic image of the gastrointestinal tract.

[0131] The positioning module 1102 is used to determine a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract.

[0132] The path prediction module 1103 is used to predict the moving path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area according to the moving path.

[0133] The sampling module 1104 is used to perform biopsy sampling on the target lesion area using the biopsy forceps after the biopsy forceps is moved to the target lesion area.

[0134] The present application provides a biopsy sampling device, which can obtain a first endoscopic image of the gastrointestinal tract, and determine a target lesion area in the first endoscopic image of the gastrointestinal tract; determine a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; predict a moving path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area according to the moving path; after the biopsy forceps moves to the target lesion area, the biopsy forceps is used to perform a biopsy sampling of the target lesion area. The present application plans the moving path of the biopsy forceps in combination with the endoscopic image of the gastrointestinal tract, avoids repeated positioning and repeated sampling during the sampling process of the biopsy forceps, and improves the sampling accuracy and sampling efficiency.

[0135] In some embodiments, the lesion area determination module 1101 may be specifically used to:

[0136] Acquiring the first endoscopic image of the gastrointestinal tract, and performing edge detection on the first endoscopic image of the gastrointestinal tract to determine an initial lesion area of ​​the first endoscopic image of the gastrointestinal tract;

[0137] Performing shape positioning analysis on the initial lesion area to determine the shape positioning lesion area of ​​the gastrointestinal endoscopic image;

[0138] Growing the shape-located lesion region using a preset region growing algorithm to obtain a growing lesion region;

[0139] An initial contour line of the growing lesion area is determined, and a target contour line of the growing lesion area is determined based on a preset contour line evolution strategy and the initial contour line, wherein the area enclosed by the target contour line is the target lesion area.

[0140] In some embodiments, the positioning module 1102 may be specifically used to:

[0141] Determining whether a preset biopsy forceps exists in the first endoscopic image of the gastrointestinal tract;

[0142] If the biopsy forceps exists in the first gastrointestinal endoscopic image, obtaining a preset biopsy forceps template image;

[0143] The first endoscopic image of the gastrointestinal tract and the biopsy forceps template image are matched to determine a first position of the biopsy forceps in the first endoscopic image of the gastrointestinal tract.

[0144] In some embodiments, the first position is used as the starting point node, and the center of the target lesion area is used as the ending point node; the path prediction module 1103 can be specifically used to:

[0145] Acquire the three-dimensional model of the gastrointestinal tract, and determine other nodes in the three-dimensional model except the starting node and the end node;

[0146] Taking the starting point node as the current node, determining the adjacent nodes around the current node;

[0147] Calculating the heuristic function value between the current node and the adjacent node to determine the target adjacent node corresponding to the minimum heuristic function value;

[0148] Taking the target adjacent node as a new current node, determining a new target adjacent node until the target adjacent node is the end node or traversing the other nodes;

[0149] The moving path is generated according to the starting point node, the end point node and a plurality of target adjacent nodes.

[0150] In some embodiments, the sampling module 1104 may be specifically used to:

[0151] When the biopsy forceps moves to the target lesion area, the biopsy forceps is controlled to perform a first sampling in the target lesion area, and a first sampling position of the first sampling is marked after the sampling is completed;

[0152] Controlling the biopsy forceps to move a preset distance in a preset direction to a second sampling position, where the second sampling position is located in the target lesion area;

[0153] The biopsy forceps is controlled to rotate at the second sampling position, and a second sampling is performed after the rotation.

[0154] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the processor is used to execute the steps of the biopsy sampling method provided in this embodiment by calling a computer program stored in the memory.

[0155] Please refer to Fig.12 , Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0156] The electronic device may include components such as a processor 1201 with one or more processing cores, a memory 1202 with one or more computer-readable storage media, a power supply 1203, and an input unit 1204. Those skilled in the art will appreciate that Fig.12 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] The processor 1201 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 1202, and calling data stored in the memory 1202, the processor 1201 performs various functions of the electronic device and processes data. Optionally, the processor 1201 may include one or more processing cores; optionally, the processor 1201 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1201.

[0158] The memory 1202 can be used to store software programs and modules. The processor 1201 executes various functional applications and data processing by running the software programs and modules stored in the memory 1202. The memory 1202 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1202 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1202 may also include a memory controller to provide the processor 1201 with access to the memory 1202.

[0159] The electronic device also includes a power supply 1203 for supplying power to each component. Optionally, the power supply 1203 can be logically connected to the processor 1201 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1203 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0160] The electronic device may further include an input unit 1204, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0161] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically in this embodiment, the processor 1201 in the electronic device will load the executable code corresponding to one or more computer programs into the memory 1202 according to the following instructions, and the processor 1201 will execute the steps in the biopsy sampling method provided in this application, such as:

[0162] Acquiring a first endoscopic image of the gastrointestinal tract and determining a target lesion area in the first endoscopic image of the gastrointestinal tract;

[0163] Determining a first position of a preset biopsy forceps in a first endoscopic image of the gastrointestinal tract;

[0164] Predicting a movement path of the biopsy forceps from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the movement path;

[0165] When the biopsy forceps are moved to the target lesion area, the biopsy forceps are used to take biopsy samples from the target lesion area.

[0166] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0167] To this end, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. The computer program is loaded by a processor to execute the steps in any one of the biopsy sampling methods provided in the embodiments of the present application.

[0168] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0169] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. Since the computer program stored in the computer-readable storage medium can execute the steps in the biopsy sampling method provided in the embodiment of the present application, the beneficial effects that can be achieved by the biopsy sampling method provided in the embodiment of the present application can be achieved, as detailed in the previous embodiments, which will not be repeated here.

[0170] When the computing device in the embodiment of the present application is a terminal device, the embodiment of the present application also provides a terminal device, such as Fig.13 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal device can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the mobile phone as an example:

[0171] Fig.13 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.13 The mobile phone includes: a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390. Those skilled in the art will understand that Fig.13 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0172] Combine the following Fig.13 A detailed introduction to the various components of the mobile phone:

[0173] The RF circuit 1310 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1380 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1310 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 1310 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0174] The memory 1320 can be used to store software programs and modules. The processor 1380 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1320. The memory 1320 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0175] The input unit 1330 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332. The touch panel 1331, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1331 or near the touch panel 1331 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1331 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1380, and can receive and execute commands sent by the processor 1380. In addition, the touch panel 1331 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1331, the input unit 1330 may also include other input devices 1332. Specifically, other input devices 1332 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0176] The display unit 1340 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1340 may include a display panel 1341. Optionally, the display panel 1341 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1331 may cover the display panel 1341. When the touch panel 1331 detects a touch operation on or near it, it is transmitted to the processor 1380 to determine the type of touch event. Subsequently, the processor 1380 provides a corresponding visual output on the display panel 1341 according to the type of touch event. Although in Fig.13 In the embodiment, the touch panel 1331 and the display panel 1341 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1331 and the display panel 1341 can be integrated to realize the input and output functions of the mobile phone.

[0177] The mobile phone may also include at least one sensor 1350, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1341 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1341 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0178] The audio circuit 1360, the speaker 1361, and the microphone 1362 can provide an audio interface between the user and the mobile phone. The audio circuit 1360 can transmit the received audio data to the speaker 1361 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1362 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1360 and converted into audio data, and then the audio data is output to the processor 1380 for processing, and then sent to another mobile phone through the RF circuit 1310, or the audio data is output to the memory 1320 for further processing.

[0179] Wi-Fi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the Wi-Fi module 1370, which provides users with wireless broadband Internet access. Fig.13 A Wi-Fi module 1370 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0180] The processor 1380 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1320, and calling data stored in the memory 1320, so as to monitor the mobile phone as a whole. Optionally, the processor 1380 may include one or more processing units; optionally, the processor 1380 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1380.

[0181] The mobile phone also includes a power supply 1390 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 1380 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.

[0182] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0183] In the embodiment of the present application, the processor 1380 included in the mobile phone also has the function of controlling the execution of the above biopsy sampling method process performed by the biopsy sampling device.

[0184] The present application also provides a server. Fig.14 , Fig.14 14 is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (full name in English: central processing units, abbreviated in English: CPU) 1422 (for example, one or more processors) and a memory 1432, and one or more storage media 1430 (for example, one or more mass storage devices) storing application programs 1442 or data 1444. Among them, the memory 1432 and the storage medium 1430 can be short-term storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1422 can be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the server 1400.

[0185] The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1158, and / or one or more operating systems 1441, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0186] The steps in the biopsy sampling method in the above embodiment can be based on the Fig.14 The structure of the server 1400 is shown. For example, the central processor 1422 performs the following operations by calling the instructions in the memory 1432:

[0187] Acquire a first endoscopic image of the gastrointestinal tract, and determine a target lesion area in the first endoscopic image of the gastrointestinal tract; determine a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; predict a moving path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area along the moving path; when the biopsy forceps moves to the target lesion area, use the biopsy forceps to perform biopsy sampling on the target lesion area.

[0188] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0189] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0190] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0191] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0193] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0194] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0195] The technical solutions provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the embodiments of the present application are explained by using specific examples in the embodiments of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the embodiments of the present application. At the same time, for those skilled in the art, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

[0196] It should be noted that when the above embodiments of the present application are applied to specific products or technologies, the relevant data of the user is involved, and the user's permission or consent is required, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A biopsy sampling method, characterized in that: The method comprises: Acquiring a first endoscopic image of the gastrointestinal tract, and determining a target lesion area in the first endoscopic image of the gastrointestinal tract; Determining a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; Predicting a movement path of the biopsy forceps from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the movement path; When the biopsy forceps is moved to the target lesion area, the biopsy forceps is used to perform biopsy sampling on the target lesion area.

2. The biopsy sampling method according to claim 1, characterized in that: The step of acquiring a first endoscopic image of the gastrointestinal tract and determining a target lesion area in the first endoscopic image of the gastrointestinal tract includes: Acquiring the first endoscopic image of the gastrointestinal tract, and performing edge detection on the first endoscopic image of the gastrointestinal tract to determine an initial lesion area of ​​the first endoscopic image of the gastrointestinal tract; Performing shape positioning analysis on the initial lesion area to determine the shape positioning lesion area of ​​the gastrointestinal endoscopic image; Growing the shape-located lesion region using a preset region growing algorithm to obtain a growing lesion region; An initial contour line of the growing lesion area is determined, and a target contour line of the growing lesion area is determined based on a preset contour line evolution strategy and the initial contour line, wherein the area enclosed by the target contour line is the target lesion area.

3. The biopsy sampling method according to claim 1, characterized in that: The step of determining a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract includes: Determining whether a preset biopsy forceps exists in the first endoscopic image of the gastrointestinal tract; If the biopsy forceps exists in the first gastrointestinal endoscopic image, obtaining a preset biopsy forceps template image; The first endoscopic image of the gastrointestinal tract and the biopsy forceps template image are matched to determine a first position of the biopsy forceps in the first endoscopic image of the gastrointestinal tract.

4. The biopsy sampling method according to claim 1, characterized in that: Taking the first position as a starting point node and the center of the target lesion area as an ending point node; The predicting a moving path of the biopsy forceps from the first position to the target lesion area, and controlling the biopsy forceps to move from the first position to the target lesion area according to the moving path, comprises: Acquire the three-dimensional model of the gastrointestinal tract, and determine other nodes in the three-dimensional model except the starting node and the end node; Taking the starting point node as the current node, determining the adjacent nodes around the current node; Calculating the heuristic function value between the current node and the adjacent node to determine the target adjacent node corresponding to the minimum heuristic function value; Taking the target adjacent node as a new current node, determining a new target adjacent node until the target adjacent node is the end node or traversing the other nodes; The moving path is generated according to the starting point node, the end point node and a plurality of target adjacent nodes.

5. The biopsy sampling method according to claim 1, characterized in that: After the biopsy forceps is moved to the target lesion area, the biopsy forceps is used to perform biopsy sampling on the target lesion area, including: When the biopsy forceps moves to the target lesion area, the biopsy forceps is controlled to perform a first sampling in the target lesion area, and a first sampling position of the first sampling is marked after the sampling is completed; Controlling the biopsy forceps to move a preset distance in a preset direction to a second sampling position, where the second sampling position is located in the target lesion area; The biopsy forceps is controlled to rotate at the second sampling position, and a second sampling is performed after the rotation.

6. The biopsy sampling method according to claim 1, characterized in that: The method further comprises: In the process of the biopsy forceps moving along the moving path, new gastrointestinal endoscopic images are continuously acquired; Tracking the positions of the target lesion area and the biopsy forceps in the new gastrointestinal endoscopic image; Dynamically adjusting the moving path according to the positions of the target lesion area and the biopsy forceps in the new gastrointestinal endoscopic image to obtain an adjusted new moving path; The biopsy forceps is controlled to move from the first position to the target lesion area according to a new moving path.

7. The biopsy sampling method according to claim 1, characterized in that: Before acquiring the first endoscopic image of the gastrointestinal tract, the method further includes: Acquire an initial endoscopic image of the gastrointestinal tract, and convert the initial endoscopic image of the gastrointestinal tract into a grayscale endoscopic image; Noise in the grayscale endoscopic image is removed to obtain the first endoscopic image of the gastrointestinal tract.

8. A biopsy sampling device, characterized in that: The device comprises: A lesion region determination module, used to obtain a first endoscopic image of the gastrointestinal tract and determine a target lesion region in the first endoscopic image of the gastrointestinal tract; A positioning module, used to determine a first position of a preset biopsy forceps in the first endoscopic image of the gastrointestinal tract; a path prediction module, used to predict a moving path of the biopsy forceps from the first position to the target lesion area, and control the biopsy forceps to move from the first position to the target lesion area according to the moving path; The sampling module is used to perform biopsy sampling on the target lesion area using the biopsy forceps after the biopsy forceps is moved to the target lesion area.

9. A biopsy forceps, characterized in that: The biopsy forceps comprises: A handle operating device and a main pipe, one end of the main pipe is connected to the handle operating device, and the other end of the main pipe is connected to the auxiliary pipe; A sliding rod is fixed on the side of the auxiliary pipe away from the main pipe, and a first clamp and a second clamp oppositely arranged are arranged on the outer side of the auxiliary pipe away from the bending pipe. The first clamp and the second clamp are connected to the sliding rod.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps of the biopsy sampling method according to any one of claims 1 to 7.

Images