Stomach tumor cell image extraction method and system

Abstract

The invention discloses a stomach tumor cell image extraction method and system, and the method comprises the steps: extracting continuous frames from a gastroscope video, carrying out the preprocessing, carrying out the positioning of a tumor cell region in a stomach image through a medical segmentation network, and extracting dynamic features including optical flow data, texture features and motion trails; predicting local displacement caused by stomach peristalsis based on a long short-term memory network, and constructing an affine transformation matrix to compensate rigid motion to obtain an image sequence subjected to motion correction; gradient direction consistency optimization is carried out on the segmentation network by embedding a deformable convolution and attention mechanism composite layer, so that the edge and details of a tumor cell region are accurately extracted and remarkably enhanced; according to the method, interference of non-rigid movement of the stomach on image segmentation and feature extraction can be effectively reduced, the tumor cell image which is accurate in positioning and clear in edge is obtained, high-precision recognition and extraction of the stomach tumor cell area are achieved, and reliable image support is provided for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of gastric tumor cell image processing, and particularly to a method and system for extracting gastric tumor cell images. Background Art

[0002] With the rapid development of medical imaging technology, gastroscopy has become one of the important means for diagnosing gastric diseases. In recent years, medical image segmentation technology based on deep learning has gradually emerged. Especially the application of deep learning models such as convolutional neural network (CNN) and long short-term memory network (LSTM) has made remarkable progress in the automatic recognition and extraction of tumor regions in gastroscopy images. However, due to the physiological characteristics of the stomach, the gastric tissue will produce complex non-rigid movements during the gastroscopy examination due to peristalsis, resulting in obvious local displacements and deformations between consecutive frame images. This kind of movement not only reduces the positioning accuracy of the tumor cell region, but also causes error accumulation in the training process of the segmentation network, thus affecting the accuracy and clarity of tumor cell image extraction.

[0003] Existing gastroscopy image processing methods usually adopt traditional rigid registration methods to compensate for the movement caused by gastric peristalsis. However, these methods often ignore the multi-scale characteristics and local non-rigid deformations of gastric peristaltic movement, and it is difficult to effectively locate the tumor cell region in gastroscopy images and extract dynamic features. In addition, traditional medical image segmentation networks usually only focus on pixel-level loss functions during the training process, while ignoring the consistency of image gradient directions, resulting in blurred or discontinuous phenomena in the tumor cell edge region of the segmentation results, which cannot meet the strict requirements of clinical diagnosis for the clarity of tumor cell images.

[0004] Therefore, how to effectively compensate for the local displacement caused by gastric peristalsis and improve the clarity of tumor cell images has become an urgent technical problem to be solved in the current field of gastroscopy image processing. Summary of the Invention

[0005] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this part, the abstract, and the title. However, such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a method for extracting gastric tumor cell images to solve the problems mentioned in the background art.

[0007] To solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a method for extracting gastric tumor cell images, including:

[0009] Extract consecutive frames from a gastroscopy video, preprocess the consecutive frames, use a medical segmentation network to locate the gastric tumor cell region, and extract the dynamic features of the gastric tumor cell region;

[0010] According to the dynamic features, predict the local displacement caused by gastric peristalsis, and construct an affine transformation matrix to compensate for the rigid motion in the local displacement, so as to obtain a compensated gastric image containing tumor cells;

[0011] Update the medical segmentation network, optimize the medical segmentation network through gradient direction consistency, and extract clear tumor cell images.

[0012] As a preferred solution of the method for extracting gastric tumor cell images according to the present invention, wherein: extracting consecutive frames from a gastroscopy video, extracting consecutive frames from a gastroscopy video, and preprocessing the consecutive frames, using a medical segmentation network to locate the gastric tumor cell region, includes:

[0013] Take the gastroscopy video as a video frame sequence, and extract a consecutive frame sequence from the video frame sequence at a fixed frame rate;

[0014] Based on the medical segmentation network, use a pre-trained segmentation model, input each frame of the gastric image in the consecutive frame sequence into the model, and obtain the position of the tumor cell region.

[0015] As a preferred solution of the method for extracting gastric tumor cell images according to the present invention, wherein: extracting the dynamic features of the gastric tumor cell region includes:

[0016] Take out the gastric images of two adjacent frames from the consecutive frame sequence, obtain the displacement vector of the pixels in each gastric image, and generate an optical flow map;

[0017] According to the position of the tumor cell region, extract the optical flow data belonging to the position of the tumor cell region from the optical flow map, and obtain the mean and variance of the optical flow data of the tumor cell region.

[0018] As a preferred solution of the method for extracting gastric tumor cell images according to the present invention, wherein: further includes:

[0019] For the tumor cell region of each frame of the gastric image, convert the gastric image into a grayscale image, calculate the gray-level co-occurrence matrix of the tumor cell region, extract the contrast, correlation, energy, and homogeneity in the gray-level co-occurrence matrix, and form a time series;

[0020] Calculate the average motion vector of the tumor cell region based on the optical flow data, accumulate the motion vectors of consecutive frames to generate the motion trajectory of the tumor region, and calculate the displacement and velocity of the motion trajectory;

[0021] Integrate the motion trajectory with the formed time series to obtain the dynamic characteristics of the gastric tumor cell region.

[0022] As a preferred embodiment of the gastric tumor cell image extraction method of the present invention, wherein: predicting the local displacement caused by gastric peristalsis according to the dynamic characteristics, including:

[0023] Design a three-layer long short-term memory network, use the dynamic characteristics of the gastric tumor cell region as the input of the long short-term memory network, and obtain the displacement vector of each local region in the next frame, that is, the local displacement, through the long short-term memory network.

[0024] As a preferred embodiment of the gastric tumor cell image extraction method of the present invention, wherein: constructing an affine transformation matrix to compensate for the rigid motion in the local displacement to obtain a compensated gastric image containing tumor cells, including:

[0025] Convert the local displacement into an affine transformation matrix, and apply the corresponding affine transformation matrix to each local region, so that the pixel points in the gastric image are transformed from the original position to the compensated position.

[0026] As a preferred embodiment of the gastric tumor cell image extraction method of the present invention, wherein: updating the medical segmentation network, optimizing the medical segmentation network through gradient direction consistency, and extracting clear tumor cell images, including:

[0027] Embed a composite layer that fuses deformable convolution and attention mechanism in the encoder and decoder of the medical segmentation network. The deformable convolution adaptively adjusts the receptive field by learning the offset, and the attention mechanism calculates the importance weight of the tumor cell region;

[0028] Calculate the consistency between the gradient of the compensated tumor cell image and the gradient direction of the original tumor cell image, and consider the changes between adjacent image pixels to establish a smoothness constraint.

[0029] In a second aspect, the present invention provides a gastric tumor cell image extraction system, which includes:

[0030] A gastric tumor cell region extraction module, configured to extract consecutive frames from a gastroscopy video, preprocess the consecutive frames, and use a medical segmentation network to locate the gastric tumor cell region and extract the dynamic characteristics of the gastric tumor cell region;

[0031] A gastric image compensation module, configured to predict local displacements caused by gastric peristalsis according to the dynamic features, and construct an affine transformation matrix to compensate for the rigid motion in the local displacements, so as to obtain a compensated gastric image containing tumor cells;

[0032] A gastric tumor cell image output module, configured to update a medical segmentation network, optimize the medical segmentation network through gradient direction consistency, and extract clear tumor cell images.

[0033] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the processor executes the computer program, any step of the above method is implemented.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by a processor, any step of the above method is implemented.

[0035] Compared with the prior art, the beneficial effects of the invention are as follows:

[0036] 1. By extracting consecutive frames from a gastroscopy video and performing preprocessing, and using a medical segmentation network to accurately locate the gastric tumor cell region and extract its dynamic features, the present invention effectively solves the problem of positioning errors caused by non-rigid motion caused by gastric peristalsis;

[0037] 2. By analyzing the dynamic features to predict local displacements caused by gastric peristalsis and constructing an affine transformation matrix to compensate for the rigid motion in the local displacements, it is possible to eliminate image displacements and deformations caused by gastric peristalsis, overcome the limitations of traditional rigid registration methods that ignore multi-scale features and local non-rigid deformations, and obtain more stable and clear gastric tumor cell images;

[0038] 3. By optimizing the medical segmentation network through gradient direction consistency and paying attention to the gradient direction features of the tumor cell region, the present invention avoids the problem of blurred or discontinuous tumor edges caused by only focusing on pixel-level losses in traditional methods, and enables the quality of the extracted images to meet the strict requirements of clinical diagnosis for high definition and accuracy. Description of the Drawings

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts. Among them:

[0040] Figure 1The overall flowchart of the method for extracting gastric tumor cell images according to an embodiment of the present invention. Detailed implementation manners

[0041] To make the above objects, features and advantages of the present invention more apparent and understandable, the following detailed description of the specific implementation manners of the present invention will be made with reference to the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] In the following description, many specific details are set forth to facilitate a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0043] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure or characteristic that can be included in at least one implementation manner of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0044] The present invention will be described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for the convenience of explanation, the cross-sectional views showing the device structure will be enlarged locally out of the general scale, and the schematic diagrams are only examples, which should not limit the protection scope of the present invention herein. In addition, in actual production, three-dimensional spatial dimensions including length, width and depth should be included.

[0045] Meanwhile, in the description of the present invention, it should be noted that the orientation or positional relationships indicated by terms such as "upper, lower, inner and outer" are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0046] Unless otherwise clearly defined and limited in the present invention, the terms "installation, connection and coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, and can also be indirectly connected through an intermediate medium, or can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0047] Example 1

[0048] Reference Figure 1 , which is the first embodiment of the present invention. This embodiment provides a method for extracting gastric tumor cell images, including:

[0049] S1. Extract consecutive frames from the gastroscopy video, preprocess the consecutive frames, use a medical segmentation network to locate the gastric tumor cell region, and extract the dynamic features of the gastric tumor cell region;

[0050] Specifically, obtain the gastroscopy video of the patient by operating the gastroscopy device. The gastroscopy device consists of a gastroscopy probe, a control system, and a display device; the gastroscopy probe is a slender tube with a camera and a lighting device at its end, and the camera has a recording function; insert the gastroscopy probe into the patient's body, adjust the position and angle of the gastroscopy probe through the control system, and use the lighting device to view the dynamic image inside the patient's stomach in real time; in addition, to reduce the discomfort of the patient and retain the patient's personalized data, the present invention saves the real-time dynamic image inside the patient's stomach as a standard video format (such as MP4 or AVI) by turning on the recording function of the camera and stores it in the personal database of the corresponding patient;

[0051] It should be noted that the resolution and frame rate of the saved gastroscopy video will vary depending on the camera model. For example, using a high-definition camera can provide higher image quality, but the corresponding equipment cost will increase;

[0052] Furthermore, take the gastroscopy video as a video frame sequence, and extract a consecutive frame sequence from the video frame sequence at a fixed frame rate (such as 30 frames per second) to ensure the continuity of each frame of the gastric image at multiple time points;

[0053] Specifically, based on the medical segmentation network, use a pre-trained segmentation model that has been trained on a gastroscopy image dataset with tumor annotations. Input each frame of the gastric image into the model, and the model will output a mask corresponding to the input gastric image and mark the position of the tumor cell region;

[0054] Specifically, the medical segmentation network includes a medical segmentation network and a discriminator network;

[0055] Furthermore, take out the gastric images of two adjacent frames from the consecutive frame sequence, obtain the displacement vector of the pixels in each gastric image, and generate an optical flow map;

[0056] Specifically, the optical flow map V(x, y, t) can be expressed as:

[0057] V(x, y, t) = [u(x, y, t), v(x, y, t)]

[0058] Wherein, x and y respectively represent pixel coordinates, t represents a frame index, u represents the displacement vector of the pixel in the horizontal direction, with the unit of pixel; v represents the displacement vector of the pixel in the vertical direction;

[0059] Furthermore, according to the position of the tumor cell region, optical flow data belonging to the tumor cell region is extracted from the optical flow map, and the mean and variance of the optical flow data at the position of the tumor cell region are obtained;

[0060] Specifically, the mean of the optical flow data of the tumor cell region is expressed as:

[0061]

[0062] Wherein, F is the target (tumor cell) region, N is the number of pixels in the target region, μ u represents the average displacement of the displacement vector in the horizontal direction, μ v represents the average displacement of the displacement vector in the vertical direction;

[0063] Specifically, the variance of the optical flow data of the tumor cell region is expressed as:

[0064]

[0065] Wherein, represents the variance of the displacement vector in the horizontal direction, represents the variance of the displacement vector in the vertical direction;

[0066] It should be noted that this process can visualize the optical flow data as a color map to check whether the pixel displacement in the optical flow map is reasonable. For example, the length of the displacement vector is represented by brightness, and the brighter it is, the faster the movement; by calculating the optical flow between adjacent frames, the movement characteristics of the tumor cell region can be extracted;

[0067] Further, for the tumor cell region of each frame of the gastric image, its gray value is extracted (the gastric image is converted into a gray image I), the gray level co-occurrence matrix of the tumor cell region is calculated, and the contrast, correlation, energy, and homogeneity in the gray level co-occurrence matrix are extracted to form a time series;

[0068] It should be noted that before calculating the gray level co-occurrence matrix of the tumor cell region, a direction (angle) and a distance (in pixels) need to be selected to statistically calculate the co-occurrence frequency of the gray values of pixel pairs in the tumor cell region;

[0069] Specifically, the calculation formula for extracting the contrast Ct of the gray level co-occurrence matrix is:

[0070] Ct = ∑ i,j (i - j) 2 p(i, j)

[0071] Specifically, the calculation formula for extracting the correlation degree Cr of the gray - level co - occurrence matrix is:

[0072]

[0073] Specifically, the calculation formula for extracting the energy En of the gray - level co - occurrence matrix is:

[0074] En = ∑ i,j p(i, j) 2

[0075] Specifically, the calculation formula for extracting the homogeneity Hg of the gray - level co - occurrence matrix is:

[0076]

[0077] where p(i, j) is an element of the gray - level co - occurrence matrix, and μ and σ are the mean and standard deviation respectively;

[0078] Specifically, the time series L is expressed as:

[0079] L = {Ct, Cr, En, Hg}

[0080] It should be noted that by extracting the eigenvalues in the gray - level co - occurrence matrix, the changing trend of the texture on the surface of tumor cells over time can be reflected; in addition, although the gray - level co - occurrence matrix can be directly used for the calculation of gray - scale images, the optical flow map can assist in locating the changing range of the tumor cell region to ensure that the texture analysis focuses on the region related to the motion characteristics;

[0081] Furthermore, according to the optical flow data, calculate the average motion vector of the tumor cell region, accumulate the motion vectors of consecutive frames to generate the motion trajectory of the tumor region, and calculate the displacement and velocity of the motion trajectory;

[0082] Specifically, calculate the average motion vector V avg (t) with the formula:

[0083] V avg (t) = [μ u (t), μ(t)]

[0084] Specifically, the motion trajectory P(t) of the tumor region is expressed as:

[0085]

[0086] where k represents the k - th frame;

[0087] Specifically, the displacement D(t) and velocity S(t) of the motion trajectory are expressed as:

[0088]

[0089] where Δt is the inter-frame time interval;

[0090] Furthermore, the motion trajectory and the time series are integrated to obtain the dynamic characteristics of the gastric tumor cell region;

[0091] Specifically, the dynamic characteristics M of the gastric tumor cell region t are expressed as:

[0092]

[0093] S2. According to the dynamic characteristics, predict the local displacement caused by gastric peristalsis, and construct an affine transformation matrix to compensate for the rigid motion in the local displacement, so as to obtain a compensated gastric image containing tumor cells;

[0094] It should be noted that since gastric peristalsis is a periodic and complex motion process with short-term fluctuations and long-term regularities, using multi-scale LSTM can capture the motion characteristics of these different time scales through different levels, so as to more accurately predict the local displacement;

[0095] Further, design a three-layer long short-term memory network LSTM, and use the dynamic characteristics M of the gastric tumor cell region t as the input of the long short-term memory network. Each layer corresponds to a different time window to capture the multi-scale dynamic characteristics of gastric peristalsis;

[0096] Specifically, through the long short-term memory network, the displacement vector of each local region in the next frame can be automatically predicted, that is, the local displacement (Δx, Δy). The local displacement vector reflects the rigid motion (translation and rotation) caused by gastric peristalsis;

[0097] Furthermore, according to the local displacement (Δx, Δy), let t x = Δx, t y = Δy; for the rotation θ, if the local displacement contains a rotation component, the rotation method of the optical flow map needs to be used for estimation;

[0098] Specifically, convert the predicted local displacement into an affine transformation matrix A, which is expressed as:

[0099]

[0100] where θ represents the rotation angle, reflecting the rotational motion of the local region; t x and ty Denoted as the translation amounts in the horizontal and vertical directions, which reflect the translational motion of the local area;

[0101] Furthermore, apply the corresponding affine transformation matrix A to each local area, and transform the pixel points in the gastric image from the original position (x, y) to the compensated position (x′, y′):

[0102]

[0103] It should be noted that since the compensated pixel positions may be non-integer coordinates, if they are non-integer coordinates, the pixel values in the new image need to be calculated by interpolation methods;

[0104] S3. Update the medical segmentation network, optimize the medical segmentation network through gradient direction consistency, and extract clear tumor cell images;

[0105] Furthermore, embed a composite layer that fuses deformable convolution and attention mechanism in the encoder and decoder of the medical segmentation network. The deformable convolution adaptively adjusts the receptive field by learning the offset The calculation formula is expressed as: The calculation formula is expressed as:

[0106]

[0107] Among them, Denoted as the weight of the nth learning amount, I t Denoted as the input feature, that is, the compensated gastric image, and R is denoted as the regular grid of the convolution kernel; Denoted as the nth learning offset; w att (x, y) denotes the importance weight of calculating the tumor cell area by the attention mechanism, which is expressed as:

[0108]

[0109] Among them, Q(x, y) and K(x, y) are respectively denoted as the query feature and the key feature, and their function is to highlight the edge area of the tumor cells;

[0110] It should be noted that by embedding a composite layer that fuses deformable convolution and attention mechanism, the medical segmentation network can dynamically focus on the edge area of the tumor cell image and improve the clarity of the generated tumor cell image;

[0111] Furthermore, calculate the consistency between the gradient of the compensated tumor cell image and the gradient direction of the original tumor cell image to avoid incorrect feature extraction in the medical segmentation network;

[0112] Specifically, the gradient direction consistency formula is expressed as:

[0113]

[0114] Among them, is expressed as the loss value, and B is the total number of pixels; is expressed as the gradient of the original tumor cell image; is expressed as the gradient of the tumor cell image after compensation;

[0115] Furthermore, to ensure that there is no drastic change in the gradient direction between adjacent image pixels, it is necessary to add a smoothness constraint

[0116]

[0117] It should be noted that through the optimization of the gradient direction consistency, the medical segmentation network not only focuses on the pixel-level information of the tumor cell region, but also focuses on the gradient direction features of the tumor cell region, retains the characteristic information of the tumor cell invasion direction, and provides reference value for clinical diagnosis and analysis.

[0118] Further, this embodiment also provides a gastric tumor cell image extraction system, including:

[0119] A gastric tumor cell region extraction module, configured to extract consecutive frames from a gastroscopy video, preprocess the consecutive frames, and use a medical segmentation network to locate the gastric tumor cell region and extract the dynamic features of the gastric tumor cell region;

[0120] A gastric image compensation module, configured to predict the local displacement caused by gastric peristalsis according to the dynamic features, and construct an affine transformation matrix to compensate for the rigid motion in the local displacement, so as to obtain a compensated gastric image containing tumor cells;

[0121] A gastric tumor cell image output module, configured to update the medical segmentation network, optimize the medical segmentation network through gradient direction consistency, and extract a clear tumor cell image.

[0122] This embodiment also provides a computer device, applicable to the case of the gastric tumor cell image extraction method, including:

[0123] A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the gastric tumor cell image extraction method proposed in the above embodiment.

[0124] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0125] This embodiment also provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the gastric tumor cell image extraction method proposed in the above embodiment.

[0126] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. For technical details not described in detail in this embodiment, reference can be made to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0127] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages. For example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.

[0128] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementation in the processFigure 1 a process or processes and / or blocks Figure 1 means for the functions specified in a block or blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in a process Figure 1 a process or processes and / or blocks Figure 1 specified in a block or blocks.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in a process Figure 1 a process or processes and / or blocks Figure 1 specified in a block or blocks.

[0131] Although the preferred embodiments of the present application have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments as well as all changes and modifications that fall within the scope of the present application.

[0132] It is obvious that those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A method for extracting gastric tumor cell images, characterized in that, Including: Extract consecutive frames from the gastroscopy video, preprocess the consecutive frames, localize the gastric tumor cell region using a medical segmentation network, and extract the dynamic features of the gastric tumor cell region; According to the dynamic features, predict the local displacement caused by gastric peristalsis, and construct an affine transformation matrix to compensate for the rigid motion in the local displacement, obtaining a gastric image containing tumor cells after compensation; Update the medical segmentation network, optimize the medical segmentation network through gradient direction consistency, and extract a clear tumor cell image.

2. The method for extracting gastric tumor cell images according to claim 1, characterized in that, Extract consecutive frames from the gastroscopy video, extract consecutive frames from the gastroscopy video, and preprocess the consecutive frames. Localizing the gastric tumor cell region using a medical segmentation network includes: Take the gastroscopy video as a video frame sequence, and extract a consecutive frame sequence from the video frame sequence at a fixed frame rate; Based on the medical segmentation network, using a pre-trained segmentation model, input each frame of the gastric image in the consecutive frame sequence into the model to obtain the position of the tumor cell region.

3. The method for extracting gastric tumor cell images according to claim 2, wherein Extracting the dynamic features of the gastric tumor cell region includes: Take out the gastric images of two adjacent frames from the consecutive frame sequence, obtain the displacement vector of pixels in each gastric image, and generate an optical flow map; According to the position of the tumor cell region, extract the optical flow data belonging to the position of the tumor cell region from the optical flow map, and obtain the mean and variance of the optical flow data of the tumor cell region.

4. The method for extracting gastric tumor cell images according to claim 2 or 3, characterized in that, Also including: For the tumor cell region of each frame of the gastric image, convert the gastric image into a grayscale image, calculate the gray-level co-occurrence matrix of the tumor cell region, extract the contrast, correlation, energy, and homogeneity in the gray-level co-occurrence matrix, and form a time series; According to the optical flow data, calculate the average motion vector of the tumor cell region, accumulate the motion vectors of consecutive frames, generate the motion trajectory of the tumor region, and calculate the displacement and speed of the motion trajectory; Integrate the motion trajectory with the formed time series to obtain the dynamic features of the gastric tumor cell region.

5. The method for extracting gastric tumor cell images according to claim 4, characterized in that, Predicting the local displacement caused by gastric peristalsis according to the dynamic features includes: Design a three-layer long short-term memory network, use the dynamic features of the gastric tumor cell region as the input of the long short-term memory network, and through the long short-term memory network, obtain the displacement vector of each local region in the next frame, that is, the local displacement.

6. The method for extracting gastric tumor cell images according to claim 5, characterized in that, Constructing an affine transformation matrix to compensate for the rigid motion in the local displacement, obtaining a gastric image containing tumor cells after compensation, includes: Convert the local displacement into an affine transformation matrix, and apply the corresponding affine transformation matrix to each local region, so that the pixel points in the gastric image are transformed from the original position to the compensated position.

7. The method for extracting gastric tumor cell images according to claim 2 or 6, characterized in that, Updating the medical segmentation network, optimizing the medical segmentation network through gradient direction consistency, and extracting a clear tumor cell image, includes: Embed a composite layer that fuses deformable convolution and attention mechanism in the encoder and decoder of the medical segmentation network. The deformable convolution adaptively adjusts the receptive field by learning the offset, and the attention mechanism calculates the importance weight of the tumor cell region; Calculate the consistency between the gradient of the compensated tumor cell image and the gradient direction of the original tumor cell image, and consider the changes between adjacent image pixels to establish a smoothness constraint.

8. A gastric tumor cell image extraction system, based on the gastric tumor cell image extraction method according to any one of claims 1 to 7, characterized in that, Including: A gastric tumor cell region extraction module, configured to extract consecutive frames from a gastroscopy video, preprocess the consecutive frames, use a medical segmentation network to locate the gastric tumor cell region, and extract the dynamic features of the gastric tumor cell region; A gastric image compensation module, configured to predict the local displacement caused by gastric peristalsis according to the dynamic features, and construct an affine transformation matrix to compensate for the rigid motion in the local displacement, so as to obtain a compensated gastric image containing tumor cells; A gastric tumor cell image output module, configured to update the medical segmentation network, optimize the medical segmentation network through gradient direction consistency, and extract a clear tumor cell image.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

Images