Intelligent intestinal preparation evaluation method, device, equipment, medium and program product

Abstract

The invention relates to the technical field of computer-aided medical treatment, and discloses an intelligent intestinal preparation evaluation method, device and equipment, a medium and a program product, and the method comprises the steps: obtaining an excrement picture; based on a pre-trained image segmentation model, identifying the excrement observation area and the water seal of the drainage hole, and acquiring an identification output probability for the water seal of the drainage hole; brightness feature and texture feature extraction is carried out based on the excrement observation area, and the identification output probability for the water seal of the drainage hole is used as a transparency feature; and according to the brightness feature, the texture feature and the transparency feature, based on a pre-trained intestinal tract preparation quality discrimination model, carrying out quality evaluation. According to the method, intestinal preparation of a colonoscope patient is guided more accurately, and the satisfaction degree of the patient on intestinal preparation is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer-aided medical technology, and in particular relates to an intelligent intestinal preparation evaluation method, device, equipment, medium and program product. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Early colorectal cancer manifests as very subtle mucosal lesions. In order to ensure that subtle lesions can be detected, careful and complete mucosal examination and optimal bowel preparation are required. Among them, insufficient bowel preparation leading to insufficient bowel cleansing is an important factor in incomplete colonoscopy.

[0004] In clinical practice, bowel preparation before colonoscopy is mostly performed by patients themselves. There are currently the following problems: (1) During the bowel preparation process, patients are usually unable to accurately judge the bowel preparation after taking laxatives. (2) Patients usually perform bowel preparation during non-business hours (such as at night or in the early morning). The feasibility of manual guidance of patients during bowel preparation is low. (3) Most current bowel preparation instructions cannot provide personalized bowel preparation plans for the patients themselves. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides an intelligent intestinal preparation evaluation method, device, equipment, medium and program product, which can help patients evaluate their intestinal preparation at any time as needed before colonoscopy.

[0006] To achieve the above object, the first aspect of the present invention provides an intelligent intestinal preparation evaluation method, comprising the following steps:

[0007] Get images of excrement;

[0008] Based on the pre-trained image segmentation model, the excrement observation area and the water seal of the manhole are identified, and the recognition output probability of the water seal of the manhole is obtained;

[0009] The brightness and texture features are extracted based on the excrement observation area, and the recognition output probability of the water seal at the manhole is used as the transparency feature;

[0010] Quality evaluation is performed according to the brightness features, texture features and transparency features based on a pre-trained intestinal preparation quality discrimination model.

[0011] A second aspect of the present invention provides an intelligent intestinal preparation evaluation device, comprising:

[0012] The image acquisition module is configured to: acquire an image of excrement;

[0013] The image segmentation module is configured to: identify the excrement observation area and the sewer water seal based on a pre-trained image segmentation model, and obtain an identification output probability for the sewer water seal;

[0014] The feature advance module is configured to: extract brightness features and texture features based on the excrement observation area, and use the recognition output probability of the water seal at the drain hole as a transparency feature;

[0015] The quality evaluation module is configured to: perform quality evaluation based on the brightness features, texture features and transparency features and based on a pre-trained intestinal preparation quality discrimination model.

[0016] A third aspect of the present invention provides an electronic device, comprising:

[0017] a processor adapted to execute a computer program;

[0018] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the intelligent intestinal preparation evaluation method is implemented.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the intelligent intestinal preparation evaluation method.

[0020] A fifth aspect of the present invention provides a computer program product, characterized in that the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the intelligent intestinal preparation evaluation method.

[0021] A sixth aspect of the present invention provides an intelligent intestinal preparation auxiliary system, comprising a user terminal and a server; the user terminal is configured to: obtain excrement pictures and send them to the server; and receive intestinal preparation evaluation results fed back by the server;

[0022] The server is configured to execute the intelligent bowel preparation evaluation method and send the evaluation result to the user terminal.

[0023] One or more of the above technical solutions have the following beneficial effects:

[0024] The brightness and texture of the excrement observation area are used to reflect the fecal residue, foam and other attachments, and the identification of the water seal at the drain hole is used to reflect the transparency of fecal water. The excrement pictures are evaluated based on whether there are fecal residues, foam, the transparency of fecal water and the color, and then the intestinal preparation status is evaluated, and personalized guidance is generated according to the patient's condition. It can more accurately guide the intestinal preparation of colonoscopy patients and improve patients' satisfaction with intestinal preparation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a flow chart of the intelligent intestinal preparation evaluation method in an embodiment of the present invention;

[0027] Figure 2 This is an example of an excrement picture in an embodiment of the present invention;

[0028] Figure 3 This is an example of manual labeling of excrement pictures in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0030] In the description of the embodiments of the present application, the term “including” and similar terms should be understood as open inclusion, that is, “including but not limited to.” The term “based on” should be understood as “based at least in part on.”

[0031] One or more embodiments of the present invention propose an intelligent intestinal preparation auxiliary system, which is used to set up a full range of intestinal preparation guidance reminders according to the patient's colonoscopy time, including dietary reminders before colonoscopy, guidance on how to take laxatives, reminders for taking laxatives, reminders for uploading excrement pictures, and real-time evaluation of intestinal preparation status. The system includes a user terminal and a server, and the user terminal and the server are communicatively connected. Among them, the user terminal is used to upload the colonoscopy time and excrement pictures, and receive guidance information sent by the server; the server formulates and sends dietary reminders and medication reminders according to the colonoscopy time, and evaluates the intestinal preparation status in response to excrement pictures uploaded by the patient at any time.

[0032] The "drain hole water seal" referred to in one or more embodiments of the present invention refers to a water column of a certain height in the toilet when the toilet is not in use. Identifying the "drain hole water seal" can be understood as identifying the edge of the highest point of the drain hole, and the "excrement observation area" refers to the water surface area after the toilet is used.

[0033] Specifically, the intelligent intestinal preparation evaluation method is as follows: Figure 1 As shown, including:

[0034] Step 1: Get the excrement picture;

[0035] Step 2: Based on the pre-trained image segmentation model, identify the excrement observation area and the water seal of the manhole, and obtain the recognition output probability for the water seal of the manhole;

[0036] Step 3: Extract brightness features and texture features based on the excrement observation area, and use the recognition output probability of the water seal at the manhole as the transparency feature;

[0037] Step 4: Perform quality evaluation based on the brightness features, texture features and transparency features and a pre-trained intestinal preparation quality discrimination model.

[0038] By evaluating the patient's excrement images before colonoscopy, using the brightness and texture of the excrement observation area to reflect the presence of fecal residue, foam and other attachments, and using the identification of the water seal at the drain hole to reflect the transparency of the fecal water, and rating based on the presence of fecal residue, foam, fecal water transparency and color, it is possible to accurately identify whether the excrement image is qualified.

[0039] In step 1, the excrement picture should be taken from directly above the toilet and the picture should be clear. An important reason affecting the subsequent judgment results is the light spot caused by the reflected light source. Therefore, when collecting excrement images, the influence of the light source should be controlled as much as possible to reduce the generation and interference of light spots. In addition, it is understandable that in the bathroom, the clarity and noise of the image vary greatly under natural light during the day and lighting conditions at night, and whether the camera turns on the flash. Therefore, when preparing the data set, it is necessary to collect images under different light sources.

[0040] In step 2, the image segmentation model is used to identify the excrement observation area and the water seal of the manhole. Those skilled in the art will appreciate that image segmentation models can be constructed for the excrement observation area and the water seal of the manhole respectively, or the two targets can be identified based on the same image segmentation model. The method for constructing the image segmentation model includes:

[0041] (1) Collect patient excrement images and complete manual annotation to obtain a sample dataset. As an example, collect excrement images of patients during colonoscopy preparation and use annotation tools to annotate the excrement observation area ( Figure 3 Red mark) and the water seal of the drain hole ( Figure 3 Green marking) is manually annotated and records are made of whether the intestinal preparation quality represented by the image is qualified;

[0042] (2) The sample data set is divided into training, validation, and test sets according to the ratio of 8:2:2. The data is trained using a deep learning segmentation network to obtain an image segmentation model A.

[0043] Those skilled in the art will appreciate that the deep learning segmentation network can be implemented using an existing convolutional neural network, as long as the excrement observation area and the water seal of the manhole can be identified. In some embodiments, the deep learning segmentation network uses a fully convolutional network (FCN), which can accept image inputs of any size, upsample the last feature map through a deconvolution operation, and restore it to the size of the input image, thereby generating a prediction for each pixel and achieving pixel-level classification of the image.

[0044] Image segmentation model A uses the U-Net model. In order to improve the robustness against interference factors such as dirt in the toilet, the attention mechanism is introduced to strengthen the model's attention to important areas. In addition, samples of dirt and other interferences are added to the training data set, and the robustness of the model to these interferences is improved through data enhancement. For example, the training can be simulated for conditions such as dirt and stains, so that the model can still accurately segment the area of ​​interest under these conditions. Specifically, the U-Net model includes a contracting path and an expanding path. The contracting path is used to extract the features of the image, and the resolution of the image is gradually reduced through a series of convolutional layers and pooling layers, while the number of feature channels is increased to obtain high-level semantic information. The expanding path is used to restore the feature map to the size of the original image, and through a series of upsampling layers and convolutional layers, the deep semantic information and the shallow position information are combined to finally output the segmentation result. In order to improve the robustness of the model to dirt and other interferences, a channel attention module is added after the layer at the junction of each contracting path and expanding path. The weight of the feature map is thus dynamically adjusted so that the model pays more attention to the excrement observation area and is less sensitive to dirt.

[0045] In order to facilitate the subsequent evaluation of intestinal preparation quality based on characteristics such as brightness and texture, it is also necessary to eliminate the influence of brightness and light spots in the water.

[0046] In some embodiments, in order to eliminate the influence of brightness, the standard brightness of the toilet is set. After the water seal of the drain hole is identified, the size of the area inside the toilet is estimated according to the area of ​​the water seal of the drain hole, and the estimated area of ​​the inner surface of the toilet is obtained. The brightness values ​​of multiple locations in the estimated area of ​​the inner surface of the toilet are sampled, and the average of the multiple brightness values ​​is calculated. If the average is less than the standard brightness of the toilet, the contrast is stretched according to the difference between the average of the multiple brightness values ​​and the standard brightness of the toilet. Through the above method, the overall brightness of the image can be judged by comparing the brightness of the inner wall of the toilet in the picture, and there is no need to use machine learning, and the calculation can be performed only with the proportional relationship of the water seal in the toilet. After the brightness adjustment is performed, other operations are performed.

[0047] In some embodiments, in order to eliminate the influence of light spots, after the excrement observation area is identified, light spot recognition is also performed on the image of the excrement observation area, specifically including: converting the image of the excrement observation area into a grayscale image; performing light spot recognition according to a pre-trained light spot recognition model; counting the number of pixels occupied by the light spots, if the number of pixels occupied by the light spots in the image of the excrement observation area exceeds a set threshold, it is considered that it will affect subsequent evaluation results, and the excrement picture is determined to be unqualified, and the user is reminded to retake the picture.

[0048] Since the performance of light spots will be different in water with different turbidity, the light spot recognition model is trained separately according to different turbidity. Taking two different turbidities as an example, the recognition output probability of the water seal at the manhole is used as the transparency feature, and a transparency threshold is set. A support vector machine (SVM) is used to train the training data below and above the threshold respectively, and classification models for two different turbidities are obtained.

[0049] In step 3, brightness, texture richness and transparency are used to reflect the color of manure water, the content of attachments such as manure residue and foam, and the transparency of manure water for feature extraction.

[0050] (1) Brightness characteristics

[0051] The brightness histogram statistics of the segmentation results of the excrement observation area are obtained to obtain the data sample D = {(0,m0), (1,m1), (2,m2)......(255,m 255 )}, where 0 to 255 represents pixel values, and m represents the number of pixel points of a certain pixel value in the excrement observation area.

[0052] Convert the number of pixels into percentages and arrange them in order of pixel value index to obtain the sample brightness feature array L:

[0053]

[0054] (2) Texture features

[0055] The segmentation result of the excrement observation area is converted into a grayscale image, and the texture is detected using the Laplacian operator, and the variance of the texture features is calculated to obtain the texture richness of the sample. The larger the variance, the richer the texture information in the image, that is, the higher the content of feces, foam and other attachments.

[0056] Opencv was used to perform contour pixel area statistics on the segmentation results of the excrement observation area, and this was used as the denominator and the above texture richness as the numerator to calculate the texture richness R per pixel of the sample.

[0057] (3) Transparency characteristics

[0058] The recognition output probability of the water seal at the drain hole is used as the transparency feature. The recognition result of the water seal at the drain hole of the sample image is recorded. If the water seal at the drain hole is not detected, the transparency T of the sample is recorded as 0. If the water seal at the drain hole is detected, the transparency T is the model recognition output probability.

[0059] The model is constructed with three inputs: brightness feature array L, unit pixel texture richness R, and transparency T. The three inputs are connected to the fully connected layer for dimensional conversion, and then concatenated and merged after unifying the dimensions. The fully connected layer is then used to design a binary classification output. The model is trained with the marked preparation quality as the category label to obtain the preparation quality discrimination model B.

[0060] Based on the above-mentioned intestinal preparation evaluation method, one or more embodiments of the present invention further provide an intelligent intestinal preparation evaluation device, including:

[0061] The image acquisition module is configured to: acquire an image of excrement;

[0062] The image segmentation module is configured to: identify the excrement observation area and the sewer water seal based on a pre-trained image segmentation model, and obtain an identification output probability for the sewer water seal;

[0063] The feature advance module is configured to: extract brightness features and texture features based on the excrement observation area, and use the recognition output probability of the water seal at the drain hole as a transparency feature;

[0064] The quality evaluation module is configured to: perform quality evaluation based on the brightness features, texture features and transparency features and based on a pre-trained intestinal preparation quality discrimination model.

[0065] One or more embodiments of the present invention further provide an electronic device that can be used to implement the intestinal preparation evaluation method in the above embodiments. The electronic device includes one or more processors, one or more memories coupled to the processors, and a communication module coupled to the processors.

[0066] The memory may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, at least one of the following: read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD) or other magnetic storage and / or optical storage. Examples of volatile memories include, but are not limited to, at least one of the following: random access memory (RAM), or other volatile memory that does not last during the duration of a power outage. The computer program may be stored in the ROM. When the processor executes the computer program, the above-mentioned cross-regional geographic entity data coordination processing method is implemented.

[0067] In some embodiments, the program may be tangibly contained in a computer-readable medium, which may be included in a device (such as in a memory) or other storage device accessible by the device. The program may be loaded from the computer-readable medium to the RAM for execution. The computer-readable medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, and the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned cross-regional geographic entity data coordination processing method.

[0068] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a server or terminal, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a server or terminal or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (such as a floppy disk, a hard disk and a tape, etc.), an optical medium (such as a digital video disk (digital video disk, DVD), etc.), or a semiconductor medium (such as a solid-state hard disk, etc.).

[0069] Based on the above-mentioned intestinal preparation evaluation method, one or more embodiments of the present invention further provide an intelligent intestinal preparation auxiliary system, including a user terminal and a server, wherein the user terminal and the server are communicatively connected.

[0070] The user terminal is configured to include:

[0071] An image acquisition module is used to obtain excrement pictures and send them to the server;

[0072] A result receiving module, used for receiving the intestinal preparation evaluation result fed back by the server;

[0073] The server is configured to include:

[0074] An image acquisition module, used to acquire excrement images;

[0075] An image segmentation module, used to identify the excrement observation area and the sewer water seal based on a pre-trained image segmentation model, and obtain the recognition output probability for the sewer water seal;

[0076] A feature extraction module is used to extract brightness features and texture features based on the excrement observation area, and use the recognition output probability of the water seal at the drain hole as a transparency feature;

[0077] A quality evaluation module, used for performing quality evaluation based on the brightness feature, texture feature and transparency feature and a pre-trained intestinal preparation quality discrimination model;

[0078] The result feedback module transmits the evaluation result to the user terminal.

[0079] The user terminal is configured to further include:

[0080] The guidance acquisition module is used to obtain the user's colonoscopy examination time and send it to the server; and receive the diet, medication taking reminders, and excrement uploading reminders sent by the server;

[0081] The server is configured to further include:

[0082] The intestinal preparation guidance module is used to obtain the user's colonoscopy test time, formulate a medication time reminder plan according to the medication time requirements, and send the reminder to the user terminal on time. In addition, a dietary reminder is also sent to the user terminal. The medication time reminder also includes guidance on medication method.

[0083] Those skilled in the art can understand that the user terminal can be a terminal device such as a mobile phone, and the above functions can be implemented through APP or mini-programs, for example, the above functions can be implemented through WeChat public accounts. Based on the WeChat public account function, according to the patient's colonoscopy time, intestinal preparation guidance reminders are set, including dietary reminders before colonoscopy, guidance on how to take laxatives, reminders for taking laxatives, reminders for uploading excrement pictures, and artificial intelligence assessment of excrement picture grading. After the user takes a photo and uploads the image, the program calls the image segmentation model A to segment the target area, and post-processes the segmentation results to statistically calculate the brightness feature array L, unit pixel texture richness R, and transparency T. Finally, the preparation quality discrimination model B is called for final judgment, and the user's recognition result is fed back.

[0084] Although adopting specific order to describe each operation, this should be understood as requiring such operation to be performed in the specific order shown or in sequential order, or requiring all illustrated operations to be performed to obtain desired results. Under certain environment, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the application. Some features described in the context of separate embodiments can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in the mode of any suitable sub-combination.

[0085] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An intelligent intestinal preparation evaluation method, characterized in that: The following steps are involved: Get images of excrement; Based on the pre-trained image segmentation model, the excrement observation area and the water seal of the manhole are identified, and the recognition output probability of the water seal of the manhole is obtained; The brightness and texture features are extracted based on the excrement observation area, and the recognition output probability of the water seal at the manhole is used as the transparency feature; Quality evaluation is performed according to the brightness features, texture features and transparency features based on a pre-trained intestinal preparation quality discrimination model.

2. The intelligent intestinal preparation evaluation method according to claim 1, characterized in that: After identifying the excrement observation area, light spot recognition is also performed on the excrement observation area image. If the number of pixels occupied by the light spot in the excrement observation area image exceeds a set threshold, it is determined that the excrement picture is unqualified.

3. The intelligent intestinal preparation evaluation method according to claim 1, characterized in that: The brightness feature extraction comprises: Perform brightness histogram statistics on the excrement observation area image to obtain the number of pixels for each pixel value in the excrement observation area image; The proportion of the number of pixels corresponding to each pixel value in the excrement observation area image is calculated, and the pixel values ​​are sorted as indexes to obtain a brightness feature array.

4. The intelligent intestinal preparation evaluation method according to claim 1, characterized in that: The texture feature extraction comprises: Convert the excrement observation area image into a grayscale image; Use the Laplacian operator to perform texture detection on the grayscale image of the excrement observation area; The variance of texture features is calculated to obtain texture richness; The texture richness per unit pixel of the sample is calculated by taking the number of pixels of the excrement observation area image as the denominator and the texture richness as the numerator.

5. An intelligent intestinal preparation evaluation device, characterized in that: include: The image acquisition module is configured to: acquire an image of excrement; The image segmentation module is configured to: identify the excrement observation area and the sewer water seal based on a pre-trained image segmentation model, and obtain an identification output probability for the sewer water seal; The feature advance module is configured to: extract brightness features and texture features based on the excrement observation area, and use the recognition output probability of the water seal at the drain hole as a transparency feature; The quality evaluation module is configured to: perform quality evaluation based on the brightness features, texture features and transparency features and based on a pre-trained intestinal preparation quality discrimination model.

6. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the intelligent intestinal preparation evaluation method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the intelligent intestinal preparation evaluation method according to any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the intelligent intestinal preparation evaluation method according to any one of claims 1 to 4.

9. An intelligent bowel preparation auxiliary system, characterized in that: It includes a user terminal and a server; the user terminal is configured to: obtain an excrement picture and send it to the server; and receiving bowel preparation evaluation results fed back by the server; The server is configured to execute the intelligent intestinal preparation evaluation method according to any one of claims 1 to 4, and send the evaluation result to the user terminal.

10. The intelligent bowel preparation auxiliary system according to claim 9, characterized in that: The server is also configured to: obtain the user's colonoscopy examination time, formulate a medication time reminder plan according to the medication taking time requirement, and send the reminder to the user terminal on time.

Images