Colorectal cancer self-service screening system based on image recognition

Abstract

The invention provides a colorectal cancer self-service screening system based on image recognition, and relates to the technical field of image processing.The colorectal cancer self-service screening system can effectively integrate lesion information in a time sequence by collecting endoscope images during multiple times of follow-up visit, the endoscope images are trained and processed through a Faster R-CNN network, and the image recognition accuracy is improved. The lesion area can be automatically identified and marked. By labeling positive and negative labels of training images, the system not only can accurately detect lesions, but also can quickly identify different lesion types, construct lesion indexes of each image group, and generate a regression model based on the lesion indexes of each image group, thereby realizing prediction of lesion development. The high-risk reference time can be generated according to the change of the lesion index, and the reference evaluation time length of the current time is calculated. Once a set danger threshold value is reached, the system sends out a high risk prompt in time, so that the lesion can be found and treated in an early stage, and a more accurate basis is provided for subsequent early warning and treatment.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a self-service colorectal cancer screening system based on image recognition. Background Art

[0002] Colorectal cancer is a common malignant tumor, and early diagnosis and treatment are crucial for improving the survival rate of patients. With the progress of medical imaging technology, colonoscopy, as an effective screening tool, has been widely used in the early detection of colorectal cancer. However, due to the lack of obvious early symptoms of colorectal cancer and the usually slow progression of tumors, traditional screening methods usually rely on doctors' experience and judgment, resulting in possible missed or misdiagnosed lesions. In addition, the analysis of endoscopic images is usually carried out manually, making it difficult to efficiently process a large amount of data and provide accurate early warnings.

[0003] Currently, most colon cancer screenings still rely on manual analysis of endoscopic images. With the continuous development of computer vision technology, image recognition-based auxiliary diagnostic tools have gradually become a trend. Some studies have adopted machine learning and deep learning algorithms, such as convolutional neural network (CNN) models like Faster R-CNN, for automatic tumor recognition and classification. These systems can automatically extract features from endoscopic images and perform preliminary diagnoses, promising to improve the accuracy and efficiency of screening. In addition, classification and annotation based on lesion types to assist doctors in quantitative analysis of lesions also provide the possibility for early warnings.

[0004] Although the existing image recognition-based auxiliary screening technologies have made certain progress, there are still some problems. First, most of the existing technologies can only perform single lesion detection and lack a comprehensive analysis of the patient's follow-up history. The occurrence and development of colon cancer usually have a time factor, and single image analysis cannot reflect the evolution process of the disease. Second, most of the existing technologies cannot perform dynamic risk assessment, lack prediction of the lesion progression trend, and cannot achieve targeted warnings. In addition, the existing technologies often ignore the association of image capture time and it is difficult to provide personalized screening suggestions for patients through time series analysis. Therefore, how to use multiple follow-up data combined with image recognition technology for dynamic monitoring and accurate warning is the key problem that the current technology urgently needs to solve.

[0005] The above information disclosed in the background art section is only used to enhance the understanding of the background of the present disclosure, and thus it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] The purpose of the present invention is to provide a self-service colorectal cancer screening system based on image recognition to solve the problems raised in the above background art.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A self-service colorectal cancer screening system based on image recognition, comprising:

[0009] A data acquisition module, which is used to acquire endoscopic images during multiple follow-ups, preprocess all the endoscopic images, mark the shooting time of each image on the preprocessed endoscopic images, form an image group with the endoscopic images at the same time, map each image group and the shooting time one by one, and sort all the image groups in the order of shooting time to form a screening data set. The endoscopic images are images obtained by colonoscopy;

[0010] An image processing module, which is used to collect endoscopic images with different lesion types as training images, preprocess the training images to form a training image set, and perform positive and negative label annotation on the images within the training image set. The positive label indicates that the image contains a lesion area, and the negative label indicates that the image does not contain a lesion area;

[0011] A model construction module, which is used to label the lesion labels for the training images with positive labels. The lesion labels include the type of the lesion and the proportion of the lesion area in the total image. The images with the labeling completed are formed into a training data set, a lesion recognition model based on the Faster R-CNN network is constructed, and the training data set is input into the lesion recognition model for training;

[0012] A lesion recognition module, which is used to input the screening data set into the trained lesion recognition model, obtain the positive and negative label annotation and lesion label data of each endoscopic image in the screening data set, classify the endoscopic images with positive labels and the endoscopic images with negative labels, construct a positive label screening data set and a negative label screening data set, and calculate the proportion of the images with positive labels in each image group;

[0013] A regression analysis module, which is used to construct a mathematical analysis model according to the lesion labels of each lesion type in each image group in the positive label screening data set and the proportion of the images with positive labels in each image group, generate a lesion index for each image group, and construct a regression model with the time variable as the abscissa and the lesion index as the ordinate;

[0014] A screening and warning module, which is used to set a dangerous lesion index threshold, generate the corresponding time when the dangerous lesion index threshold is reached based on the constructed regression model, calibrate it as the high-risk reference time, calculate the time length between the high-risk reference time and the current time, calibrate it as the reference evaluation time, and issue a corresponding colorectal cancer warning prompt for the reference evaluation time.

[0015] Further, the steps of preprocessing the collected endoscopic images and training images include:

[0016] Scale the size of the image to 512*512 pixels, obtain the color values of each pixel point in the RGB color channels, perform weighted processing on the RGB components of the pixel points of the image, and obtain the grayscale image of the image. The formula is:

[0017] g(i, h) = 0.3R(i, j) + 0.6G(i, j) + 0.1B(i, h)

[0018] Wherein, g(i, h) represents the grayscale value of the pixel point at the i-th row and j-th column in the image, and R(i, h), G(i, j), and B(i, h) respectively represent the R, G, and B color values of the pixel point at the i-th row and j-th column in the RGB color channels of the image;

[0019] For the grayscale image, perform feature sharpening processing using the median filtering method, and set the grayscale value of each pixel point in the grayscale image to the median of the grayscale values of adjacent pixel points;

[0020] Perform contrast enhancement processing on the image after feature sharpening processing, calculate the gain for the high-frequency part in the image, and obtain the enhanced image. The logic is:

[0021] Calibrate the pixel point at the i-th row and j-th column in the image as (i, j), calculate the mean K and variance σ of the image within the range of (2n + 1)×(2n + 1) centered on this pixel point (i, j). The formula for mean calculation is:

[0022]

[0023] Wherein, K(i, j) represents the mean of the image within the range of (2n + 1)×(2n + 1) centered on (i, j), and (p, q) represents the grayscale value of the pixel point of the image within the range of (2n + 1)×(2n + 1) centered on;

[0024] The formula for variance calculation is:

[0025]

[0026] Wherein, (p, q) - K(i, j) represents the high-frequency part in the image. The formula for calculating the gain for the high-frequency part in the image is:

[0027] F(i, h) = K(i, j) + α[(p, q) - K(i, j)]

[0028] Among them, r(i, j) represents the gray value of the (i, j) pixel after gain, α is the gain weight, and α > 1.

[0029] Furthermore, the types of the lesions include polyp lesions and ulcer lesions. When labeling the lesion labels, the number of pixels in the polyp or ulcer area and the total number of pixels in the image are counted, and the ratio of the lesion area to the total image is obtained. The formula is as follows:

[0030]

[0031] Among them, Xz and Kz respectively represent the ratios of polyp lesions and ulcer lesions in the total image, N Xr and N Ky respectively represent the number of pixels occupied by the polyp lesion part and the ulcer lesion part in the training image in the image, and N z represents the total number of pixels in the training image;

[0032] If a training image contains both polyp lesions and ulcer lesions at the same time, the corresponding lesion types and the ratios of the corresponding lesion types are respectively marked.

[0033] Furthermore, the method for constructing the positive label screening data set and the negative label screening data set is as follows:

[0034] Each image group is classified after being recognized by the lesion recognition model. The images containing positive labels in each image group are divided into one group and labeled as the positive label image group, and the images containing negative labels are divided into one group and labeled as the negative label image group. The positive label image group and the negative label image group are respectively mapped one-to-one with the shooting time;

[0035] After all image groups are classified, all positive label image groups constitute the positive label screening data set, and all negative label image groups constitute the negative label screening data set.

[0036] Furthermore, the method for calculating the proportion of images containing positive labels in each image group is as follows:

[0037] The total number of images in each image group and the number of images in the corresponding positive label image group are counted, and the proportion of images containing positive labels in each image group is calculated. The formula is as follows:

[0038]

[0039] Among them, represents the proportion of images containing positive labels in the image group corresponding to the shooting time t m corresponding, and respectively represent the shooting time as t mIn the corresponding image group, the number of images within the corresponding positive label image group and the total number of images within the image group.

[0040] Further, the method for constructing a mathematical analysis model and generating the lesion index of each image group is as follows:

[0041] Obtain the lesion labels of each image within the corresponding positive label image group in each image group, that is, the proportion of polyp lesions and ulcer lesions in each image in the image. Combine the proportion of images with positive labels in each image group to construct a mathematical analysis model. The formula based on is:

[0042]

[0043] Among them, represents the shooting time as t m the lesion index of the corresponding image group, represents the shooting time as t m the number of images in the positive label image group, and respectively represent the proportions of polyp lesions and ulcer lesions in the total image in the w-th image in the positive label image group with the shooting time as t m γ and δ respectively represent the polyp influence weight and the ulcer influence weight, γ > δ > 0, and γ + δ = 1.

[0044] Further, the regression model is a polynomial regression model. The formula for constructing the regression model with the time variable as the abscissa and the lesion index as the ordinate is:

[0045]

[0046] Among them, Bz(t) represents the dependent variable of the constructed regression model, t represents the time variable, that is, the independent variable, ∈ represents the number of image groups, τ represents the intercept of the constructed regression model, represents the time variable the coefficient of the

[0047] Substitute the time and lesion index corresponding to all image groups into the constructed regression model. With time as the independent variable and the lesion index as the variable, construct a ∈-element linear equation, calculate the intercept of the regression model and the coefficients of each time variable power, and obtain the solved regression model.

[0048] Further, the method for calibrating the high-risk reference time is as follows:

[0049] Set the dangerous lesion index threshold, substitute the dangerous lesion index threshold as the dependent variable into the solved regression model, calculate its corresponding time variable, calibrate this time variable as the high-risk reference time, and denote it as t gw, calculate the time length between the high-risk reference time and the current time, and the formula used is:

[0050] t ck = t gw - t dq

[0051] Wherein, t ck represents the reference evaluation time, that is, the time length between the high-risk reference time and the current time, and t dq represents the current time; t ck t dq t ck ≤ 6mont6mont

[0052] When issuing the corresponding colorectal cancer warning prompt, if t ck ≤ 6mont, a high-risk risk prompt will be issued.

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] By collecting endoscopic images during multiple follow-ups, the present invention can effectively integrate the lesion information in the time series. By training and processing the endoscopic images through the Faster R-CNN network, it can automatically identify and label the lesion areas. Through the positive and negative label annotation of the training images, the system can not only accurately detect the lesions, but also quickly identify different lesion types, construct the lesion index of each image group, and generate a regression model based on the lesion index of each image group, so as to realize the prediction of the lesion development, be able to generate the high-risk reference time according to the change of the lesion index, and calculate the reference evaluation time length from the current time. Once the set danger threshold is reached, the system will promptly issue a high-risk risk prompt, which can detect and handle the lesions early, and provide a more accurate basis for subsequent early warning and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a schematic diagram of the overall system structure of the present invention;

[0056] Figure 2 is a schematic diagram of constructing the positive label screening data set and the negative label screening data set in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the following further describes the present invention in detail with reference to specific embodiments.

[0058] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the field to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0059] Embodiment:

[0060] Please refer to Figure 1 - Figure 2 , the present invention provides a technical solution:

[0061] A colorectal cancer self-screening system based on image recognition, comprising a data acquisition module, an image processing module, a model construction module, a lesion recognition module, a regression analysis module and a screening and warning module, wherein:

[0062] The data acquisition module is used to acquire endoscopic images during multiple follow-ups, preprocess all the endoscopic images, mark the shooting time of each image on the preprocessed endoscopic images, form image groups with the endoscopic images at the same time, map each image group and the shooting time one by one, and sort all the image groups in the order of shooting time to form a screening data set, and the endoscopic images are images obtained by colonoscopy.

[0063] Follow-up refers to endoscopic images obtained from examinations of the same patient at different time points. During multiple follow-ups, endoscopic images of colonoscopy are collected from follow-ups at different time points. The images of each examination need to be preprocessed to ensure the improvement and consistency of image quality. Each image will be marked with the specific shooting time, and this time label is crucial for establishing time series data, which can accurately observe and analyze the changes of lesions over time. Then, the images taken at the same time point will be grouped together, which can make the comparison and diagnosis more systematic. These image groups are mapped to their corresponding time points and sorted in the order of shooting time. This sorting method enables the entire data set to reflect the dynamic changes of the patient's condition, thus providing structured information for screening and diagnosis.

[0064] The capture time specifically refers to the exact time point when each endoscopic image is captured, including the date, which is used to record the chronological order of image acquisition. During endoscopic examinations, especially colonoscopies, doctors need to comprehensively observe the entire colon to ensure that no possible lesions or abnormalities are missed. Multiple images need to be taken during the examination to cover the entire examination area. The colon is a long and complex organ, and a single image cannot cover the entire examination area. By taking multiple images, it can be ensured that every part has been examined. Therefore, there are multiple endoscopic images during each follow-up, and these images form an image group. The endoscopic images taken during one follow-up are often those taken during one examination. Therefore, the images during one follow-up can be organized into an image group according to the examination time.

[0065] In this embodiment, the steps for preprocessing the collected endoscopic images and training images include:

[0066] Scale the size of the image to 512*512 pixels, obtain the color values of each pixel point in the RGB color channels, perform weighted processing on the RGB three-color components of the pixel points of the image, and obtain the grayscale image of the image. Its essence is image grayscale conversion, converting the RGB-mode color image into a grayscale image. The purpose is to reduce the computational amount and time during model learning and recognition. By converting the RGB image into a grayscale image, the data dimension can be reduced while maintaining the basic information of the image. The formula is as follows:

[0067] g(i, j) = 0.3R(i, j) + 0.6G(i, j) + 0.1B(i, j)

[0068] Among them, g(i, j) represents the grayscale value of the pixel point at the i-th row and j-th column in the image, and R(i, j), G(i, j), and B(i, j) respectively represent the R, G, and B color values of the pixel point at the i-th row and j-th column in the RGB color channels of the image. The weighting is designed according to the differences in the sensitivity of the human eye to different colors to ensure that the converted grayscale image can effectively represent the information of the original image.

[0069] Scaling the image to 512x512 pixels helps standardize the image size for subsequent processing and analysis. The fixed size ensures the unity of the input, thereby simplifying the subsequent model training and recognition processes and reducing the consumption of computing resources.

[0070] For grayscale images, median filtering is used for feature sharpening. The grayscale value of each pixel in the grayscale image is set to the median of the grayscale values of adjacent pixels. When performing feature sharpening on the grayscale image, median filtering is used to sharpen the image. The basic principle is to replace the grayscale value of a pixel with the median of the grayscale values around that pixel in the image, thereby reducing the noise in the image, making the edges of the cracks smoother, facilitating subsequent learning and training of the model, and also eliminating the noise in the image. While retaining the edge information, median filtering effectively removes salt-and-pepper noise and enhances the clarity of the image, making subsequent feature extraction more accurate.

[0071] The image after feature sharpening is subjected to contrast enhancement. The gain of the high-frequency part in the image is calculated to obtain the enhanced image. The contrast enhancement step improves the visibility of the region of interest in the image through gain calculation. The enhanced image makes the lesion area easier to identify, improves the visual quality of the image, and helps to improve the accuracy of the recognition algorithm. The logic is as follows:

[0072] The pixel at the i-th row and j-th column in the image is labeled as (i, j). The mean K and variance σ of the image within the range of (2n + 1)×(2n + 1) centered on the pixel (i, h) are calculated. The formula for mean calculation is:

[0073]

[0074] where K(i, j) represents the mean of the image within the range of (2n + 1)×(2n + 1) centered on (i, j), and (p, q) represents the grayscale value of the pixel in the image within the range of (2n + 1)×(2n + 1) centered on it;

[0075] The formula for variance calculation is:

[0076]

[0077] where (p, q) - K(i, j) represents the high-frequency part in the image. The formula for calculating the gain of the high-frequency part in the image is:

[0078] F(i, j) = K(i, j) + α[(p, q) - K(i, j)]

[0079] where F(i, j) represents the grayscale value of the pixel (i, j) after gain, and α is the gain weight, and α > 1.

[0080] The image after feature sharpening post - processing is subjected to contrast enhancement processing. Gain calculation is performed on the high - frequency part of the image to obtain the enhanced image. The set of enhanced images is the training image dataset. Contrast enhancement processing can help improve the quality of image data, making the features in the image more prominent and facilitating model learning. In this embodiment, performing gain processing on the high - frequency part of the image actually emphasizes the details and edge information in the image because these usually correspond to the high - frequency components of the image. By performing gain calculation on the high - frequency part, the performance of details and edges in the image is enhanced. Using the gain weight provides richer detail information for feature extraction of the image, improving the accuracy of lesion recognition and the detection sensitivity of the model.

[0081] The image processing module is used to collect endoscopic images containing different lesion types as training images, and pre - process the training images to form a training image set. Positive and negative labels are assigned to the images within the training image set. The positive label indicates that the image contains a lesion area, and the negative label indicates that the image does not contain a lesion area.

[0082] The obtained training images are a large number of endoscopic image data collected by medical institutions, etc., through endoscopic devices. These images are anonymized to ensure the protection of patient privacy. Next, the preliminarily screened images are selected by professional physicians or trained image annotators, which contain various different types of lesions, such as polyps and ulcers, to ensure the diversity and high quality of the images, so as to provide comprehensive learning materials for the model during the training process. The pre - processing steps of the training images are exactly the same as those for pre - processing endoscopic images.

[0083] By examining each image in the training image set, potential lesion areas are identified. Then, using the image annotation software tool "Labelimg" or other software, the boundaries of the lesion areas are manually outlined. To ensure the accuracy and consistency of the annotation, multiple reviews are carried out, sometimes cross - reviewed by different personnel to reduce the deviation of subjective judgment. The number of pixel points in the annotated area is counted. By calculating the ratio of the lesion pixels to the total number of pixel points in the entire image, the proportion of the lesion area in the image can be obtained. This data is not only used for label annotation but also used as a feature parameter during the training process to help the model learn the feature distribution and morphological features of different lesion types. This annotated data is crucial for the training effect of the model because it provides a clear supervision signal, guiding the model to make accurate predictions and classifications when automatically identifying lesions.

[0084] The model construction module is used to label the training images with positive labels with lesion labels, where the lesion labels include the type of the lesion and the proportion of the lesion area in the total image. The images with completed labeling are used to form a training dataset, and a lesion recognition model based on the Faster R-CNN network is constructed. The training dataset is input into the lesion recognition model for training.

[0085] Faster R-CNN is an advanced model in region convolutional neural networks. By introducing a region proposal network, it can quickly generate high-quality candidate regions in images. This mechanism significantly improves the detection speed and accuracy. For tasks that require precise localization and classification of lesion areas, Faster R-CNN provides an efficient and accurate solution. The design of Faster R-CNN makes it particularly suitable for processing complex endoscopic images. It can not only identify the lesion areas in the images but also further distinguish different types of lesions through a multi-class classifier. Its deep convolutional neural network structure can automatically learn the high-level features in the images without manual feature engineering, thus effectively capturing the subtle differences and features of the lesions.

[0086] The lesion recognition model based on the Faster R-CNN network is an advanced deep learning model that can accurately detect multiple objects in images and classify each object. The main structure of Faster R-CNN includes a feature extraction network, a region proposal network, a RoI pooling network, a classification and bounding box regression network, and non-maximum suppression. When the lesion recognition model based on the Faster R-CNN network is trained to identify the lesion areas in images, polyp lesions and ulcer lesions are regarded as a special type of object. The basic principle of the model to identify polyp lesions and ulcer lesions is similar to other object detection tasks in the prior art.

[0087] In this embodiment, the types of the lesions include polyp lesions and ulcer lesions. When labeling the lesion labels, the number of pixel points in the polyp or ulcer area and the total number of pixel points in the image are counted to obtain the proportion of the lesion area in the total image. The formula is as follows:

[0088]

[0089] where Xz and Kz respectively represent the proportions of polyp lesions and ulcer lesions in the total image, N Xr and N Ky respectively represent the number of pixel points occupied by the polyp lesion part and the ulcer lesion part in the training image in the image, and N z represents the total number of pixel points in the training image;

[0090] If a training image contains both polyp lesions and ulcer lesions, the corresponding lesion types and the proportion of the corresponding lesion types are marked respectively.

[0091] The Faster R-CNN architecture uses a deep convolutional neural network to automatically extract features from images. The convolutional layer scans the image through filters, capturing local spatial information such as edges, textures, and shapes, which is crucial for lesion recognition. As the network depth increases, the model gradually obtains more abstract and high-level features to help distinguish different types of lesions. Once the features are extracted, the Region Proposal Network (RPN) generates candidate regions, which are parts of the image that the model believes may contain lesions. The RPN generates a series of anchor boxes on the feature map through a sliding window mechanism and calculates the score for whether each box contains a lesion. After screening, the RPN outputs a set of potential lesion regions for more precise analysis by the subsequent classifier. In the classification stage, the model evaluates each candidate region in detail. The fully connected layer receives the region proposals from the RPN and assigns a class label, such as polyp, ulcer, or normal tissue, to each region through a Softmax classifier. At the same time, the regression layer provides accurate bounding box predictions for these regions. In this way, the model can not only determine the type of lesion in the image but also accurately locate the position of the lesion. By counting the number of pixel points inside the bounding box and the total number of pixel points in the image, the number of pixel points occupied by the polyp lesion part and the ulcer lesion part in the training image can be obtained. The model streamlines the detection results through non-maximum suppression technology. Since the RPN may generate multiple overlapping candidate regions, NMS is used to eliminate redundant detections and only retain the most confident results.

[0092] In the lesion recognition model, the output containing positive and negative labels is to effectively distinguish whether there are lesion regions in the image. The positive label indicates that the model detects a lesion, while the negative label indicates that the detected region is normal and lesion-free, which helps the model focus on identifying and analyzing potential lesion regions. The positive label is further divided into specific lesion types and the pixel proportion of the lesion region because different types of lesions have different importance in screening. The number of pixel points in the marked region and the total number of pixel points in the image are used to measure the size and influence range of the lesion in the image.

[0093] Based on the positive and negative label annotations, the collected images can be divided into two categories. One is the positive label, representing the presence of a lesion region, and the other is the negative label, representing the absence of a lesion region. The lesion label is only available for images with positive label annotations. The data of the lesion label includes the type of the lesion and the proportion of the lesion region in the total image. Therefore, the recognized images with positive label annotations will have the type of the lesion and the proportion of the lesion region in the total image.

[0094] The lesion recognition module is used to input the screening dataset into the trained lesion recognition model, obtain the positive and negative label annotations and lesion label data of each endoscopic image in the screening dataset, classify the endoscopic images with positive labels and those with negative labels, construct a positive label screening dataset and a negative label screening dataset, and calculate the proportion of images with positive labels in each image group.

[0095] In this embodiment, the method for constructing the positive label screening dataset and the negative label screening dataset is as follows:

[0096] After classification by the lesion recognition model for each image group, the images with positive labels in each image group are grouped together and labeled as the positive label image group, and the images with negative labels are grouped together and labeled as the negative label image group. The positive label image group and the negative label image group are respectively mapped one-to-one with the shooting time.

[0097] After classification of all image groups, all positive label image groups constitute the positive label screening dataset, and all negative label image groups constitute the negative label screening dataset.

[0098] The lesion recognition model determines the positive and negative labels for each image. A positive label indicates that there is a lesion in the image, and a negative label indicates no lesion. Then, these images are grouped according to the labels. Among the images taken at the same time or during the same follow-up, the images with lesions are labeled as the positive label image group, and vice versa for the negative label image group. Finally, these labeled image groups, combined with their shooting times, respectively constitute the positive label screening dataset and the negative label screening dataset.

[0099] Further, the method for calculating the proportion of images with positive labels in each image group is as follows:

[0100] Count the total number of images within each image group and the number of images within the corresponding positive label image group, and calculate the proportion of images with positive labels in each image group. The formula is as follows:

[0101]

[0102] Where, represents the shooting time as t m the proportion of images with positive labels in the corresponding image group, and respectively represent the number of images within the corresponding positive label image group and the total number of images within the image group corresponding to the shooting time t m in the corresponding image group.

[0103] The higher the proportion of images with positive labels in the image group, the larger the number of images with lesions in the relevant image group, which means that the incidence of lesions is relatively high at the time point or in the area where the image group is taken, thus indicating that there may be a more serious lesion condition or a more extensive lesion distribution.

[0104] The regression analysis module is used to screen the lesion labels of each lesion type in each image group in the dataset according to the positive label, and the proportion of images with positive labels in each image group, construct a mathematical analysis model, generate the lesion index of each image group, and construct a regression model with the time variable as the abscissa and the lesion index as the ordinate.

[0105] In this embodiment, the method for constructing a mathematical analysis model and generating the lesion index of each image group is as follows:

[0106] Obtain the lesion labels of each image in the corresponding positive label image group within each image group, that is, the proportion of polyp lesions and ulcer lesions in each image in the image, and combine the proportion of images with positive labels in each image group to construct a mathematical analysis model. The formula is as follows:

[0107]

[0108] Among them, represents the lesion index of the corresponding image group at the shooting time t m ; represents the number of images in the positive label image group at the shooting time t m ; and respectively represent the proportion of polyp lesions and ulcer lesions in the total image of the w-th image in the positive label image group at the shooting time t m . γ and δ respectively represent the polyp influence weight and the ulcer influence weight, γ > δ > 0, and γ + δ = 1.

[0109] The lesion index is a comprehensive index used to quantify the severity of lesions in the image group, reflecting the risk of having colorectal cancer. It takes into account factors such as the proportion of different types of lesions in the image, the number of positive label images, and the weights of lesion types. The more the total amount of lesions in the image group, or the larger the proportion of the lesion area, the higher the lesion index. When the proportion of lesion types such as polyps or ulcers in the image is relatively high, the lesion index will increase. The lesion index not only reflects the number of lesions, but also considers the severity of different types of lesions. Polyp lesions are considered to have a greater impact on the progression of colorectal cancer. Therefore, when calculating the lesion index, polyp lesions are given a higher weight. If the proportion of polyp lesions in the image group is relatively large, the lesion index will increase.

[0110] Colorectal cancer usually evolves gradually from pre - existing lesions such as polyps. Polyps are the main precursors of colorectal cancer. In particular, large polyps have a higher potential for canceration. The larger the lesion index, the higher the proportion of lesions such as polyps or ulcers in the image. Especially larger polyps with a risk of canceration are more likely to appear in the image group with a high lesion index. This indicates that at this stage, the patient may already be in a pre - cancerous or high - risk state of cancer development. The larger the lesion index, the larger the lesion area or the greater the number of lesions in the image group, which usually indicates a wider range of lesions in the large intestine, possibly involving a larger area or more lesion sites. A larger lesion area, especially multiple lesions, usually means a higher risk of canceration because cancer often starts from local lesions and spreads or develops into multiple foci.

[0111] Ulcerative lesions are another potential sign of colorectal cancer, especially when the ulcerative lesions are extensive or show a trend of deterioration. Ulcerative lesions can lead to damage to the intestinal wall and loss of intestinal barrier function, further promoting the occurrence of canceration. The larger the lesion index, the more ulcerative lesions may be included in the image group, and these lesions usually indicate a higher cancer risk. Therefore, a higher lesion index is usually directly proportional to the likelihood of cancer development.

[0112] The lesion index not only represents the presence of lesions but also reflects the potential for deterioration or transformation of the lesions. As the number of lesions increases, especially when these lesions have malignant characteristics (such as changes in the size and shape of polyps, or the depth and scope of ulcers), the risk of cancer increases. A high lesion index means that the patient may have experienced long - term accumulation of lesions, and the lesions gradually evolve into forms or characteristics with the potential for canceration, thus increasing the probability of colorectal cancer. The larger the lesion index, the more severe the degree and scope of the lesions usually are, especially when these lesions have a high risk of canceration, such as large polyps or ulcerative lesions. These lesions are often early signs of the development of colorectal cancer. Therefore, the larger the lesion index, the higher the risk of the patient developing colorectal cancer.

[0113] The lesion index represents the overall lesion degree of the image group and is a comprehensive indicator for measuring the lesion situation of the endoscopic images at that moment. By calculating the lesion information of all positive-label images in each image group and combining the proportion of these images, the lesion index can quantify the severity of the lesions, thus providing a basis for subsequent screening and early warning, as well as the number of positive-label images of the lesions. Since positive-label images represent the presence of lesion areas in the images, their number directly affects the calculation of the lesion index. Specifically, the more positive-label images there are, the larger the affected range of the lesions may be. Therefore, it needs to be taken into account during the calculation. Each positive-label image contains different types of lesions, such as polyp lesions and ulcer lesions. By calculating the proportion of polyp lesions and ulcer lesions in the total images, the actual share of each lesion type in the images can be accurately reflected. At the same time, the weight coefficients in the formula are used to reflect the different influencing degrees of different lesion types on the lesion index. The influence of polyp lesions is greater, while the influence of ulcer lesions is relatively smaller.

[0114] By performing a weighted sum of the proportions of polyp lesions and ulcer lesions in each image, the influence of different lesion types in each image is considered, and the contributions of polyp and ulcer lesions to the lesion index are adjusted according to the weights γ and δ. This weighted sum method can comprehensively reflect the different contributions of different lesion types to the overall lesion index, ensuring that the calculation results are more in line with the actual lesion situation. The proportion of positive-label images, that is, the frequency of lesions in the image group, can reflect the universality of the lesions. The higher the proportion of positive-label images, the more common the lesions are in the image group, indicating a higher risk. By combining the proportion and frequency of the lesions, the lesion index can provide a comprehensive assessment of the total risk of the lesions.

[0115] Furthermore, the regression model is a polynomial regression model. Taking the time variable as the abscissa and the lesion index as the ordinate, the formula on which the regression model is based is:

[0116]

[0117] Among them, Bz(t) represents the dependent variable of the constructed regression model, t represents the time variable, that is, the independent variable, ∈ represents the number of image groups, τ represents the intercept of the constructed regression model, represents the time variable the coefficient of the

[0118] Substitute the time and lesion index corresponding to all image groups into the constructed regression model. Taking time as the independent variable and the lesion index as the variable, construct a ∈-element linear equation, calculate the intercept of the regression model and the coefficients of each power of the time variable to obtain the solved regression model.

[0119] The change in the lesion index is usually not linear, especially when the lesion develops over time. For example, as the degree of the lesion increases, there may be a trend of accelerating or slowing down. Using a polynomial regression model can better fit this non-linear change. By introducing higher-order terms of the time variable, the polynomial regression model provides greater flexibility and can fit the complex relationship that may exist between time and the lesion index. For example, the initial lesion may grow relatively slowly, and as the lesion develops, an accelerating trend may occur. Polynomial regression can effectively capture the change characteristics at different stages. The change law of the lesion index over time may be different for each image group. By combining the data of all image groups, the regression model can more accurately fit the lesion index at each time point, revealing the overall trend and the changes at each time stage. By calculating the coefficients in the regression model, the specific relationship between the lesion index and time can be quantified, providing more reliable data support for subsequent disease prediction and risk assessment.

[0120] The number of terms in the constructed model is closely related to the number of follow-up visits. As the number of follow-up visits (i.e., time points) increases, the model can obtain more lesion index data, and the model can use richer time series data for fitting, so as to more accurately capture the pattern of the change of the lesion index over time. With more data points, the model can better capture the true change trend of the lesion, reducing the fitting error caused by missing or incomplete data. It can be seen that the performance of the model in this embodiment is related to the number of follow-up visits, and the corresponding accuracy is also related to the number of follow-up visits. By introducing more higher-order terms of the time variable, the regression model can more flexibly and accurately fit the non-linear relationship between the lesion index and time. More follow-up data and the number of terms can improve the stability and generalization ability of the model and help identify the long-term change trend of the lesion index, thus making the prediction more accurate.

[0121] The screening and warning module is used to set the threshold of the dangerous lesion index, generate the corresponding time when the dangerous lesion index threshold is reached based on the constructed regression model, and mark it as the high-risk reference time. Calculate the time length from the high-risk reference time to the current time, mark it as the reference evaluation time, and issue the corresponding colorectal cancer warning prompt for the reference evaluation time.

[0122] In this embodiment, the method for calibrating the high-risk reference time is to set the threshold of the dangerous lesion index, substitute the dangerous lesion index threshold as the dependent variable into the completed regression model to calculate its corresponding time variable, mark this time variable as the high-risk reference time, and denote it as t. gw, The calculation of the high-risk reference time is actually a reverse process based on a regression model. By using the known threshold of the risk lesion index, the corresponding time is calculated backward. A risk lesion index threshold is set. When the lesion index reaches this threshold, it indicates entering the high-risk state. Substitute the set risk lesion index threshold as the dependent variable into the regression model, and when the high-risk reference time is reached in the model, the lesion index reaches this threshold. According to the regression equation, the high-risk reference time is a solution process that requires solving for the corresponding time variable through the known regression coefficients and threshold. There may be multiple solutions for this solution, so it is necessary to consider selecting a reasonable solution. For example, usually a positive time solution or a solution that conforms to the actual situation is selected. The high-risk reference time is obtained by setting the risk lesion index threshold, substituting it into the regression model, and calculating backward. The essence of the calculation is to solve a polynomial equation to obtain the corresponding time point. Through this time point, the reference evaluation time can be further calculated to issue a warning prompt for colorectal cancer. The risk lesion index threshold is obtained by acquiring the endoscopic images of existing stage III colorectal cancer patients, processing them using the above-mentioned preprocessing method, and then calculating it through the above-mentioned lesion index formula. Stage III is the high-risk stage of colon cancer because the cancer cells have started to metastasize to adjacent lymph nodes at this time. Therefore, the images of stage III are selected as a reference.

[0123] The formula for calculating the time length between the high-risk reference time and the current time is as follows:

[0124] t ck = t gw - t dq

[0125] where t ck represents the reference evaluation time, that is, the time length between the high-risk reference time and the current time, and t dq represents the current time.

[0126] When issuing the corresponding warning prompt for colorectal cancer, if t ck ≤ 6 mont, a high-risk warning is issued. The meaning of 6 mont is 6 months. That is, when the time to stage III colorectal cancer is less than 6 months, a high-risk warning is issued. 6 months is a common clinical monitoring period, and this time length can ensure that the metastasis risk of patients in the early stage of cancer can be evaluated in a timely manner. If it exceeds 6 months, the condition may have developed to a more serious stage, and the effect of early warning may be reduced.

[0127] The above formulas are all dimensionless and take their numerical values for calculation. The formula is obtained by collecting a large amount of data and performing software simulation to get a formula closest to the actual situation. The preset parameters in the formula are set by technicians in this field according to the actual situation.

[0128] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. They may be located in one place or distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the objectives of the solution of this embodiment.

[0130] As described above, the specific implementation manners of the present application are only described, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application.

Claims

1. A self-service colorectal cancer screening system based on image recognition, characterized in that: include: A data acquisition module, the data acquisition module is used to collect endoscopic images during multiple follow-up visits, and pre-process all endoscopic images, mark the shooting time of each image on the pre-processed endoscopic images, form an image group with the endoscopic images of the same time, map each image group with the shooting time one by one, and sort all the image groups in the order of shooting time to form a screening data set, wherein the endoscopic images are images obtained by colonoscopy; An image processing module, wherein the image processing module is used to collect endoscopic images containing different types of lesions as training images, and preprocess the training images to form a training image set, and annotate the images in the training image set with positive and negative labels, where a positive label indicates that the image contains a lesion area, and a negative label indicates that the image does not contain a lesion area; A model building module, wherein the model building module is used to label lesion labels for training images with positive labels, wherein the lesion labels include the type of lesion and the proportion of the lesion area in the total image, the labeled images constitute a training data set, a lesion recognition model based on the Faster R-CNN network is constructed, and the training data set is input into the lesion recognition model for training; A lesion recognition module, which is used to input the screening data set into the trained lesion recognition model, obtain the positive and negative label annotations and lesion label data of each endoscopic image in the screening data set, classify the endoscopic images with positive labels and the endoscopic images with negative labels, construct a positive label screening data set and a negative label screening data set, and calculate the proportion of images with positive labels in each image group; A regression analysis module, wherein the regression analysis module is used to construct a mathematical analysis model based on the lesion labels of each lesion type in each image group in the positive label screening data set and the proportion of images containing positive labels in each image group, generate a lesion index for each image group, and construct a regression model with the time variable as the horizontal axis and the lesion index as the vertical axis; A screening and early warning module is used to set a dangerous lesion index threshold, generate the time corresponding to the dangerous lesion index threshold based on the constructed regression model, and mark it as a high-risk reference time, calculate the time length from the high-risk reference time to the current time, mark it as a reference evaluation time, and issue a corresponding colorectal cancer early warning prompt for the reference evaluation time.

2. The self-service colorectal cancer screening system based on image recognition according to claim 1, characterized in that: The steps of preprocessing the acquired endoscopic images and training images include: The image size is scaled to 512*512 pixels, the color value of each pixel in the RGB color channel is obtained, and the RGB components of the image pixels are weighted to obtain the grayscale image of the image. The formula is: g(i,j)=0.3R(i,j)+0.6G(i,j)+0.1B(i,j) Where g(i, j) represents the grayscale value of the pixel in the i-th row and j-th column of the image, R(i, j), G(i, j) and B(i, j) represent the R, G and B color values ​​of the pixel in the i-th row and j-th column of the image in the RGB color channel respectively; For grayscale images, the median filter method is used to perform feature sharpening processing, and the grayscale value of each pixel in the grayscale image is set to the median of the grayscale values ​​of each adjacent pixel; The image that has been processed after feature sharpening is subjected to contrast enhancement processing, and the enhanced image is obtained after gain calculation for the high-frequency part of the image. The logic is as follows: The pixel point in the i-th row and j-th column of the image is marked as (i, j), and the mean K and variance σ of the image within the range (2n+1)×(2n+1) centered at the pixel point (i, j) are calculated. The formula for the mean calculation is: Where K(i, j) represents the mean value of the image within the range of (2n+1)×(2n+1) centered at (i, j), and (p, q) represents the grayscale value of the pixel point of the image within the range of (2n+1)×(2n+1) centered at (p, q). The variance calculation is based on the formula: Among them, (p, q)-K(i, j) represents the high-frequency part of the image, and the formula for calculating the gain of the high-frequency part of the image is: F(i,j)=K(i,j)+α[(p,q)-K(i,j)] Wherein, F(i, j) represents the grayscale value of the (i, j) pixel after gain, α is the gain weight, and α>1.

3. The self-service colorectal cancer screening system based on image recognition according to claim 1, characterized in that: The types of lesions include polyp lesions and ulcer lesions. When labeling lesion labels, the number of pixels occupied by the polyp or ulcer area and the total number of pixels in the image are counted to obtain the proportion of the lesion area in the total image. The formula is: Among them, Xz and Kz represent the proportion of polyp lesions and ulcer lesions in the total image, respectively, and N Xr and N Ky Respectively represent the number of pixels occupied by the polyp lesion part and the ulcer lesion part in the training image, N Z Represents the total number of pixels in the training image; If a training image contains both polyp lesions and ulcer lesions, the corresponding lesion types and the proportion of the corresponding lesion types are marked respectively.

4. The self-service colorectal cancer screening system based on image recognition according to claim 3, characterized in that: The method used to construct the positive label screening dataset and the negative label screening dataset is: Classify each image group after being identified by the lesion recognition model, divide the images with positive labels in each image group into one group and mark them as positive label image groups, divide the images with negative labels into one group and mark them as negative label image groups, and map the positive label image groups and negative label image groups to the shooting time one by one; After all image groups are classified, all positive label image groups constitute the positive label screening dataset, and all negative label image groups constitute the negative label screening dataset.

5. The self-service colorectal cancer screening system based on image recognition according to claim 4, characterized in that: The method to calculate the proportion of images with positive labels in each image group is: Count the total number of images in each image group and the number of images in the corresponding positive label image group, and calculate the proportion of images with positive labels in each image group. The formula is: in, Indicates the shooting time is t m The proportion of images with positive labels in the corresponding image group, and They represent the shooting time as t m In the corresponding image group, the number of images within the corresponding positive label image group and the total number of images in the image group.

6. The self-service colorectal cancer screening system based on image recognition according to claim 5, characterized in that: The method of constructing a mathematical analysis model and generating the lesion index for each image group is as follows: Obtain the lesion labels of each image in the corresponding positive label image group in each image group, that is, the proportion of polyp lesions and ulcer lesions in each image. Combined with the proportion of images with positive labels in each image group, a mathematical analysis model is constructed based on the formula: in, Indicates the shooting time is t m The lesion index of the corresponding image group, Indicates the shooting time is t m The number of images in the positive label image group, and They represent the shooting time as t m The proportion of polyp lesions and ulcer lesions in the w-th image in the positive label image group in the total image, γ and δ represent the influence weight of polyps and ulcers, respectively, γ>δ>0, and γ+δ=1.

7. The self-service colorectal cancer screening system based on image recognition according to claim 1, characterized in that: The regression model is a polynomial regression model, and the formula for constructing the regression model is based on the time variable as the horizontal axis and the lesion index as the vertical axis: Among them, Bz(t) represents the dependent variable of the constructed regression model, t represents the time variable, i.e., the independent variable, ∈ represents the number of image groups, and τ represents the intercept of the constructed regression model. Represents time variable The coefficient of the power; Substitute the time and lesion index corresponding to all image groups into the constructed regression model, take time as the independent variable and lesion index as the variable, construct an ∈ linear equation, calculate the intercept of the regression model and the coefficient of each power of the time variable, and obtain the solved regression model.

8. The self-service colorectal cancer screening system based on image recognition according to claim 7, characterized in that: The method for calibrating the high-risk reference time is: Set the threshold of the dangerous lesion index, substitute the threshold of the dangerous lesion index as the dependent variable into the solved regression model, calculate the corresponding time variable, mark the time variable as the high-risk reference time, and record it as t gw , calculate the time length from the high-risk reference time to the current time, based on the formula: t ck =t gw -t dq Among them, t ck Indicates the reference assessment time, that is, the time length from the high-risk reference time to the current time, t dq Indicates the current time; When the corresponding colorectal cancer warning is issued, if t ck If ≤6mont, a high-risk warning will be issued.

Images