Disease condition diagnosis auxiliary method and system based on pupil image processing

Abstract

The invention belongs to the technical field of medical image processing, and particularly relates to an illness state diagnosis auxiliary method and system based on pupil image processing, and the method comprises the steps: collecting an image based on pupil focusing, and carrying out the preprocessing; carrying out image recognition on the preprocessed image again to obtain a pupil area, and then carrying out calibration; performing image cutting based on the calibrated pupil position area, then performing pupil edge feature extraction, and performing image cutting again to obtain a final pupil image; color shape features are extracted and obtained, and color shape feature vectors and size information of left and right pupils are established; repeatedly executing acquisition of color shape feature vectors and size information of the left and right pupils at multiple time nodes; calculating color and shape change information of the pupils along with time, and drawing a size change curve of the left and right pupils of the patient along with time; therefore, the illness state change trend of the patient is obtained. Pupil change and pupil data under each diagnosis time node can be accurately acquired to assist in disease diagnosis of a patient.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and particularly relates to a method and system for assisting in disease diagnosis based on pupil image processing. Background Art

[0002] Pupil reflex is an important part of ophthalmological and neurological examinations. By observing the characteristics and dynamic changes of the pupillary light reflex, the functional status of the nervous system, especially the oculomotor nerve (the third cranial nerve) and the sympathetic nervous system, can be judged. Pupil changes are helpful for the condition judgment of patients with coma, convulsions, shock, poisoning, etc., and are particularly important for the localization diagnosis of craniocerebral lesions. For example, bilateral pupillary dilation and disappearance of direct and indirect light reflexes may indicate bilateral optic nerve damage. Pupil changes are helpful for the condition judgment of patients with coma, convulsions, shock, poisoning, etc., and are particularly important for the localization diagnosis of craniocerebral lesions. For example, bilateral pupillary dilation and disappearance of direct and indirect light reflexes may indicate bilateral optic nerve damage. Specific pupillary reflex abnormalities, such as disappearance or asymmetry of the pupillary light reflex, can provide important clues for neurological diseases. Pupil examination is also helpful for predicting the prognosis of certain eye diseases. Therefore, pupil examination is of great significance in diagnostics and can provide important information about the patient's nervous system, eyes and systemic diseases for doctors, so as to guide subsequent treatment and care.

[0003] In current clinical medical diagnosis, the pupil state is used to assist in disease diagnosis in some cases. However, at present, the disease diagnosis is still assisted based on the doctor's visual inspection of the pupil. Since the size of the pupil itself is relatively small, when there are relatively small changes in the pupil, the doctor's visual diagnosis may not be accurate, making it impossible to provide a precise data basis for assisting in disease diagnosis.

[0004] Therefore, how to process the pupil image to accurately obtain the change data of the pupil and various pupil data at each diagnostic time node to assist in the disease diagnosis of patients and timely implement corresponding treatment plans is a technical problem that urgently needs to be solved at present. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for assisting in disease diagnosis based on pupil image processing, which can accurately obtain the change data of the pupil and various pupil data at each diagnostic time node by processing the pupil image, so as to assist in the disease diagnosis of patients and timely implement corresponding treatment plans.

[0006] To solve the above technical problems, the technical solutions adopted by the present invention are as follows: A method and system for assisting in disease diagnosis based on pupil image processing.

[0007] A method for assisting in disease diagnosis based on pupil image processing, comprising the following steps: S1: At the first time node, collect images of the left and right eyes with completely exposed pupils at a specified distance from the patient's eyes. During the image collection process, preliminarily identify the pupil position, focus based on the pupil position and then collect the image, and preprocess the collected image; S2: Perform image recognition again on the preprocessed image, obtain the pupil area from the preprocessed image, enlarge the obtained pupil area by a specified ratio with the center point as the center, and calibrate the enlarged pupil position area by a specified ratio; S3: Perform image cutting based on the calibrated pupil position area, extract the pupil edge features of the cut pupil position area, and perform image cutting again based on the extracted pupil edge features to obtain the final pupil image; S4: Extract the color features and shape features of the pupils from the final pupil image, establish the color feature vectors and shape feature vectors of the left and right pupils at the first time node, measure the pupil size, and obtain the size information of the left and right pupils at the first time node; S5: Repeat steps S1 - S4 to obtain the color feature vectors and size information of the left and right pupils at the second time node, the third time node, and other specified time nodes; S6: Calculate the color and shape change information of the patient's pupils over time based on the color feature vectors at each time node, and respectively draw the size change curves of the patient's left and right pupils over time based on the size information at each time node; S7: Retrieve the corresponding color data, shape data, and size data of the standard pupils according to the patient's basic information, compare them with the pupil color data, shape data, and size data at the specified time node to assist the doctor in obtaining the patient's condition at the specified time node, and assist in obtaining the trend of the patient's condition change based on the patient's color and shape change information and size change curve.

[0008] Preferably, the specific process of preliminarily identifying the pupil position, focusing based on the pupil position and then collecting the image, and preprocessing the collected image in step S1 is as follows: S11: Preset the pupil color, shape, and size range data in the image collection module. When the image collection module performs image collection, automatically obtain the area that meets the preset color, shape, and size range, and focus based on this area; S12: Remove the invalid images, images with a resolution lower than the preset threshold, and duplicate images in the collected images, and adjust the brightness values of each pixel point in the R, G, and B channels of the image. The specific adjustment formula is: R = G = B = (Wr * R + Wg * G + Wb * B) / 3; Wherein, Wr refers to the preset weight value for the R channel, Wg refers to the preset weight value for the G channel, and Wb refers to the preset weight value for the B channel; S13: Subject the image processed in step S12 to a specified transformation, then filter out the spectral components with sudden pixel gray value changes in the spectral components of the image after the specified transformation through a preset filter, and finally obtain the final image through the reverse process of the specified transformation.

[0009] Preferably, in step S2, the preprocessed image is subjected to image recognition again, and the specific process of obtaining the pupil area from the preprocessed image is as follows: S21: Create an image recognition model, input a specified number of labeled pupil image data into the image recognition model, and train the image recognition model; S22: Input the preprocessed image into the image recognition model, and the image recognition model divides the input image into uniformly sized image blocks of a specified size, and then detects the targets whose centers fall within the image blocks; S23: The image recognition model sequentially predicts the bounding boxes in each image block. The information contained in the predicted bounding boxes includes the center coordinates of the bounding box, the height and width of the bounding box, and the confidence value of the target included in the bounding box. The image recognition model predicts the probabilities of a specified number of categories in each image block; S24: Preset a confidence threshold, compare the confidence value of each bounding box predicted by the image recognition model with the confidence threshold, and extract the bounding boxes whose confidence values are greater than the confidence threshold; S25: Remove the overlapping bounding boxes in each category, and retain the bounding boxes with the highest confidence value and the ratio of the intersection to the union between the corresponding predicted bounding box and the standard bounding box lower than the preset value; S26: Combine the prediction results in all image blocks to obtain the final bounding boxes and class labels.

[0010] Preferably, the image recognition model includes an input layer, two combined layers of convolutional layer + pooling layer, and a fully connected layer. Among them, the first convolutional layer in the combined layer is provided with 3 2x2 convolutional kernels, the second convolutional layer is provided with 5 1x1 convolutional kernels, the first pooling layer downsamples the feature map obtained by the first convolutional layer, and the second pooling layer downsamples the feature map obtained by the second convolutional layer.

[0011] Preferably, the specific process of extracting pupil edge features from the cut pupil position area in step S3 is as follows: S31: Calculate the gradient intensity and direction of each pixel in the pupil position area of the image, retain the pixels with local maxima in the gradient direction, and remove the pixels with gradient directions lower than the local maxima. S32: Preset two gradient thresholds, including a large gradient threshold and a small gradient threshold. Map the gradient amplitude to the two preset gradient thresholds, mark the pixels with values greater than the large gradient threshold as strong edges, and mark the pixels between the large gradient threshold and the small gradient threshold as non-edges to obtain the edge features of the pupil. S33: Connect adjacent strong edge pixels in sequence to form the final pupil edge.

[0012] Preferably, in step S4, the specific process of extracting the color features and shape features of the pupil from the final pupil image and establishing the color feature vectors of the left and right pupils at the first time node is as follows: S41: Extract the values on the R channel of the pixels in the pupil image and establish an R channel matrix according to the pixel coordinates. S42: Extract the values on the G channel of the pixels in the pupil image and establish a G channel matrix according to the pixel coordinates. S43: Extract the values on the B channel of the pixels in the pupil image and establish a B channel matrix according to the pixel coordinates. S44: Construct the color feature vector of the pupil image from the R channel matrix, G channel matrix, and B channel matrix.

[0013] Preferably, in step S4, the specific process of measuring the pupil size and obtaining the size information of the left and right pupils at the first time node is as follows: S45: Randomly select an edge pixel of the pupil as the starting point, obtain the pixel that is centrosymmetric with the starting point among the edge pixels, and obtain the distance between this pixel and the starting point. S46: Traverse the distances between all pixels from the starting point to the centrosymmetric pixel and their corresponding centrosymmetric pixels in a clockwise or counterclockwise direction. S47: Take the average value of all the distances as the diameter of the pupil, which is the size of the pupil.

[0014] In a second aspect, a disease diagnosis assistance system based on pupil image processing is provided for implementing the disease diagnosis assistance method based on pupil image processing described above. The system includes an image acquisition module, a preprocessing module, an image recognition module, an image calibration module, an image cutting module, a feature extraction module, a size measurement module, and a curve drawing module. The image acquisition module is connected to the preprocessing module, the preprocessing module is connected to the image recognition module, the image recognition module is connected to the image calibration module, the image calibration module is connected to the image cutting module, the image cutting module is connected to the feature extraction module, the feature extraction module is connected to the size measurement module, and the size measurement module is connected to the curve drawing module; The image acquisition module is configured to collect images of the left and right eyes with fully exposed pupils at a position at a specified distance from the patient's eyes at a first time node, preliminarily identify the pupil position during the image acquisition process, and collect images after focusing based on the pupil position; The preprocessing module is configured to preprocess the collected images; The image recognition module is configured to perform image recognition on the preprocessed images again and obtain the pupil region from the preprocessed images; The image calibration module is configured to magnify the obtained pupil region by a specified ratio with the center point as the center and calibrate the magnified pupil position region by the specified ratio; The image cutting module is configured to perform image cutting based on the calibrated pupil position region and perform image cutting again based on the extracted pupil edge features to obtain the final pupil image; The feature extraction module is configured to extract pupil edge features from the cut pupil position region, extract the color features and shape features of the pupils from the final pupil image, and establish color feature vectors and shape feature vectors of the left and right pupils at the first time node; The size measurement module is configured to measure the pupil size and obtain the size information of the left and right pupils at the first time node; The curve drawing module is configured to calculate the color and shape change information of the patient's pupils over time based on the color feature vectors at each time node, and draw the size change curves of the patient's left and right pupils over time based on the size information at each time node.

[0015] The beneficial effects of the present invention include: The disease diagnosis assistance method and system based on pupil image processing provided by the present invention preprocess the image after collecting the image based on pupil focusing; perform secondary image recognition on the preprocessed image to obtain the pupil area and then perform calibration; perform image cutting based on the calibrated pupil position area and then extract the pupil edge features, and perform secondary image cutting to obtain the final pupil image; extract the color features, establish the color feature vectors and size information of the left and right pupils; repeatedly execute to obtain the color feature vectors and size information of the left and right pupils at multiple time nodes; calculate the color and shape change information of the pupils over time, and draw the size change curves of the left and right pupils of the patient over time; assist in obtaining the disease change trend of the patient according to the color and shape change information and size change curves of the patient. It can accurately obtain the pupil changes and pupil data at each diagnosis time node to assist in the disease diagnosis of the patient.

[0016] First, preset the data of the pupil color, shape and size range in the image acquisition module, and focus on the pupil area, so that the resolution of the pupil area is high, which is convenient for subsequent image processing. Remove the noise image data, adjust the brightness values of each pixel point on the three channels of the image, thereby greatly reducing the complexity of subsequent image processing and improving the image processing efficiency. After the specified transformation processing, then filter the spectral components with sudden changes in pixel gray levels in the spectral components of the image after the specified transformation processing through a preset filter, and then obtain the final image through the reverse processing of the specified transformation, so that the pupil area is significantly enhanced, providing an effective data basis for subsequent image feature extraction. Compared with the existing image preprocessing process, the processed quality is significantly improved and the processing efficiency is higher.

[0017] Second, create and train an image recognition model. Input the preprocessed image, divide the input image into uniformly sized image blocks of a specified size, and then detect the targets whose centers fall within the image blocks; the image recognition model sequentially predicts the bounding boxes in each image block and predicts the probabilities of a specified number of classes, and extracts the bounding boxes with confidence levels greater than the confidence threshold; retain the bounding boxes with the highest confidence value and the ratio of the intersection to the union between the corresponding predicted bounding box and the standard bounding box lower than the preset value; merge the prediction results in all image blocks to obtain the final bounding box and class label, which can achieve accurate recognition of the pupil area, and then obtain the pupil area, facilitating subsequent precise processing of this area.

[0018] Third, calculate the gradient intensity and direction of the pixel points in the pupil area, retain the pixel points with local maximum values in the gradient direction, preset a large gradient threshold and a small gradient threshold, mark the pixel points greater than the large gradient threshold as strong edges, and mark the pixel points between the large gradient threshold and the small gradient threshold as non-edges to obtain the edge features of the pupil. Connect the adjacent strong edge pixel points in sequence to accurately obtain the pupil edge, providing an accurate object for subsequent color feature extraction and size measurement of the pupil, especially improving the accuracy of the size measurement results.

[0019] Fourth, extract the values of the pixel points in the R, G, and B channels of the pupil image, establish R, G, and B channel matrices according to the pixel point coordinates respectively, and construct the color feature vector of the pupil image from the R channel matrix, G channel matrix, and B channel matrix. By establishing the color feature vectors at each time node, it is convenient to analyze the color features to assist doctors in disease diagnosis.

[0020] Fifth, randomly select the edge pixel points of the pupil as the starting point, obtain the pixel points that are centrosymmetric with the starting point among the edge pixel points, obtain the distance between this pixel point and the starting point, and traverse all the pixel points between the starting point and the centrosymmetric pixel point in the clockwise or counterclockwise direction to obtain the distances between the corresponding centrosymmetric pixel points respectively; take the average value of all the distances as the diameter of the pupil, which is the size of the pupil, achieving accurate measurement of the pupil size. Compared with the doctor's visual judgment in the prior art, it greatly improves the accuracy of the pupil size, and can combine the size change curves of multiple time nodes to accurately assist in disease diagnosis. Description of the Drawings

[0021] Figure 1 It is a schematic flowchart of a disease diagnosis assistance method based on pupil image processing according to the present invention.

[0022] Figure 2 It is a schematic flowchart of image recognition and obtaining the pupil area according to the present invention.

[0023] Figure 3 It is a schematic architecture diagram of a disease diagnosis assistance system based on pupil image processing according to the present invention. Detailed Embodiments

[0024] The following further elaborates on the present invention in conjunction with the attached Figures 1 to 3 for a more detailed description: Embodiment 1 Referring to the attached Figure 1 As shown, a disease diagnosis assistance method based on pupil image processing includes the following steps: S1: At the first time node, image acquisition is performed on the left and right eyes with the pupils completely exposed at a specified distance from the patient's eyes. During the image acquisition process, the pupil position is initially identified, and after focusing based on the pupil position, an image is acquired and the acquired image is preprocessed. The first time node is the first image acquisition of the pupils after the patient is admitted to the hospital. At the first time node, the doctor will also obtain other data of the patient to facilitate the doctor's comprehensive diagnosis of the condition by combining other data and pupil data, making the diagnosis of the condition more accurate.

[0025] S2: Perform image recognition again on the preprocessed image, obtain the pupil area from the preprocessed image, magnify the obtained pupil area by a specified ratio with the center point as the center, and calibrate the magnified pupil position area by a specified ratio. When the pupil area in the image is initially obtained, due to the lack of accurate feature extraction, the accuracy of image recognition of the pupil area is relatively low. The recognition of the pupil area at this stage is to obtain a rough image processing area to avoid processing the entire image, which can greatly reduce the data processing volume, reduce the complexity of subsequent image processing, and improve the efficiency of subsequent image processing.

[0026] S3: Perform image cutting based on the calibrated pupil position area, so that only the cut area is subjected to feature extraction in the subsequent process. The area for feature extraction is reduced, greatly improving the feature extraction efficiency. Perform pupil edge feature extraction on the cut pupil position area, and perform image cutting again based on the extracted pupil edge features to obtain the final pupil image.

[0027] S4: Extract the color features and shape features of the pupils from the final pupil image, establish the color feature vectors and shape feature vectors of the left and right pupils at the first time node, measure the pupil size, and obtain the size information of the left and right pupils at the first time node. Step S4 obtains extremely accurate pupil size information, facilitating the doctor's diagnosis of the patient's condition by combining other data.

[0028] S5: Repeat steps S1 - S4 to obtain the color feature vectors and size information of the left and right pupils at the second time node, the third time node, and other specified time nodes. Through the color feature vectors and size information at multiple time nodes, it is convenient to obtain the color and shape change information and size change data of the patient's pupils over time.

[0029] S6: Calculate the color and shape change information of the patient's pupils over time based on the color feature vectors at each time node, and draw the size change curves of the patient's left and right pupils over time based on the size information at each time node.

[0030] S7: Retrieve the color data, shape data, and size data of the corresponding standard pupil according to the patient's basic information, compare them with the pupil color data, shape data, and size data at the specified time node, assist the doctor in obtaining the patient's condition at the specified time node, and assist in obtaining the trend of the patient's condition change based on the patient's color and shape change information and size change curve.

[0031] In this embodiment, first, preprocess the image after collecting the image based on pupil focusing; then perform image recognition again on the preprocessed image to obtain the pupil area and perform calibration; extract the pupil edge features after image cutting based on the calibrated pupil position area, and perform image cutting again to obtain the final pupil image; extract the color features, establish the color feature vectors and size information of the left and right pupils; repeat to obtain the color feature vectors and size information of the left and right pupils at multiple time nodes; calculate the color and shape change information of the pupils over time, and draw the size change curves of the patient's left and right pupils over time; assist in obtaining the trend of the patient's condition change based on the patient's color and shape change information and size change curve. This process can accurately obtain the pupil changes and the pupil data at each diagnosis time node to assist in the diagnosis of the patient's condition.

[0032] Embodiment 2 Based on Embodiment 1, in step S1, during the image acquisition process, preliminarily identify the pupil position, focus on the image based on the pupil position, and the specific process of preprocessing the acquired image is as follows: S11: Preset the pupil color, shape, and size range data in the image acquisition module. When the image acquisition module performs image acquisition, it automatically obtains the area that meets the preset color, shape, and size range, and focuses based on this area. The image acquisition module presets information such as pupil shape and size. Therefore, when performing image acquisition, it will automatically capture the feature of the object in the picture, match it with the preset pupil shape and size, obtain the pupil area, and perform focusing based on the pupil area.

[0033] S12: Remove the invalid images, images with a resolution lower than the preset threshold, and duplicate images in the acquired images, and adjust the brightness values of each pixel point on the R, G, and B channels of the image. The specific adjustment formula is: R = G = B = (Wr * R + Wg * G + Wb * B) / 3; Where, Wr refers to the preset weight value for the R channel, Wg refers to the preset weight value for the G channel, and Wb refers to the preset weight value for the B channel.

[0034] S13: Process the image processed in step S12 through a specified transformation, then filter out the spectral components with sudden pixel gray value changes in the spectral components of the image after the specified transformation using a preset filter, and finally obtain the final image through the inverse process of the specified transformation.

[0035] In this embodiment, by presetting the pupil color, shape, and size range data in the image acquisition module and focusing based on the pupil region, the resolution of the pupil region is high, facilitating subsequent image processing. Remove the noise image data, adjust the brightness values of each pixel point on the three channels of the image, thereby greatly reducing the complexity of subsequent image processing and improving the image processing efficiency. After the specified transformation processing, then filter out the spectral components with sudden pixel gray value changes in the spectral components of the image after the specified transformation using a preset filter, and finally obtain the final image through the inverse process of the specified transformation, making the pupil region significantly enhanced, providing an effective data basis for subsequent image feature extraction. Compared with the existing image preprocessing process, the processed quality is significantly improved and the processing efficiency is higher.

[0036] The specific process in step S13 is as follows: First, regard the image as a two-dimensional array, then perform one-dimensional FFT on each row, and then perform one-dimensional FFT on each column. Specifically, first perform FFT on the N points in the 0th row (the real part has values and the imaginary part is 0), and put the real and imaginary parts of the FFT output back to the corresponding positions in the original 0th row. After calculating all the rows in this way, the real and imaginary parts of the image contain intermediate data, and then use the same method to perform FFT transformation in the column direction, finally obtaining an N*N spectrum. The result of FFT is a complex number array, which is difficult to visualize directly. Usually, it is visualized through two methods: spectrum and phase angle. The white area in the spectrum image represents the high-frequency part, while the white area in the corner represents the low-frequency part. The phase angle retains the shape characteristics of the image. In the frequency domain, various processes can be performed on the spectrum, such as low-pass filtering, high-pass filtering, etc. The processed spectrum can be converted back to the time domain through the inverse fast transformation of the specified transformation, thereby obtaining the processed image. This process converts the complex convolution operation in the spatial domain into a simple multiplication operation in the frequency domain, thus simplifying the subsequent calculation process, converting the image from the spatial domain to the frequency domain, and obtaining the spectrum diagram of the image. This helps to analyze different frequency components in the image, such as low frequency, medium frequency, and high frequency. Low frequency usually corresponds to the smooth area in the image, while high frequency corresponds to the edge and detail parts in the image.

[0037] Embodiment 3 Based on Embodiment 1 or Embodiment 2, the specific process of obtaining the pupil region from the preprocessed image by performing image recognition on the preprocessed image again in step S2 is as follows: S21: Create an image recognition model, input a specified number of labeled pupil image data into the image recognition model, and train the image recognition model. After the image recognition model is trained, in order to ensure that the image recognition model meets specific requirements, it is also necessary to verify it. Input other pupil images into the model, identify the pupil area through the image recognition model, and output the recognition result. Judge the performance of the model according to the recognition result, and execute subsequent steps after meeting the requirements.

[0038] S22: Input the preprocessed image into the image recognition model. The image recognition model divides the input image into evenly sized image blocks of a specified size, and then detects the targets whose centers fall within the image blocks. Dividing the image into image blocks facilitates subsequent target detection because it is relatively difficult to detect targets when the image size is relatively large. S23: The image recognition model sequentially predicts the bounding boxes in each image block. The information contained in the predicted bounding boxes consists of the center coordinates of the bounding box, the height and width of the bounding box, and the confidence value of the target contained in the bounding box. The image recognition model predicts the probabilities of a specified number of classes in each image block. S24: Preset a confidence threshold, compare the confidence value of each bounding box predicted by the image recognition model with the confidence threshold, and extract the bounding boxes whose confidence values are greater than the confidence threshold. Confidence reflects, on the one hand, the likelihood that the bounding box contains a plaque, and on the other hand, the accuracy of this bounding box.

[0039] S25: Remove the overlapping bounding boxes in each class, and retain the bounding boxes with the highest confidence value and the ratio of the intersection to the union between the corresponding predicted bounding box and the standard bounding box lower than the preset value. S26: Combine the prediction results in all image blocks to obtain the final bounding boxes and class labels.

[0040] The accuracy of the bounding box can be characterized by the intersection over union of the predicted box and the actual box. It is the product of two factors, and the accuracy of the predicted box is also reflected in it. The size and position of the bounding box can be characterized by the values of 4 elements: (x, y, w, h), where (x, y) are the center coordinates of the bounding box, and w and h are the width and height of the bounding box. The predicted values of the center coordinates (x, y) are the offset values relative to the upper left corner coordinate point of each cell, and the unit is relative to the cell size. The predicted values of w and h of the bounding box are the ratios relative to the width and height of the entire picture. The sizes of the 4 elements should be in the range of [0, 1]. In this way, the predicted value of each bounding box actually contains 5 elements: (x, y, w, h, c), where the first 4 characterize the size and position of the bounding box, and the last value is the confidence.

[0041] In this embodiment, the image recognition model includes an input layer, a combined layer of convolutional layer + pooling layer * 2, and a fully connected layer. Among them, the first convolutional layer in the combined layer is provided with 3 2x2 convolutional kernels, the second convolutional layer is provided with 5 1x1 convolutional kernels, the first pooling layer downsamples the feature map obtained by the first convolutional layer, and the second pooling layer downsamples the feature map obtained by the second convolutional layer.

[0042] Embodiment 4 Based on Embodiment 1, Embodiment 2, or Embodiment 3, the specific process of extracting the pupil edge features from the cut pupil position area in step S3 is as follows: S31: Calculate the gradient intensity and direction of each pixel point in the pupil position area of the image, and retain the pixel points with local maximum values in the gradient direction, and remove the pixel points with values lower than the local maximum in the gradient direction. S32: Preset two gradient thresholds, including a large gradient threshold and a small gradient threshold. Map the gradient amplitude to the two preset gradient thresholds, mark the pixel points greater than the large gradient threshold as strong edges, and mark the pixel points between the large gradient threshold and the small gradient threshold as non-edges to obtain the edge features of the pupil. S33: Connect adjacent strong edge pixel points in sequence to form the final pupil edge.

[0043] The above process calculates the gradient intensity and direction of the pixel points in the pupil area, retains the pixel points with local maximum values in the gradient direction, presets the large gradient threshold and the small gradient threshold, marks the pixel points greater than the large gradient threshold as strong edges, marks the pixel points between the large gradient threshold and the small gradient threshold as non-edges to obtain the edge features of the pupil, and connects adjacent strong edge pixel points in sequence, realizing the accurate acquisition of the pupil edge, providing an accurate object for subsequent extraction of the color features and size measurement of the pupil, especially improving the accuracy of the size measurement result.

[0044] In this embodiment, the specific process of extracting the color features and shape features of the pupil from the final pupil image and establishing the color feature vector of the left and right pupils at the first time node in step S4 is as follows: S41: Extract the values on the R channel of the pixel points in the pupil image and establish an R channel matrix according to the pixel point coordinates. S42: Extract the values on the G channel of the pixel points in the pupil image and establish a G channel matrix according to the pixel point coordinates. S43: Extract the values on the B channel of the pixel points in the pupil image and establish a B channel matrix according to the pixel point coordinates. S44: Construct the color feature vector of the pupil image from the R channel matrix, G channel matrix, and B channel matrix.

[0045] In step S4, the pupil size is measured. Refer to Figure 2 As shown, the specific process of obtaining the size information of the left and right pupils at the first time node is as follows: S45: Randomly select an edge pixel point of the pupil as the starting point, obtain the pixel point that is centrosymmetric with the starting point among the edge pixel points, and obtain the distance between this pixel point and the starting point; S46: Traverse the distances between all pixel points from the starting point to the centrosymmetric pixel point in a clockwise or counterclockwise direction and their corresponding centrosymmetric pixel points respectively; S47: Take the average value of all the distances as the diameter of the pupil, which is used as the size of the pupil.

[0046] The system presets a corresponding table of possible color feature vectors, shape feature vectors, and pupil size and diseases. When the color feature vector and size data of the pupil at a certain time node are obtained, the color feature vector and size data are matched with the corresponding table, which can assist doctors in diagnosing the corresponding condition at this time node. Since the color feature vectors and size data at multiple time nodes over time are obtained, the color shape change information and size change information of the pupil within a certain period of time can be known. Since the color change and size change of the pupil reflect the condition change of the patient to a certain extent, when doctors diagnose the change of the condition within a certain period of time, they can use the color shape change and size change data for auxiliary diagnosis. For example, within the period when the color feature vectors and size data at multiple time points are obtained, it can be known whether the patient's condition is worsening or improving according to the degree of color change. When the size of the patient's pupil is continuously increasing during this period, it reflects to a certain extent that the patient's condition is worsening. If there are fluctuations in the size change of the patient's pupil during this period, it may reflect that there are also certain fluctuations in the patient's condition. Then, combined with other data of the patient, the patient's condition can be judged more accurately.

[0047] A disease diagnosis assistance system based on pupil image processing is used to implement the above-mentioned disease diagnosis assistance method based on pupil image processing. Refer to Figure 3 As shown, it includes an image acquisition module, a preprocessing module, an image recognition module, an image calibration module, an image cutting module, a feature extraction module, a size measurement module, and a curve drawing module. The image acquisition module is connected to the preprocessing module, the preprocessing module is connected to the image recognition module, the image recognition module is connected to the image calibration module, the image calibration module is connected to the image cutting module, the image cutting module is connected to the feature extraction module, the feature extraction module is connected to the size measurement module, and the size measurement module is connected to the curve drawing module.

[0048] The image acquisition module is used to acquire images of the left and right eyes with completely exposed pupils at a position with a specified distance from the patient's eyes at the first time node, preliminarily identify the pupil position during the image acquisition process, and acquire images after focusing based on the pupil position; the preprocessing module is used to preprocess the acquired images; the image recognition module is used to perform image recognition again on the preprocessed images and obtain the pupil region from the preprocessed images; the image calibration module is used to magnify the obtained pupil region by a specified ratio with the center point as the center and calibrate the pupil position region magnified by the specified ratio.

[0049] The image cutting module is used to perform image cutting based on the calibrated pupil position region and perform image cutting again based on the extracted pupil edge features to obtain the final pupil image; the feature extraction module is used to extract the pupil edge features of the cut pupil position region and extract the color features and shape features of the pupil for the final pupil image, and establish the color feature vectors and shape feature vectors of the left and right pupils at the first time node; the size measurement module is used to measure the pupil size and obtain the size information of the left and right pupils at the first time node; the curve drawing module is used to calculate the color and shape change information of the patient's pupils over time according to the color feature vectors at each time node, and draw the size change curves of the patient's left and right pupils over time according to the size information at each time node.

[0050] In summary, the disease diagnosis assistance method and system based on pupil image processing provided by the present invention preprocess the image after collecting the image based on pupil focusing; perform secondary image recognition on the preprocessed image to obtain the pupil area and then calibrate it; perform image cutting based on the calibrated pupil position area and then extract the pupil edge features, and perform secondary image cutting to obtain the final pupil image; extract the color features, establish the color feature vectors and size information of the left and right pupils; repeatedly execute to obtain the color feature vectors and size information of the left and right pupils at multiple time nodes; calculate the color and shape change information of the pupils over time, and draw the size change curves of the left and right pupils of the patient over time; assist in obtaining the disease change trend of the patient according to the color and shape change information and size change curves of the patient. It can accurately obtain the pupil changes and pupil data at each diagnosis time node to assist in the disease diagnosis of the patient. The preset pupil color, shape and size range data in the image acquisition module are focused based on the pupil area, so that the resolution of the pupil area is high, which is convenient for subsequent image processing. Remove the noise image data, adjust the brightness values of each pixel point on the three channels of the image, thereby greatly reducing the complexity of subsequent image processing and improving the image processing efficiency. After the specified transformation processing, then filter the spectral components with sudden changes in pixel gray levels in the spectral components of the image after the specified transformation processing through a preset filter, and then obtain the final image through the reverse processing of the specified transformation, so that the pupil area is significantly enhanced, providing an effective data basis for subsequent image feature extraction. Compared with the existing image preprocessing process, the processed quality is significantly improved and the processing efficiency is higher.

[0051] By creating an image recognition model and training it, input the preprocessed image, divide the input image into uniformly sized specified image blocks, and then detect the targets whose centers fall within the image blocks; the image recognition model predicts the bounding boxes in each image block in turn and predicts the probabilities of specified categories, and extracts the bounding boxes with confidence levels greater than the confidence threshold; retain the bounding boxes with the highest confidence value and the ratio of the intersection to the union between the corresponding predicted bounding box and the standard bounding box lower than the preset value; merge the prediction results in all image blocks to obtain the final bounding box and class label, which can achieve accurate recognition of the pupil area, and then obtain the pupil area, facilitating subsequent precise processing of this area. Calculate the gradient intensity and direction of the pixel points in the pupil area, retain the pixel points with local maximum values in the gradient direction, preset the large gradient threshold and the small gradient threshold, mark the pixel points greater than the large gradient threshold as strong edges, and mark the pixel points between the large gradient threshold and the small gradient threshold as non-edges to obtain the edge features of the pupil, and connect the adjacent strong edge pixel points in turn, realizing the accurate acquisition of the pupil edge, providing an accurate object for subsequent color feature extraction and size measurement of the pupil, especially improving the accuracy of the size measurement results.

[0052] Extract the values of the pixels in the pupil image on the R, G, and B channels, and establish R, G, and B channel matrices according to the pixel coordinates respectively. Construct a color feature vector of the pupil image from the R channel matrix, G channel matrix, and B channel matrix. By establishing the color feature vectors at each time node, it is convenient to analyze the color features to assist doctors in diagnosing diseases. Randomly select an edge pixel point of the pupil as the starting point, obtain the pixel point that is centrosymmetric with the starting point among the edge pixel points, obtain the distance between this pixel point and the starting point, and traverse all the pixel points between the starting point and the centrosymmetric pixel point in the clockwise or counterclockwise direction to obtain the distances between the corresponding centrosymmetric pixel points respectively; take the average value of all the distances as the diameter of the pupil, which is used as the size of the pupil, realizing the accurate measurement of the pupil size. Compared with the judgment by doctors with the naked eye in the prior art, the accuracy of the pupil size is greatly improved, and the size change curves of multiple time nodes can be combined to accurately assist in disease diagnosis.

Claims

1. A disease diagnosis auxiliary method based on pupil image processing, characterized in that: The following steps are involved: S1: at a first time point, at a position at a specified distance from the patient's eyes, images of the left and right eyes with completely exposed pupils are collected, the pupil position is preliminarily identified during the image collection process, images are collected after focusing based on the pupil position, and the collected images are preprocessed; S2: performing image recognition again on the preprocessed image, obtaining a pupil area from the preprocessed image, enlarging the obtained pupil area by a specified ratio with the center point as the center of the circle, and calibrating the pupil position area enlarged by the specified ratio; S3: performing image segmentation based on the calibrated pupil position area, extracting pupil edge features from the segmented pupil position area, and performing image segmentation again based on the extracted pupil edge features to obtain a final pupil image; S4: extracting color features and shape features of the pupil from the final pupil image, establishing color feature vectors and shape feature vectors of the left and right pupils at the first time node, measuring pupil size, and obtaining size information of the left and right pupils at the first time node; S5: Repeat steps S1-S4 to obtain color feature vectors and size information of the left and right pupils at the second time node, the third time node, and other specified time nodes; S6: calculating the color and shape change information of the patient's pupil over time according to the color feature vector of each time node, and drawing the size change curves of the patient's left and right pupils over time according to the size information of each time node; S7: retrieve the corresponding standard pupil color data, shape data and size data according to the patient's basic information, compare them with the pupil color data, shape data and size data at a specified time point, assist the doctor in obtaining the patient's condition at the specified time point, and assist in obtaining the patient's condition change trend according to the patient's color and shape change information and size change curve.

2. The disease diagnosis auxiliary method based on pupil image processing according to claim 1, characterized in that: In step S1, the pupil position is preliminarily identified during the image acquisition process, and the image is acquired after focusing based on the pupil position. The specific process of preprocessing the acquired image is as follows: S11: Preset pupil color, shape and size range data in the image acquisition module. When the image acquisition module is performing image acquisition, it automatically obtains an area that meets the preset color, shape and size range, and focuses based on the area. S12: Invalid images, images with resolutions lower than a preset threshold, and duplicate images are removed from the collected images, and the brightness values ​​of the R, G, and B channels of each pixel in the image are adjusted. The specific adjustment formula is: R=G=B=(Wr*R+Wg*G+Wb*B) / 3; Among them, Wr refers to the preset weight value for the R channel, Wg refers to the preset weight value for the G channel, and Wb refers to the preset weight value for the B channel; S13: The image processed in step S12 is subjected to a specified transformation, and then the spectral components of the spectral components of the image processed by the specified transformation with sudden changes in pixel grayscale are filtered out through a preset filter, and then the final image is obtained by inverse processing of the specified transformation.

3. The disease diagnosis auxiliary method based on pupil image processing according to claim 1, characterized in that: In step S2, the preprocessed image is subjected to image recognition again, and the specific process of obtaining the pupil area from the preprocessed image is as follows: S21: creating an image recognition model, inputting a specified number of labeled pupil image data into the image recognition model, and training the image recognition model; S22: inputting the preprocessed image into the image recognition model, the image recognition model divides the input image into image blocks of uniform specified size, and then detects the target whose center falls within the image block; S23: the image recognition model predicts the bounding box in each image block in turn, the predicted bounding box includes information such as the center coordinates of the bounding box, the height and width of the bounding box, and the confidence value of the object contained in the bounding box, and the image recognition model predicts a specified category probability in each image block; S24: Preset a confidence threshold, compare the confidence value of each bounding box predicted by the image recognition model with the confidence threshold, and extract the bounding boxes whose confidence values ​​are greater than the confidence threshold; S25: removing overlapping bounding boxes from each category, and retaining the bounding boxes with the highest confidence values ​​and whose corresponding predicted bounding boxes and standard bounding boxes have a ratio of intersection to union that is lower than a preset value; S26: Merge the prediction results in all image blocks to obtain the final bounding box and category label.

4. The disease diagnosis auxiliary method based on pupil image processing according to claim 3 is characterized in that: The image recognition model includes an input layer, a combination layer of convolution layer + pooling layer*2, and a fully connected layer, wherein the first convolution layer in the combination layer is set with 3 2x2 convolution kernels, the second convolution layer is set with 5 1x1 convolution kernels, the first pooling layer downsamples the feature map obtained by the first convolution layer, and the second pooling layer downsamples the feature map obtained by the second convolution layer.

5. The disease diagnosis auxiliary method based on pupil image processing according to claim 1, characterized in that: The specific process of extracting pupil edge features from the cut pupil position area in step S3 is as follows: S31: Calculate the gradient strength and direction of each pixel in the pupil position area of ​​the image, retain the pixel with the local maximum value in the gradient direction, and remove the pixel with the gradient direction lower than the local maximum value; S32: Preset two gradient thresholds, including a large gradient threshold and a small gradient threshold, map the gradient amplitude to the two preset gradient thresholds, mark the pixel points whose value is greater than the large gradient threshold as strong edges, and mark the pixel points whose value is between the large gradient threshold and the small gradient threshold as non-edges, so as to obtain the edge feature of the pupil; S33: Connect adjacent strong edge pixels in sequence to form a final pupil edge.

6. The disease diagnosis auxiliary method based on pupil image processing according to claim 1, characterized in that: In step S4, the specific process of extracting the color features and shape features of the pupil from the final pupil image and establishing the color feature vectors of the left and right pupils at the first time node is as follows: S41: extracting the value of the R channel of the pixel point in the pupil image, and establishing an R channel matrix according to the pixel point coordinates; S42: extracting the values ​​of the G channel of the pixel points in the pupil image, and establishing a G channel matrix according to the coordinates of the pixel points; S43: extracting the value of the B channel of the pixel point in the pupil image, and establishing a B channel matrix according to the pixel point coordinates; S44: Construct a color feature vector of the pupil image from the R channel matrix, the G channel matrix, and the B channel matrix.

7. The disease diagnosis auxiliary method based on pupil image processing according to claim 1, characterized in that: The specific process of measuring the pupil size in step S4 and obtaining the size information of the left and right pupils at the first time node is as follows: S45: randomly selecting an edge pixel point of the pupil as a starting point, obtaining a pixel point among the edge pixels that is centrally symmetric to the starting point, and obtaining a distance between the pixel point and the starting point; S46: traverse the distances between all pixel points between the starting point and the centrally symmetric pixel point and the corresponding centrally symmetric pixel point in a clockwise or counterclockwise direction; S47: Take the average value of all the distances as the pupil diameter, which is the size of the pupil.

8. A disease diagnosis assistance system based on pupil image processing, used to implement a disease diagnosis assistance method based on pupil image processing according to any one of claims 1 to 7, characterized in that: It includes an image acquisition module, a preprocessing module, an image recognition module, an image calibration module, an image cutting module, a feature extraction module, a dimension measurement module, and a curve drawing module, wherein the image acquisition module is connected to the preprocessing module, the preprocessing module is connected to the image recognition module, the image recognition module is connected to the image calibration module, the image calibration module is connected to the image cutting module, the image cutting module is connected to the feature extraction module, the feature extraction module is connected to the dimension measurement module, and the dimension measurement module is connected to the curve drawing module; The image acquisition module is used to acquire images of the left and right eyes with completely exposed pupils at a position at a specified distance from the patient's eyes at a first time node, preliminarily identify the pupil position during the image acquisition process, and acquire the image after focusing based on the pupil position; The preprocessing module is used to preprocess the collected images; The image recognition module is used to perform image recognition again on the preprocessed image and obtain the pupil area from the preprocessed image; The image calibration module is used to enlarge the acquired pupil area by a specified ratio with the center point as the center of the circle, and calibrate the pupil position area enlarged by the specified ratio; The image cutting module is used to perform image cutting based on the calibrated pupil position area, and to perform image cutting again based on the extracted pupil edge features to obtain a final pupil image; The feature extraction module is used to extract pupil edge features from the cut pupil position area, and to extract pupil color features and shape features from the final pupil image, and to establish color feature vectors and shape feature vectors of the left and right pupils at the first time node; The size measurement module is used to measure the pupil size and obtain the size information of the left and right pupils at the first time node; The curve drawing module is used to calculate the color and shape change information of the patient's pupil over time according to the color feature vector of each time node, and to draw the size change curves of the patient's left and right pupils over time according to the size information of each time node.

Images