Premature infant fundus auxiliary screening management method and system

Abstract

The invention belongs to the technical field of auxiliary screening of fundus of premature infants, and particularly relates to an auxiliary screening management method and system for fundus of premature infants, and the method comprises the steps: obtaining various data of existing premature infants, training a prediction model, predicting fundus abnormality of to-be-screened premature infants, and screening the premature infants needing fundus screening from a prediction result. The eye fundus images of the premature infants to be screened are collected for image analysis, whether the eye fundus abnormal recognition result exists or not is judged and output, the existing premature infant eye fundus screening process is improved in combination with judgment of a doctor, premature infants needing eye fundus screening are screened out in advance, and the screening efficiency is improved. According to the method, all premature infants do not need to be subjected to eye processing and then eye fundus images are obtained, due to the fact that the premature infants are painful in the eye processing and eye fundus photographing process, certain eye fundus screening is carried out by means of image processing and analysis assistance to judge eye fundus abnormity, and the efficiency and accuracy of eye fundus screening are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of preterm infant fundus auxiliary screening, and particularly relates to a preterm infant fundus auxiliary screening management method and system. Background Art

[0002] Preterm infants are babies born before 37 weeks of gestation. Medically, preterm infants are further divided into extremely preterm (gestational age less than 28 weeks), very preterm (gestational age between 28 and 32 weeks), and preterm (gestational age between 32 and 37 weeks).

[0003] Fundus screening for preterm infants is a very important examination. For preterm infants, their gestational period is shorter, and the probability of suffering from retinopathy of prematurity is relatively high. If severe retinopathy of prematurity is not treated in time, it will lead to the serious consequence of blindness. Most cases can preserve certain useful vision if treated in time at an early stage. The smaller the gestational age and the lighter the birth weight, the more likely it is to develop retinopathy of prematurity. The purpose of fundus screening is to detect retinopathy of prematurity in time and give corresponding treatment in time.

[0004] Currently in clinical practice, for preterm infants, doctors almost always recommend that the parents of preterm infants undergo fundus screening in order to screen out preterm infants with retinopathy or detect congenital abnormalities and other lesions in the fundus in time, so as to be able to give corresponding treatment in time. However, in clinical practice, not all preterm infants have fundus abnormalities. A large number of preterm infants have no fundus abnormalities, but in order to rule out fundus abnormalities, all preterm infants will choose to undergo fundus screening. Conducting fundus screening for all preterm infants will, on the one hand, increase the economic cost of the parents of preterm infants, and on the other hand, increase the pain of preterm infants during fundus screening. And during the fundus screening process for preterm infants, doctors use an ophthalmoscope to analyze and judge whether there are fundus abnormalities in the fundus of preterm infants. This method relies on the experience of doctors, and both the efficiency and accuracy are relatively low.

[0005] Therefore, how to improve the existing preterm infant fundus screening process, screen out preterm infants who need to undergo fundus screening in advance, and assist in judging fundus abnormalities by means of image processing and analysis, so as to improve the efficiency and accuracy of fundus screening, is a technical problem that urgently needs to be solved at present. Summary of the Invention

[0006] The purpose of the present invention is to provide a preterm infant fundus auxiliary screening management method and system, which are used to improve the existing preterm infant fundus screening process, screen out preterm infants who need to undergo fundus screening in advance, and assist in judging fundus abnormalities by means of image processing and analysis, so as to improve the efficiency and accuracy of fundus screening.

[0007] To solve the above technical problems, the technical solution adopted by the present invention is as follows: In the first aspect, a method for assisting in the fundus screening and management of premature infants is provided, including the following steps: S1: Obtain the basic data and fundus screening data of premature infants based on big data, and classify them to obtain text data and image data; S2: Preprocess the text data and image data respectively, and extract text features and image features from the preprocessed text data and image data respectively; S3: Create a fundus abnormality prediction model, and input the extracted text features and image features into the fundus abnormality prediction model to train, test, and verify the model; S4: Obtain the basic data of the premature infants to be screened and extract text features and image features. Input the extracted text features and image features into the fundus abnormality prediction model. The fundus abnormality prediction model predicts the fundus abnormalities of the premature infants to be screened. The doctor determines the premature infants who need to undergo fundus screening based on the prediction results and executes step S5; S5: Collect and preprocess the fundus images of the premature infants to be screened, input them into the fundus abnormality prediction model for fundus abnormality recognition. The doctor makes a fundus abnormality judgment based on the processed fundus images and the abnormality recognition results, and gives the fundus abnormality results; S6: Input the fundus abnormality results and their corresponding basic data into the fundus abnormality prediction model to continuously optimize and update the model.

[0008] Preferably, the basic data includes gestational age, gender, height and weight at birth, and various examination data during pregnancy.

[0009] Preferably, the specific process of classifying the basic data and fundus screening data into text data and image data in step S1 is as follows: S11: Create a data type recognition model, input the labeled text data and image data into the data type recognition model for model training, evaluate the performance of the data type recognition model. If the preset performance indicators are met, execute step S12, otherwise re-train the model; S12: Input the basic data and fundus screening data of premature infants into the trained data type recognition model, and the data type recognition model performs data type recognition on the input data; S13: The data type recognition model adds corresponding data type labels to the output data according to the data type recognition results.

[0010] Preferably, the specific process of preprocessing the text data in step S2 is as follows: S21: Extract the characters in the text data to form a text string, and split the text string to obtain phrases. S22: Based on the phrases, construct a phrase list, establish a mapping relationship between the phrases and numbers based on the phrases and their frequencies in the phrase list, and then convert the phrase list into a digital index sequence.

[0011] Preferably, the specific process of preprocessing the image data in step S2 is as follows: S23: Create a filter kernel and determine its size and standard deviation, generate a two-dimensional matrix according to the size and standard deviation of the filter kernel, and the elements in the two-dimensional matrix are generated by a specified two-dimensional function, and the two-dimensional function is expressed as follows: ; where ([[]] x , y ) are the coordinates of the pixel points of the filter kernel, ([[]] x 0 , y 0 ) are the coordinates of the central pixel point in the filter kernel, σ is the standard deviation; S24: Perform normalization processing on the two-dimensional matrix so that the sum of all matrix elements in the two-dimensional matrix is 1; S25: Perform kernel convolution operation on the image to be preprocessed and the normalized two-dimensional matrix. For each pixel in the image to be preprocessed, obtain its neighborhood with the same size as the two-dimensional matrix, multiply the pixel values in the neighborhood by the elements at the corresponding positions of the Gaussian kernel, and add the products to obtain the result as the new pixel value of the pixel.

[0012] Preferably, step S5 further includes pain management for premature infants during fundus screening, and the specific process is as follows: S51: Real-time collect the specified physiological indicators and behavioral data of premature infants undergoing fundus screening. The physiological indicators include heart rate, blood pressure, and blood oxygen saturation, and the behavioral data includes facial expressions, crying frequency and duration, and limb movements; S52: Based on the physiological indicators and behavioral data, and using a preset pain scoring strategy, perform pain scoring on premature infants undergoing fundus screening, and adjust the fundus screening process based on the pain scoring results.

[0013] Second aspect, a fundus-assisted screening management system for premature infants is provided to implement the described fundus-assisted screening management method for premature infants, including a data acquisition module, a data classification module, a preprocessing module, a feature extraction module, a fundus abnormality prediction model, and a fundus image acquisition module. The data acquisition module is connected to the data classification module, the data classification module is connected to the preprocessing module, the preprocessing module is connected to the feature extraction module, the feature extraction module is connected to the fundus abnormality prediction model, and the fundus abnormality prediction model is connected to the fundus image acquisition module; The data acquisition module is used to obtain the basic data and fundus screening data of premature infants based on big data; The data classification module is used to classify the basic data and fundus screening data to obtain text data and image data; The preprocessing module is used to preprocess the text data and image data respectively; The feature extraction module is used to extract text features and image features from the preprocessed text data and image data respectively; The fundus abnormality prediction model is used to predict the fundus abnormalities of premature infants to be screened; The fundus image acquisition module is used to collect the fundus images of premature infants to be screened.

[0014] Preferably, the fundus abnormality prediction model in step S3 is a neural network model including an input layer, a processing layer, and an output layer. The input layer has a dual-input channel, and the processing layer includes an input integration layer, a feature extraction layer, a prediction result generation layer, and a feedback link layer; The input integration layer is used to preliminarily integrate the data received from the two input channels; The feature extraction layer is used to extract different features of the integrated data using different neurons respectively; The prediction result generation layer is used to generate the prediction result of the fundus abnormalities of premature infants; The feedback connection layer is used to dynamically adjust the data processing process of the processing layer based on the input and output.

[0015] Preferably, there are several neurons in the processing layer. Each neuron has a specified activation function and learning strategy. The neuron adjusts the learning strategy of the neuron by maximizing the long-term reward and updates the neuron parameters using the stochastic gradient descent algorithm to achieve end-to-end control optimization.

[0016] The beneficial effects of the present invention include: The preterm infant fundus assisted screening management method and system provided by the present invention predict the fundus abnormalities of preterm infants to be screened by training a prediction model through obtaining various data of existing preterm infants, screen out the preterm infants who need fundus screening from the prediction results, analyze whether there are fundus abnormality recognition results by collecting the fundus images of the preterm infants to be screened and output them, and then combine with the judgment of doctors to improve the existing preterm infant fundus screening process, screen out in advance the preterm infants who need fundus screening, so that it is no longer necessary for all preterm infants to undergo eye treatment and then obtain fundus images, because it is very painful for preterm infants during the eye treatment and fundus photography process, and there is a certain amount of fundus screening to assist in judging fundus abnormalities with the help of image processing and analysis, improving the efficiency and accuracy of fundus screening.

[0017] First, train the model by obtaining the basic data and fundus screening data of preterm infants based on big data, so that the fundus abnormality prediction model can learn the relationship between the basic data of preterm infants and fundus abnormalities, and then predict the fundus abnormalities of preterm infants to be screened through the fundus abnormality prediction model, so that it is not necessary to perform fundus screening on all preterm infants, saving medical resources, reducing the economic costs of preterm infants' parents, and at the same time avoiding the pain of preterm infants during fundus screening.

[0018] Second, identify fundus abnormalities in the captured fundus images through the model, improve the efficiency of fundus screening, and combine with doctors' diagnoses to avoid mis-screening and improve the accuracy of preterm infant fundus screening.

[0019] Third, the fundus abnormality prediction model is a neural network model including an input layer, a processing layer and an output layer. The input layer is provided with dual input channels. For the dual input channels of the fundus abnormality prediction model, the processing layer is provided with an input integration layer, a feature extraction layer, a prediction result generation layer and a feedback link layer, and can be optimized and updated through the recognition of each fundus image, so that the model has good generalization and high accuracy, providing for preterm infant fundus screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flow chart of the preterm infant fundus assisted screening management method of the present invention.

[0021] Figure 2 It is a schematic architecture diagram of the fundus abnormality prediction model of the present invention.

[0022] Figure 3 It is a schematic flow chart of the fundus screening pain management of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The following will be combined with the attached Figures 1 to 3 to further elaborate on the present invention in detail: Example 1 See the appendix Figure 1 As shown, a method for assisting in the fundus screening and management of premature infants includes the following steps: S1: Based on big data, obtain the basic data and fundus screening data of premature infants, and classify them to obtain text data and image data. The basic data includes gestational age, gender, height and weight at birth, and various examination data during pregnancy. Since there are certain relationships among the various developments of premature infants, and for the various developments of the body, the present invention obtains various data of a large number of premature infants who have undergone fundus screening, including basic data and fundus screening data, and captures the relationship between the basic data and the fundus screening data by creating a neural network model, so as to predict fundus abnormalities for premature infants to be screened subsequently, and judge whether fundus screening is required according to the prediction results, so as to avoid the physical pain brought by fundus screening to premature infants who do not need to undergo fundus screening.

[0024] S2: Preprocess the text data and image data respectively, and extract text features and image features from the preprocessed text data and image data respectively. The data preprocessing process is first based on the previous data classification, and preprocessing and feature extraction are respectively carried out based on the classification results to improve the preprocessing efficiency and feature extraction efficiency.

[0025] S3: Create a fundus abnormality prediction model, and input the extracted text features and image features into the fundus abnormality prediction model to train, test and verify the model. By extracting the text features and image features of various data of a large number of premature infants who have undergone fundus screening in advance to train the fundus abnormality prediction model, the fundus abnormality prediction model can directly capture the relationship between the text features, image features and fundus abnormalities, and improve the model training efficiency.

[0026] S4: Obtain the basic data of premature infants to be screened and extract text features and image features. Input the extracted text features and image features into the fundus abnormality prediction model. The fundus abnormality prediction model predicts the fundus abnormalities of premature infants to be screened. Doctors determine the premature infants who need to undergo fundus screening according to the prediction results and execute step S5. When doctors determine whether premature infants to be screened need to undergo fundus screening, they first obtain the prediction results of the fundus abnormality prediction model for the fundus abnormalities of premature infants to be screened, and then finally judge whether fundus screening is required in combination with the various data of premature infants. For premature infants in the critical condition or close to the critical condition between those who need to be screened and those who do not need to be screened, it is determined that they need to undergo fundus screening, so as to avoid misjudging the situation where there are fundus abnormalities but no fundus screening is carried out, and ensure that the premature infants determined not to need fundus screening really do not need to be screened.

[0027] S5: Collect and preprocess the fundus images of preterm infants to be screened, input them into the fundus abnormality prediction model for fundus abnormality recognition. Doctors make fundus abnormality judgments based on the processed fundus images and the abnormality recognition results, and give the fundus abnormality results.

[0028] S6: Input the fundus abnormality results and their corresponding basic data into the fundus abnormality prediction model to continuously optimize and update the model.

[0029] Retinopathy of prematurity is a disease of impaired retinal vascular development in preterm infants. The incidence rate of preterm infants is relatively high. Fundus screening at a specified time after birth is one of the operations for preventable childhood blindness globally. Therefore, in current medical practice, almost all preterm infants, regardless of how many weeks prematurely they were born, are screened to avoid missing the opportunity to detect fundus abnormalities through fundus screening in a timely manner and receive timely treatment. However, in fact, among them, a certain proportion of preterm infants, especially those with a gestational age between 32 and 37 weeks, have a relatively large proportion of normal fundus development. For these preterm infants with normal fundus development, if they also need to undergo fundus screening, on the one hand, it will waste medical resources, on the other hand, it will increase the economic cost of parents. Most importantly, these preterm infants undergoing fundus screening will endure the pain during the screening process. Because during fundus screening, eye drops for dilating the pupils need to be dropped into the eyes of preterm infants, then the eyelids are pried open, and then fundus photography is performed or fundus observation is carried out through an ophthalmoscope to determine whether there are abnormalities in the fundus.

[0030] By training the prediction model with the existing data of preterm infants and then predicting the fundus abnormalities of preterm infants to be screened, screening out the preterm infants who need to undergo fundus screening from the prediction results, analyzing whether there are fundus abnormality recognition results by collecting the fundus images of preterm infants to be screened and outputting them, and then combining the doctor's judgment, the existing fundus screening process for preterm infants is improved, screening out in advance the preterm infants who need to undergo fundus screening, so that it is no longer necessary for all preterm infants to undergo eye treatment and then obtain fundus images, because the eye treatment and fundus photography processes are very painful for preterm infants, and there is also a certain degree of assistance in judging fundus abnormalities through image processing and analysis during fundus screening, improving the efficiency and accuracy of fundus screening.

[0031] Embodiment 2 Based on Embodiment 1, the specific process of classifying the basic data and fundus screening data into text data and image data in step S1 is as follows: S11: Create a data type recognition model. Input the labeled text data and image data into the data type recognition model for model training, and evaluate the performance of the data type recognition model. If the preset performance metrics are met, execute step S12; otherwise, retrain the model. The data type recognition model is a neural network model structure. By using the labeled data for model training, the model is enabled to have the ability to quickly recognize data types and high accuracy.

[0032] S12: Input the basic data of the premature infant and the fundus screening data into the trained data type recognition model, and the data type recognition model performs data type recognition on the input data. S13: According to the data type recognition result, the data type recognition model adds the corresponding data type label to the output data. By adding the data type labels to different data types, when the data is subsequently input into the fundus abnormality prediction model, it will be automatically input into the corresponding input channels according to the data type labels. The text data is input into the text data channel of the fundus abnormality prediction model, and the image data is input into the image data channel of the fundus abnormality prediction model.

[0033] The data type recognition model is a dual-branch hybrid neural network structure. The dual branches include a text branch and an image branch. The text branch is based on a variant of the BERT pre-trained model and is used to process the text data in the electronic medical record, including data such as gestational age, birth weight, and oxygen inhalation duration. The image branch is a lightweight MobileNetV3 network that processes fundus images, including JPEG / DICOM format files collected by RetCam. The data type recognition model also sets a fusion layer: align the text and image feature vectors through the multi-head self-attention mechanism and output a unified data type discrimination result.

[0034] During the training process of the data type recognition model, first, prepare the training data. The training data uses a labeled data set, including text data with more than 10,000 labeled samples, covering structured / unstructured texts such as laboratory reports and doctor's order records. More than 8,000 labeled fundus image data, including samples of normal fundus, various stages of ROP lesions, and artifact interference. Perform enhancement processing on the image data, apply random rotation (±15°), brightness adjustment (±20%), and Gaussian noise injection (σ≤0.05) to the image data to improve the robustness of the model.

[0035] Optimize and validate the trained data type recognition model. Set the optimization goal to minimize the cross-entropy loss function, use the Adam optimizer, with an initial learning rate of 1e-4 and a batch size of 32. When the accuracy of the validation set in performance evaluation is ≥98% and the F1 score is ≥0.97, the model is considered qualified. If not qualified, adopt the dynamic curriculum learning strategy: give priority to training easily classifiable samples, including clear fundus images, and gradually add complex cases, including hemorrhage / exudation mixed lesions.

[0036] Perform data classification. Preprocess the input data, divide the text data into unstructured text and numerical data. For unstructured text, including doctors' handwritten notes, perform OCR recognition and keyword extraction, and use regular expressions to match "GA<28 weeks". For numerical data, including data such as birth weight, perform normalization processing and map it to the interval [0,1]. Uniformly scale the image data to 512×512 pixels, and apply histogram equalization to eliminate imaging differences between devices. Infer and classify the data type recognition model. Input the preprocessed data into the trained data type recognition model: the text data generates a 128-dimensional feature vector through the text branch, and the image data generates a 256-dimensional feature vector through the image branch. Calculate the cosine similarity of the two types of data through the fusion layer. If the similarity >0.9, it is determined as the same type of data, otherwise, trigger an abnormal alarm, such as mistransmitting text as an image.

[0037] Perform labeling and channel allocation for each item of data. Add text data labels to the text data, add text_meta, metadata, such as "birth weight_1500g" or text_desc, descriptive text, such as "oxygen inhalation history_positive".

[0038] Add img_raw (original image) or img_processed (preprocessed image) to the image data label, and attach the resolution and acquisition device identifier.

[0039] Connect multi-channel inputs, construct a dynamic routing module. The text data is encapsulated in JSON format and input into the text embedding layer of the fundus abnormality prediction model. The image data is converted into HDF5 format and input into the convolutional feature extraction layer of the model. If unlabeled data types such as video streams are detected, automatically trigger data isolation and notify the administrator.

[0040] In actual hospital tests, the text / image classification error rate has dropped from 5.3% of the traditional rule engine to 0.8%. The processing time for a single case of data has been shortened from 2 minutes of manual operation to 0.5 seconds. It supports concurrent processing of more than 200 cases per second, and is compatible with more than 15 medical data formats through the dynamic routing mechanism, meeting the needs of multi-center collaboration.

[0041] In this embodiment, the specific process of preprocessing the text data in step S2 is as follows: S21: Uniformly convert the text data into a specified encoding format, remove the noise data therein, and sequentially connect the remaining characters in the original order to form a continuous text string, which is processed by a specified string operation function. Perform normalization processing on the text string, convert all letters to uppercase or lowercase, unify the representation form of numbers, etc., to reduce the diversity of the text and facilitate subsequent model analysis and processing. Check the generated text string for problems such as garbled characters and typos, and make corresponding corrections. Finally, split the text string to obtain word groups; S22: Based on the word groups, construct a word group list, establish a mapping relationship between the word groups and numbers based on the word groups and their frequencies in the word group list, and then convert the word group list into a digital index sequence. For high-frequency words, assign fixed indexes, such as 1-10. For medium- and low-frequency words, assign indexes in descending order of word frequency, starting from 11. For out-of-vocabulary words, that is, newly emerging words, uniformly mark them as 0 to establish a special index. Convert the cleaned word groups into a digital sequence according to the mapping table, and perform digital sequence normalization, truncating / filling to a fixed length.

[0042] The specific process of preprocessing the image data in step S2 is as follows: S23: Create a filter kernel and determine its size and standard deviation. Generate a two-dimensional matrix according to the size and standard deviation of the filter kernel. The elements in the two-dimensional matrix are generated by a specified two-dimensional function, and the two-dimensional function is expressed as follows: ; where ([[]] x , y ) are the coordinates of the pixel points of the filter kernel, G([[]] x , y ) is the two-dimensional function value of the pixel point ([[]] x , y ) in the two-dimensional matrix, ([[]] x 0 , y 0 ) are the coordinates of the central pixel point in the filter kernel, σ is the standard deviation, and the standard deviation is dynamically adjusted according to the noise level of the fundus image. By default σ is 1.5 and increases to 2.5 when the noise is severe; The size of the filter kernel is determined by k size = 2* int (3σ) + 1. For example σ =1.5 generates a 7×7 kernel, that is, 3 σ=4.5 → take the integer part as 4 → kernel width = 9 → corrected to 7 to ensure its symmetry.

[0043] S24: Normalize the two-dimensional matrix so that the sum of all matrix elements in the two-dimensional matrix is 1; Calculate the sum of all elements of the current kernel matrix. For example, the initial sum of a 5×5 kernel is 0.4787, and normalize each element: ; Among them, is the value after normalization of the two-dimensional function value of the pixel point in the two-dimensional matrix. After normalization, the sum of the kernel matrix elements is strictly equal to 1, ensuring that the overall brightness of the image remains unchanged after convolution.

[0044] S25: Perform kernel convolution operation on the image to be preprocessed through the normalized two-dimensional matrix, and traverse each pixel point in the image in the specified order of pixel points. For each pixel in the image to be preprocessed, obtain its neighborhood with the same size as the two-dimensional matrix, multiply the pixel values in the neighborhood by the elements at the corresponding positions of the Gaussian kernel, and add the products. The result obtained is used as the new pixel value of this pixel. When performing kernel convolution operation, add one or more layers of pixel points with pixel value 0 outside the image edge to perform a complete kernel convolution operation on all pixel points.

[0045] Edge zero-padding strategy. For the input image, assuming the size is H×W, add pad width = k size / 2 layers of zero pixels. For example, a 7×7 kernel needs to be padded with 3 layers, and the size of the padded image is: (H + 6)×(W + 6).

[0046] Sliding window convolution. Traverse each pixel point in the image in raster scan order ( i , j ), extract the i , j ) centered k size × k size neighborhood window, and calculate the weighted sum: ; Among them, is the convolution weighted sum of the pixel points in the neighborhood window, is the value after normalization of the two-dimensional function value of the pixel point in the two-dimensional matrix, is the convolution value of the pixel point .

[0047] The matrix multiply-accumulate operation is accelerated using SIMD instructions to optimize the image, and the single-channel processing speed for 512×512 pixel grayscale images reaches 120 frames per second. In the test set of premature infant fundus images, the signal-to-noise ratio is increased from 18.7 dB of the original image to 24.3 dB, and the clarity of blood vessel boundaries is improved by 35%. Compared with traditional mean filtering, in the 7×7 kernel image, the computing processing speed is increased by 2.3 times. Support σ value real-time adjustment (0.5 < σ < 3.0) to adapt to the noise characteristics of different imaging devices.

[0048] Example 3 Based on Example 1 or Example 2, refer to Figure 2 , the fundus abnormality prediction model in step S3 is a neural network model, which is an improved DD-DC neuron network model, including an input layer, a processing layer and an output layer. The input layer is provided with dual input channels. For the dual input channels of the fundus abnormality prediction model, one of the channels is used to input text data or text features, and the input format of the text data is: 128-dimensional structured text vector and 768-dimensional unstructured text embedding vector.

[0049] Another input channel is used to input image data or image features, and the input format is: a normalized fundus image of 512×512 pixels.

[0050] The processing layer includes an input integration layer, a feature extraction layer, a prediction result generation layer and a feedback link layer. The input integration layer is used to preliminarily integrate the data received from the two input channels. The feature extraction layer is used to use different neurons to respectively extract different features of the integrated data. The prediction result generation layer is used to generate the prediction result of premature infant fundus abnormalities. The feedback connection layer is used to dynamically adjust the data processing process of the processing layer based on the input and output.

[0051] The processing layer is also provided with two processing units connected through an information interaction layer. The two processing units are respectively connected to the two input channels and are respectively used to process text data or image data. In the process of processing text data and image data by the processing unit for processing text data and the processing unit for processing image data respectively, information interaction will occur, that is, different processing units are not completely independent in the process of processing text data and images, and in fact, there will also be a process of information interaction.

[0052] The core components of the processing layer include the input integration layer, which uses the cross-attention mechanism to fuse bimodal features, and the specific formula is as follows: ; Among them, Q textis the query vector generated by the linear transformation of the text feature, K image and V image are the key-value pairs generated for the image feature, d k is the dimension size of the key vector.

[0053] The processing layer sets a dual-processing unit interaction mechanism. The text processing unit includes a 3-layer BiLSTM network with 512 hidden units in each layer, and the image processing unit includes 4 residual convolutional blocks with the number of channels in the residual convolutional blocks being 64→256.

[0054] Set a phased training protocol. Phase 1 is the first 50 rounds, freezing the image channels and only training the text-related modules. Phase 2 is rounds 51 - 150, performing dual-channel joint training and reducing the learning rate to 1e -5 . Phase 3 is after round 151, enabling the feedback connection layer and adding gradient clipping.

[0055] Each has a number of neurons. Each neuron has a specified activation function and learning strategy. Each neuron will generate feedback control according to the input data and its own characteristics as a feedback controller. Each neuron may have an independent control strategy to map the input to the output, and there are feedback connections between neurons. Neurons in the later processing chain can feedback and affect neurons in the earlier processing chain.

[0056] In another implementation manner of this embodiment, the neurons adjust the learning strategy of the neurons by maximizing the long-term reward and update the neuron parameters using the stochastic gradient descent algorithm to achieve end-to-end control optimization. First, initialize the parameters in the neurons, that is, set an initial value for each parameter of the neurons, and the initial value is randomly determined within a specified range. Propagate the input data through the neurons. The input data of each layer is multiplied by the weights of this layer in matrix multiplication and then add a bias, and obtain the output of this layer through a preset activation function. The output of this layer is used as the input of its next layer until the final output is obtained. After obtaining the output through the model, calculate the loss function value according to the model output and the true label to measure the difference between the model output and the true result. Then, starting from the output layer, calculate the gradient of each layer's parameters according to the loss function. Use the chain rule to calculate the partial derivative of the loss function with respect to each parameter, and the partial derivatives form the gradient. Through backpropagation, the gradient information gradually propagates from the output layer to the input layer, and finally the gradient of each parameter is obtained. Based on the gradient of each parameter, set a specified function to update the parameters and achieve the optimization of the model.

[0057] Step S5 also includes pain management for preterm infant retinopathy screening. See Figure 3 and the specific process is as follows: S51: Real-time collect the specified physiological indicators and behavioral data of premature infants undergoing fundus screening. The physiological indicators include heart rate, blood pressure, and blood oxygen saturation, and the behavioral data includes facial expressions, crying frequency and duration, and limb movements. During the fundus screening process, premature infants need to endure a great deal of pain because the various operations of fundus screening will bring pain and stress to premature infants. Since premature infants are of a young age and unable to express themselves verbally, relevant data needs to be collected during the fundus screening process for premature infants to facilitate subsequent pain assessment of premature infants through the collected data.

[0058] S52: Based on the physiological indicators and behavioral data, use a preset pain scoring strategy to score the pain of premature infants undergoing fundus screening, and adjust the fundus screening process based on the pain scoring results. By performing pain assessment based on various data of premature infants during the fundus screening process, understand the pain information of premature infants, and make adjustments during the operation. When the pain score value reaches the threshold, relevant operations need to be paused. After the reaction of the premature infant has been relieved to a certain extent, corresponding operations can be carried out again.

[0059] A premature infant fundus-assisted screening management system for implementing the described premature infant fundus-assisted screening management method, including a data acquisition module, a data classification module, a preprocessing module, a feature extraction module, a fundus abnormality prediction model, and a fundus image acquisition module. The data acquisition module is connected to the data classification module, the data classification module is connected to the preprocessing module, the preprocessing module is connected to the feature extraction module, the feature extraction module is connected to the fundus abnormality prediction model, and the fundus abnormality prediction model is connected to the fundus image acquisition module. The data acquisition module is used to obtain the basic data and fundus screening data of premature infants based on big data. The data classification module is used to classify the basic data and fundus screening data to obtain text data and image data. The preprocessing module is used to preprocess the text data and image data respectively. The feature extraction module is used to extract text features and image features from the preprocessed text data and image data respectively. The fundus abnormality prediction model is used to predict the fundus abnormalities of premature infants to be screened. The fundus image acquisition module is used to acquire the fundus images of premature infants to be screened.

[0060] In summary, the method and system for assisted screening and management of premature infants' fundus provided by the present invention predict the fundus abnormalities of premature infants to be screened by training a prediction model through obtaining various data of existing premature infants, screen out the premature infants who need fundus screening from the prediction results, analyze whether there are fundus abnormality recognition results by collecting the fundus images of premature infants to be screened and output them, and then, in combination with the doctor's judgment, improve the existing process of premature infants' fundus screening, screen out in advance the premature infants who need fundus screening, so that it is no longer necessary for all premature infants to undergo eye treatment and then obtain fundus images, because the process of eye treatment and fundus photography is very painful for premature infants, and there is a certain degree of judgment of fundus abnormalities assisted by image processing and analysis in fundus screening, which improves the efficiency and accuracy of fundus screening.

Claims

1. A method for auxiliary screening and management of fundus of premature infants, characterized in that: The following steps are involved: S1: Obtain basic data of premature infants and fundus screening data based on big data, and classify them to obtain text data and image data; S2: preprocessing the text data and the image data respectively, and performing text feature extraction and image feature extraction on the preprocessed text data and the image data respectively; S3: Create a fundus abnormality prediction model, input the extracted text features and image features into the fundus abnormality prediction model to train, test and verify the model; S4: basic data of the premature infants to be screened are obtained and text features and image features are extracted, and the extracted text features and image features are input into the fundus abnormality prediction model. The fundus abnormality prediction model predicts the fundus abnormality of the premature infants to be screened. The doctor determines the premature infants who need to undergo fundus screening according to the prediction results, and executes step S5; S5: The fundus images of the premature infants to be screened are collected and preprocessed, and input into the fundus abnormality prediction model to identify fundus abnormalities. The doctor makes a fundus abnormality judgment based on the processed fundus images and abnormality identification results, and gives the fundus abnormality results; S6: Inputting the fundus abnormality results and their corresponding basic data into the fundus abnormality prediction model, and continuously optimizing and updating the model.

2. A method for auxiliary fundus screening management of premature infants according to claim 1, characterized in that: The basic data include gestational period, gender, height and weight at birth, and various examination data during pregnancy.

3. A method for auxiliary fundus screening management of premature infants according to claim 1, characterized in that: The specific process of classifying the basic data and fundus screening data to obtain text data and image data in step S1 is as follows: S11: Create a data type recognition model, input the labeled text data and image data into the data type recognition model for model training, evaluate the performance of the data type recognition model, and if the preset performance index is met, execute step S12, otherwise re-train the model; S12: inputting the basic data of the premature infant and the fundus screening data into the trained data type recognition model, wherein the data type recognition model performs data type recognition on the input data; S13: The data type recognition model adds corresponding data type labels to the output data according to the data type recognition results.

4. A method for auxiliary fundus screening management of premature infants according to claim 3, characterized in that: The specific process of preprocessing the text data in step S2 is as follows: S21: extracting characters from the text data to form a text string, and splitting the text string into phrases; S22: constructing a phrase list based on the phrases, establishing a mapping relationship between phrases and numbers based on the phrases and their frequencies in the phrase list, and then converting the phrase list into a digital index sequence.

5. A method for auxiliary fundus screening management of premature infants according to claim 3, characterized in that: The specific process of preprocessing the image data in step S2 is as follows: S23: creating a filter kernel and determining its size and standard deviation, generating a two-dimensional matrix according to the size and standard deviation of the filter kernel, wherein the elements in the two-dimensional matrix are generated by a specified two-dimensional function; S24: performing normalization processing on the two-dimensional matrix so that the sum of all matrix elements in the two-dimensional matrix is ​​1; S25: Perform a kernel convolution operation on the image to be preprocessed and the normalized two-dimensional matrix. For each pixel in the preprocessed image, obtain its neighborhood with the same size as the two-dimensional matrix, multiply the pixel value in the neighborhood with the element at the corresponding position of the Gaussian kernel, add the products, and use the result as the new pixel value of the pixel.

6. A premature infant fundus auxiliary screening management method according to claim 1, characterized in that: Step S5 also includes fundus screening pain management for premature infants, and the specific process is as follows: S51: Real-time collection of designated physiological indicators and behavioral data of premature infants screened by fundus screening, wherein the physiological indicators include heart rate, blood pressure, and blood oxygen saturation, and the behavioral data include facial expressions, crying frequency and duration, and body movements; S52: Based on the physiological indicators and behavioral data and using a preset pain scoring strategy, a pain score is performed on the premature infant undergoing fundus screening, and the fundus screening process is adjusted based on the pain scoring result.

7. A premature infant fundus auxiliary screening management system, used to implement a premature infant fundus auxiliary screening management method according to any one of claims 1 to 6, characterized in that: It includes a data acquisition module, a data classification module, a preprocessing module, a feature extraction module, a fundus abnormality prediction model and a fundus image acquisition module, wherein the data acquisition module is connected to the data classification module, the data classification module is connected to the preprocessing module, the preprocessing module is connected to the feature extraction module, the feature extraction module is connected to the fundus abnormality prediction model, and the fundus abnormality prediction model is connected to the fundus image acquisition module; The data acquisition module is used to acquire basic data and fundus screening data of premature infants based on big data; The data classification module is used to classify the basic data and fundus screening data to obtain text data and image data; The preprocessing module is used to preprocess the text data and the image data respectively; The feature extraction module is used to extract text features and image features from the preprocessed text data and image data respectively; The fundus abnormality prediction model is used to predict fundus abnormalities in premature infants to be screened; The fundus image acquisition module is used to acquire fundus images of premature infants to be screened.

8. The premature infant fundus auxiliary screening management system according to claim 7, characterized in that: The fundus abnormality prediction model in step S3 is a neural network model including an input layer, a processing layer and an output layer, wherein the input layer is provided with a dual input channel, and the processing layer includes an input integration layer, a feature extraction layer, a prediction result generation layer and a feedback link layer; The input integration layer is used to perform preliminary integration on the input data received from the two input channels; The feature extraction layer is used to use different neurons to extract different features of the integrated data respectively; The prediction result generation layer is used to generate prediction results of fundus abnormalities in premature infants; The feedback connection layer is used to dynamically adjust the data processing process of the processing layer based on the input and output.

9. The premature infant fundus auxiliary screening management system according to claim 7, characterized in that: The processing layer is provided with a number of neurons, each of which is provided with a specified activation function and a learning strategy. The neurons adjust the learning strategy of the neurons by maximizing long-term rewards, and update the neuron parameters using a stochastic gradient descent algorithm.

Images