Eye data processing and transmitting method and system and ophthalmoscope

By introducing data processing transmission technology and preset segmentation model in the ophthalmoscope, the automatic segmentation and recognition of the fundus vision disc image is achieved, and the problem of insufficient efficiency and accuracy of the vision disc evaluation in traditional ophthalmological examinations is solved, and the overall efficiency of the ophthalmological examination is improved. and quality.

CN118887230APending Publication Date: 2024-11-01EYE & ENT HOSPITAL SHANGHAI MEDICAL SCHOOL FUDAN UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410880951.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-02
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

In traditional ophthalmic examinations, doctors rely on clinical experience to evaluate the disc. The efficiency and accuracy are insufficient, and it is difficult to effectively use the disc image segmentation technology to improve the efficiency and accuracy of the fundus disc examination.

Method used

By introducing data processing and transmission technology into the ophthalmoscope, the transmission relationship between the ophthalmoscope and the terminal equipment is established, the shooting request signal and control signal are received, and the fundus image is segmented using a preset segmentation model to realize automated and accurate visual disk area recognition and return the result to the terminal device.

Benefits of technology

It improves the efficiency and accuracy of doctors in completing visual disc evaluation, improves the overall efficiency and quality of ophthalmic examinations, and achieves more meticulous monitoring and diagnostic support for pathological myopia.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887230A_ABST
    Figure CN118887230A_ABST
Patent Text Reader

Abstract

The invention provides an eye data processing and transmitting method, which is applied to an ophthalmoscope and comprises the following steps: establishing a transmission relationship between the ophthalmoscope and terminal equipment; receiving a shooting request signal sent by the terminal equipment; receiving a shooting control signal; controlling an image sensor to acquire at least one fundus image according to the shooting request signal and the shooting control signal; inputting the eye fundus image into a processor, and calling a preset segmentation model by the processor to perform optic disc image segmentation on the eye fundus image to obtain an optic disc image segmentation result; in response to the shooting request signal, returning the optic disc image segmentation result and the eye fundus image to the terminal device; according to the technical scheme, optic disc image shooting can be carried out on the ophthalmoscope, optic disc segmentation is carried out through the preset segmentation model, the segmentation result is transmitted, operation is convenient, and optic disc segmentation efficiency is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of medical data processing and transmission technology, and in particular to an eye data processing and transmission technology. Background Art

[0002] Myopia, as a common eye disease, not only affects the individual's visual function, but may also induce a variety of eye health problems. In certain populations, myopia may evolve into a more serious pathological state, namely pathological myopia. In addition to significant high myopia, the characteristics of pathological myopia are more likely to cause fundus lesions, including retinal detachment, macular degeneration, and abnormal optic disc structure. Changes in fundus structure, especially abnormalities in the optic disc area, are of great significance for evaluating and monitoring the progression of pathological myopia. In the case of pathological myopia, due to the continuous growth of the eyeball, the optic disc may undergo morphological changes such as tilt, morphological distortion, or deepening of depression (especially deepening of the depression in the central part of the optic disc). These changes may not only induce or aggravate visual impairment, but may also become a risk factor for other eye diseases such as glaucoma. Therefore, it is particularly important to conduct careful observation and in-depth analysis of patients with pathological myopia.

[0003] To meet this clinical need, the development of optic disc image segmentation technology came into being. By accurately segmenting the optic disc area from the fundus image, doctors can comprehensively and carefully evaluate the morphological characteristics and pathological conditions of the optic disc. The application of optic disc image segmentation technology, especially in combination with advanced computer vision and machine learning algorithms, has significantly improved the objectivity and accuracy of the evaluation process.

[0004] In the traditional ophthalmic examination process, doctors often use traditional ophthalmoscopes to take photos of the optic disc of the fundus and then evaluate the optic disc. However, this evaluation method mainly relies on the doctor's clinical experience, and its efficiency is often unsatisfactory. Therefore, the key problem that needs to be solved urgently is how to effectively use optic disc image segmentation technology to improve the efficiency and accuracy of traditional ophthalmoscopes in fundus optic disc examinations. Summary of the invention

[0005] This application aims to propose an eye data processing and transmission method, system and ophthalmoscope equipment, the core goal of which is to achieve automatic segmentation and accurate recognition of fundus optic disc images by introducing cutting-edge data processing and transmission technology. Through this innovative means, we hope to assist doctors in completing optic disc assessment more efficiently, reduce assessment time, improve assessment accuracy, and thus comprehensively improve the overall efficiency and quality level of ophthalmic examinations.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, some embodiments of the present application provide an eye data processing and transmission method, which is applied to an ophthalmoscope, including: establishing a transmission relationship between an ophthalmoscope and a terminal device; receiving a shooting request signal sent by the terminal device; receiving a shooting control signal; according to the shooting request signal and the shooting control signal, controlling an image sensor to acquire at least one fundus image; inputting the fundus image into a processor, the processor calling a preset segmentation model to perform optic disc image segmentation on the fundus image to obtain an optic disc image segmentation result; in response to the shooting request signal, returning the optic disc image segmentation result and the fundus image to the terminal device.

[0007] Optionally, the shooting request signal includes a patient unique identity; the processor marks the optic disc image segmentation result and the captured fundus image based on the patient unique identity and the unique device identity of the ophthalmoscope; wherein, after returning the optic disc image segmentation result and the fundus image to the terminal device, the terminal device establishes a correspondence between the patient file and the optic disc image segmentation result and the fundus image according to the patient unique identity, and records the unique device identity of the ophthalmoscope that captured the fundus image.

[0008] Optionally, before the processor calls the preset segmentation model to perform optic disc image segmentation on the fundus image, the method also includes: when taking multiple fundus images, using the processor to screen the multiple fundus images for image quality and image content, and selecting one fundus image that meets preset requirements from the fundus images as the fundus image input into the preset segmentation model; when only one fundus image is taken, using the processor to judge the image quality and image content of the fundus image, if it does not meet the requirements, issuing a voice prompt instructing reshooting, if it meets the requirements, using the fundus image as the fundus image input into the preset segmentation model.

[0009] Optionally, the preset segmentation network model includes: an encoder and a decoder with a U-shaped symmetrical structure; the encoder includes five layers from top to bottom, each layer includes an SCNN block, a general convolution block connected to the SCNN block, and downsampling is performed; the decoder includes five layers from bottom to top, each layer includes a general convolution block, and the top 4 layers include multi-layer cross attention modules, and upsampling is performed; jump connections are used between the top 4 layers of encoders and decoders; the general convolution block consists of a convolution layer with a kernel size of 3x3, a step size of 1, and a padding of 1, and is connected to a batch normalization and ReLU activation function layer; the downsampling uses maximum pooling, a filter window size of 2x2, and a step size of 2.

[0010] Optionally, the SCNN block includes two groups of deep convolution-point convolution groups, and are arranged in sequence as follows: deep convolution layer, LN layer, point convolution layer, BN layer, SiLU layer, deep convolution layer, BN layer, GSiLU layer, point convolution layer and BN layer; the SCNN block is defined as follows:

[0011] f' l =DepthwiseConv(f l-1 )+f l-1 ;

[0012] f” l =σ2(BN{PointwiseConv(LN(f' l ))});

[0013] f” l =σ1(BN{DeptwiseConv(f” l )});

[0014] f l =BN(PointwiseConv(f” l ))+f' l ;

[0015] Among them, f l Represents the feature map of the Lth layer output of the SCNN block; σ1 represents the GSiLU layer, σ2 represents the SiLU layer, BN represents batch normalization, LN represents layer normalization, DepthwiseConv represents depthwise convolution, and PointwiseConv represents pointwise convolution.

[0016] Optionally, the optic disc image segmentation of the fundus image includes: inputting the fundus image into the encoder, performing feature extraction on each layer through the SCNN block to obtain a feature image, and increasing the number of channels of the feature image through the general convolution block to obtain a shallow feature image of each layer; wherein the shallow feature image of each layer of the encoder is input to the next layer of the encoder until the fifth layer after downsampling, and the shallow feature image of each layer is connected to the multi-layer cross attention module of the decoder through a jump connection; the shallow feature image of the fifth layer output by the encoder is used as the input of the first layer of the decoder, and each layer of the decoder obtains a deep feature image through upsampling and inputs it to the multi-layer cross attention module of the next layer of the decoder; the multi-layer cross attention module of each layer of the decoder performs feature fusion on the shallow features of the encoder with the upsampled deep features from the previous layer of the decoder to obtain a fused feature image, and reduces the number of channels of the fused feature image through the general convolution block; and the fused feature image output by the fifth layer of the decoder is obtained as the optic disc image segmentation result.

[0017] Optionally, the first layer of the encoder is jump-connected to the fifth layer of the decoder; the second layer of the encoder is jump-connected to the fourth layer of the decoder; the third layer of the encoder is jump-connected to the third layer of the decoder; and the fourth layer of the encoder is jump-connected to the second layer of the decoder.

[0018] Optionally, the multi-layer cross attention module is defined as follows:

[0019] f' s =α1(BN(Conv(f s )));

[0020] f' d =f' s +α1(BN(Conv(f d )));

[0021] C-Map(d)=α2(MLP(Maxpool(f' d ))+MLP(Avgpool(f' d )));

[0022] S-Map(s)=α2(α1(BN(Conv(Concat[MaxPool(f' s ); AvgPool(f' s )]))));

[0023] f out =f s ⊙C-map(d)+f d ⊙S-map(s);

[0024] Among them, f d represents the deep features after upsampling, f s represents the shallow features of the skip connection, f out represents the output feature; α1 represents the ReLU activation function; α2 represents the Sigmoid activation function; BN represents batch normalization; MaxPool represents maximum pooling; AvgPool represents average pooling; MLP represents a multilayer perceptron with one hidden layer, where the feature map size of the hidden layer is C / r×1×1, and r is the scaling ratio; Conv represents convolution; concat represents the connection operator; ⊙ represents the dot product operation.

[0025] According to another aspect of the present application, the present application also provides an eye data processing and transmission system, which is applied to an ophthalmoscope, comprising:

[0026] A communication module, used to establish a transmission relationship with a terminal device;

[0027] A receiving module, used for receiving a shooting request signal and a shooting control signal sent by a terminal device;

[0028] A control module, controlling the image sensor to acquire at least one fundus image according to the shooting request signal and the shooting control signal;

[0029] A processing module, inputting the fundus image into a processor, wherein the processor uses a preset segmentation model to perform optic disc image segmentation on the fundus image to obtain an optic disc image segmentation result;

[0030] The transmission module, in response to the shooting request signal, returns the optic disc image segmentation result and the fundus image to the terminal device.

[0031] According to another aspect of the present application, the present application further provides an ophthalmoscope, the ophthalmoscope comprising: a power chip, an image sensor, a communication interface, a processor, a memory and a bus, wherein the power chip, the image sensor, the communication interface, the processor and the memory are interconnected via the bus;

[0032] The memory stores computer program instructions, which, when executed, cause the processor to perform the steps of the eye data processing and transmission method.

[0033] The technical solution of this application aims to establish a data transmission mechanism between an ophthalmoscope and a terminal device, and introduce a segmentation model for image data processing. The specific implementation process is as follows: first, a shooting request signal from a terminal device is received; then, a shooting control signal is further received; then, according to the shooting request signal and the shooting control signal, the image sensor is accurately controlled to capture at least one fundus image; then, the acquired fundus image is imported into the processor for subsequent processing, and the processor will call a preset segmentation model to perform automatic segmentation of the optic disc image on the fundus image, and then obtain the optic disc image segmentation result; finally, according to the response requirements of the shooting request signal, the optic disc image segmentation result and the original fundus image are returned to the terminal device. This technical solution realizes the automation, accuracy and identification of fundus optic disc images by introducing advanced data processing and transmission technology and applying it to an ophthalmoscope. This innovative technical means aims to assist doctors to complete optic disc evaluation more efficiently, reduce the time required for evaluation, and improve the accuracy of evaluation results, thereby comprehensively improving the overall efficiency and quality level of ophthalmic examinations. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a flow chart of an eye data processing and transmission method provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the network structure of a preset segmentation model provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the network structure of the SCNN block provided in the embodiment of the present application;

[0037] Figure 4 A schematic diagram of the network structure of a multi-layer cross attention block provided in an embodiment of the present application;

[0038] Figure 5 An example of a sample image of a data set provided in an embodiment of the present application;

[0039] Figure 6 An example of annotated samples of dataset images provided in the embodiments of the present application;

[0040] Figure 7 An example of a dataset image enhancement sample provided in an embodiment of the present application;

[0041] Figure 8 A comparison chart of the image segmentation results provided in the embodiment of the present application and the results of other U-type networks;

[0042] Fig. 9 A schematic diagram of the system structure of an ophthalmoscope provided in an embodiment of the present application;

[0043] Fig.10 This is an optical principle diagram of the ophthalmoscope provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] As an important part of modern medical technology, the eye data processing and transmission method is widely used in ophthalmoscope equipment. The detailed steps are as follows:

[0046] Step S101: Establishing a transmission relationship between the ophthalmoscope and the terminal device. This step is the basis of the ocular data processing and transmission method, which ensures that the ophthalmoscope can communicate with the terminal device stably and efficiently. In actual operation, it is usually necessary to achieve a connection between the two through wireless communication technologies such as Bluetooth and Wi-Fi to ensure the real-time and accuracy of data transmission.

[0047] Step S102: Receive a shooting request signal sent by the terminal device. This step is the starting point of the entire eye data processing and transmission method. The terminal device sends a shooting request signal to the ophthalmoscope to trigger the subsequent image acquisition and processing process. The shooting request signal may include parameters such as shooting mode, exposure time, focal length, etc. to ensure that the quality of the acquired fundus image meets the medical diagnosis requirements.

[0048] Step S103: receiving a shooting control signal. After the shooting request signal is sent, a shooting control signal is generated by the shooting control button on the ophthalmoscope to guide the ophthalmoscope to shoot fundus images. The shooting control signal may include operations such as triggering shooting, adjusting focus, and changing light source brightness to ensure stability and accuracy during image acquisition.

[0049] Step S104: According to the shooting request signal and the shooting control signal, the image sensor is controlled to obtain at least one fundus image. After receiving the shooting request and the shooting control signal, the image sensor inside the ophthalmoscope will shoot the fundus image according to the preset parameters. The high resolution and fast response characteristics of the image sensor make the acquired fundus image clear and detailed, which helps doctors make accurate diagnosis.

[0050] Step S105: Input the fundus image into the processor, and the processor calls the preset segmentation model to perform optic disc image segmentation on the fundus image to obtain the optic disc image segmentation result. The fundus image contains a large amount of physiological information, and the optic disc, as a key part of it, is of great significance for the diagnosis of ophthalmic diseases. Therefore, the processor will use the pre-trained segmentation model to segment the optic disc area of ​​the fundus image and extract the image information of the optic disc part. The accuracy and efficiency of this step are crucial for subsequent diagnostic analysis. In the present application, the present application innovatively loads the self-developed innovative image segmentation algorithm model into the processor of the ophthalmoscope to perform efficient segmentation of the optic disc image.

[0051] Step S106: In response to the shooting request signal, the optic disc image segmentation result and the fundus image are returned to the terminal device. After completing the optic disc image segmentation, the ophthalmoscope will send the segmentation result and the original fundus image back to the terminal device. The doctor can view these images on the terminal device and perform further diagnostic analysis. In addition, the terminal device can also store, print or share the received image data to meet the usage requirements in different scenarios.

[0052] Through the detailed description of the above steps, we can see that the application of the eye data processing and transmission method in ophthalmoscope equipment is highly practical and advanced. It can not only realize the rapid acquisition and transmission of fundus images, but also use advanced image processing technology to accurately analyze and interpret images, providing strong diagnostic support for ophthalmologists.

[0053] As an optional embodiment:

[0054] The photographing request signal includes a unique identification of the patient;

[0055] The processor marks the optic disc image segmentation result and the captured fundus image based on the patient's unique identification and the ophthalmoscope's unique device identification; that is, after completing the optic disc image segmentation, the processor marks the segmentation result together with the captured fundus image to ensure the correlation and integrity between the two.

[0056] After the optic disc image segmentation result and the fundus image are returned to the terminal device, the terminal device establishes a correspondence between the patient file and the optic disc image segmentation result and the fundus image according to the patient's unique identity, and records the unique device identifier of the ophthalmoscope that took the fundus image.

[0057] Specifically, when the capture request signal is sent to the processor, the processor will mark the optic disc image segmentation result and the captured fundus image according to the patient's unique identification and the ophthalmoscope's unique device identification in the signal. This step is crucial because it ensures that each patient's medical data is closely linked to their identity, and also helps doctors quickly and accurately identify the patient's information during the subsequent diagnosis process.

[0058] After the marking is completed, these optic disc image segmentation results and fundus images will be returned to the terminal device. On the terminal device, the system will create a detailed file for the patient based on the patient's unique identity. This file not only records the patient's personal information, but also includes their eye examination data, such as optic disc image segmentation results and fundus images. At the same time, the file will also record the unique device identification of the ophthalmoscope used to take the fundus image for subsequent traceability and management.

[0059] It is worth mentioning that by establishing patient files and recording device identification, medical institutions can not only better manage patients' medical information, but also improve the accuracy and efficiency of diagnosis. When doctors need to review patients' historical records, they can easily find relevant information through the patient's unique identification, so as to have a more comprehensive understanding of the patient's condition. In addition, the recording of device identification also helps medical institutions manage and maintain equipment, ensuring the normal operation of equipment and the accuracy of inspection.

[0060] As an optional embodiment:

[0061] Before the processor calls the preset segmentation model to perform optic disc image segmentation on the fundus image, the method also includes: when taking multiple fundus images, using the processor to screen the multiple fundus images for image quality and image content, and selecting one fundus image that meets preset requirements from the fundus images as the fundus image input to the preset segmentation model; when only one fundus image is taken, using the processor to judge the image quality and image content of the fundus image, if it does not meet the requirements, issuing a voice prompt to instruct reshooting, if it meets the requirements, using the fundus image as the fundus image input to the preset segmentation model.

[0062] In the entire process of fundus image processing, the screening of image quality and content is a crucial step. This step determines the accuracy of subsequent optic disc image segmentation, so it must be carried out rigorously and meticulously. When taking multiple fundus images, we need to use advanced processor technology to strictly screen these images.

[0063] First, the processor will evaluate the image quality of each fundus image. This includes checking the image's clarity, contrast, brightness, and whether there are interference factors such as blur and noise. By using advanced image processing algorithms, the processor can automatically determine whether the image quality meets the preset requirements. If the quality of a fundus image does not meet the requirements, the processor will exclude it from the subsequent processing flow.

[0064] Secondly, the processor will also screen the content of the fundus image. This is mainly to check whether the image contains complete fundus structures, such as optic discs, blood vessels, etc. At the same time, it is also necessary to determine whether there are abnormalities such as lesions and hemorrhages in the image. By analyzing the image content, the processor can select a fundus image that best meets the preset requirements as the input for subsequent optic disc image segmentation.

[0065] When only one fundus image is taken, the processor will also judge the image quality and image content. If the fundus image does not meet the preset requirements, the processor will issue a voice prompt to instruct to retake the image. This helps to promptly discover and correct problems in the shooting process and ensure the accuracy and reliability of subsequent processing. If the fundus image meets the requirements, the processor will directly use it as the fundus image to input the preset segmentation model.

[0066] As an optional embodiment, Figure 2As shown, the preset segmentation network model includes: an encoder and a decoder with a U-shaped symmetrical structure; the encoder includes five layers from top to bottom, each layer includes an SCNN block, a general convolution (GeneralConv Block) connected to the SCNN block, and downsampling is performed; the decoder includes five layers from bottom to top, each layer includes a general convolution block, and the top 4 layers include multi-layer cross attention modules, and upsampling is performed; jump connections are used between the top 4 layers of encoders and decoders; the general convolution block consists of a convolution layer with a kernel size of 3x3, a step size of 1, and a padding of 1, and is connected to a batch normalization and ReLU activation function layer; the downsampling uses maximum pooling, a filter window size of 2x2, and a step size of 2.

[0067] Specifically, the preset segmentation network model of this application is named SCU-Net network structure. Figure 2 The SCU-Net network is divided into two main stages: the encoder stage and the decoder stage, which are closely connected through jump connections. In the encoder stage, the network first uses SCNN blocks (spatial convolutional neural network blocks) to extract rich spatial information of the image at different levels. The SCNN block enhances the network's perception of the spatial structure of the image by introducing convolution operations in the spatial dimension, thereby capturing more refined feature information.

[0068] After extracting rich spatial information, the encoder stage further uses general convolution blocks to increase the number of channels. This step aims to improve the network's feature representation capabilities, allowing the network to learn more dimensional image features. By increasing the number of channels, the network can better capture the details in the image, providing strong support for subsequent feature fusion and decoding processes.

[0069] Entering the decoder stage, SCU-Net adopts a multi-scale cross-attention module. This module aims to fuse the upsampled deep features with the shallow features from the jump connection. Through the multi-scale cross-attention mechanism, feature information at different levels is used to achieve feature complementarity and enhancement. This fusion method helps to improve the robustness and generalization ability of the network, enabling SCU-Net to handle image tasks in various complex scenarios.

[0070] Finally, at the end of the decoder stage, SCU-Net uses general convolution blocks again to reduce the number of channels. This step not only reduces the computational complexity of the network, but also helps to increase the inference speed of the model. By reducing the number of channels, the network can achieve more efficient image processing and analysis while ensuring performance.

[0071] As an optional embodiment, Figure 3 As shown, the SCNN block includes two groups of deep convolution-point convolution groups, which are arranged in sequence as follows: deep convolution layer, LN layer, point convolution layer, BN layer, SiLU layer, deep convolution layer, BN layer, GSiLU layer, point convolution layer and BN layer; the SCNN block is defined as follows:

[0072] f' l =DepthwiseConv(f l-1 )+f l-1 ;

[0073] f” l =σ2(BN{PointwiseConv(LN(f' l ))});

[0074] f” l =σ1(BN{DeptwiseConv(f” l )});

[0075] f l =BN(PointwiseConv(f” l ))+f' l ;

[0076] Among them, f l represents the feature map output by the Lth layer of the SCNN block; σ1 represents the GSiLU layer, σ2 represents the SiLU layer, BN represents batch normalization, LN represents layer normalization, DepthwiseConv represents depthwise convolution, and PointwiseConv represents pointwise convolution. Since the SCNN block does not change the resolution and number of channels of the feature map, we then use a general convolution block to double the number of channels.

[0077] The SCNN block plays a major role in the feature extraction part of the encoder. Specifically, it is an SCNN Block composed of two depthwise convolutions-pointwise convolutions, also known as depthwise separable convolutions. Depthwise convolution uses the number of groups equal to the number of channels to extract the spatial information of the image, followed by pointwise convolutions with a kernel size of 1×1, which merges cross-channel information while maintaining the spatial dimension. Instead of using traditional depthwise convolutions with large convolution kernels, the SCNN block uses two 3×3 convolution kernels and a stride of 1. This thin and deep architecture expands the receptive field to capture more spatial information, and the small-sized convolution kernels significantly reduce the difficulty of training. In addition, the SCNN Block uses an inverted bottleneck design between two depthwise convolution-pointwise convolution pairs, enlarging the hidden dimension between convolution layers to three times the input dimension before, to capture information of different spatial dimensions. The first depthwise convolution and the last residual connection enhance the efficiency of feature propagation. In addition, there are layer normalization (LN), batch normalization (BN), Sigmoid linear unit (SiLU) and global Sigmoid linear unit (GSiLU, which combines the characteristics of Sigmoid function and linear function, making the neural network have better stability during training and can adapt to different input ranges and output ranges) between convolution blocks.

[0078] As an optional embodiment, the optic disc image segmentation of the fundus image includes: inputting the fundus image into the encoder, performing feature extraction through the SCNN block at each layer to obtain a feature image, and increasing the number of channels of the feature image through the general convolution block to obtain shallow feature images of each layer; wherein the shallow feature image of each layer of the encoder is input to the next layer of the encoder until the fifth layer after downsampling, and the shallow feature image of each layer is connected to the multi-layer cross attention module of the decoder through a jump connection; the shallow feature image of the fifth layer output by the encoder is used as the input of the first layer of the decoder, and each layer of the decoder obtains a deep feature image through upsampling and inputs it to the multi-layer cross attention module of the next layer of the decoder; the multi-layer cross attention module of each layer of the decoder performs feature fusion on the shallow features of the encoder connected by jump connection and the upsampled deep features from the previous layer of the decoder to obtain a fused feature image, and reduces the number of channels of the fused feature image through the general convolution block; and the fused feature image output by the fifth layer of the decoder is obtained as the optic disc image segmentation result.

[0079] As an optional embodiment, the first layer of the encoder is jump-connected to the fifth layer of the decoder; the second layer of the encoder is jump-connected to the fourth layer of the decoder; the third layer of the encoder is jump-connected to the third layer of the decoder; the fourth layer of the encoder is jump-connected to the second layer of the decoder. Optionally, the fifth layer of the encoder and the first layer of the decoder share a general convolutional block.

[0080] like Figure 4 As shown in Figure 1, the structure of the multi-layer cross attention module is provided. Due to the dimension and scale differences between deep and shallow features, it is impossible to effectively align their information through simple upsampling. In order to better fuse features at different levels, we introduce multi-layer cross attention. Figure 4 As shown, the dotted line represents the input of the shallow feature, and the solid line represents the input of the deep feature. We first perform convolution, GELU activation, batch normalization (BN), and skip connection on the upsampled deep and shallow features, respectively. Then, the processed shallow features are added element-by-element with the deep features. Since the shallow features contain more spatial information and the deep features have richer channel information, we multiply the channel attention of the deep features with the shallow features element-by-element, and then multiply the spatial attention of the shallow features with the deep features element-by-element. Finally, we add the products of the two to obtain the fused features. By combining different levels of spatial-channel attention, the cross-attention gate can pay attention to both deep channel and shallow spatial information, thereby enhancing the network's ability to extract multi-level feature information.

[0081] Further, the multi-layer cross-attention module is defined as follows:

[0082] f' s =α1(BN(Conv(f s )));

[0083] f' d =f' s +α1(BN(Conv(f d )));

[0084] C-Map(d)=α2(MLP(Maxpool(f' d ))+MLP(Avgpool(f' d )));

[0085] S-Map(s)=α2(α1(BN(Conv(Concat[MaxPool(f' s ); AvgPool(f' s )]))));

[0086] f out =f's ⊙C-map(d)+f' d ⊙S-map(s);

[0087] Among them, f d represents the deep features after upsampling, f s represents the shallow features of the skip connection, f out represents the output feature; α1 represents the ReLU activation function; α2 represents the Sigmoid activation function; BN represents batch normalization; MaxPool represents maximum pooling; AvgPool represents average pooling; MLP represents a multilayer perceptron with one hidden layer, where the feature map size of the hidden layer is C / r×1×1, and r is the scaling ratio; Conv represents convolution; concat represents the connection operator; ⊙ represents the dot product operation.

[0088] As an optional embodiment, the present application also includes: a fundus image dataset for establishing a lightweight image segmentation network SCU-Net model. The present application utilizes fundus image datasets from the Baidu Brain Open Platform: PM Challenge, GON Challenge and AMD Challenge. The PM Challenge focuses on pathological myopia (PM) and contains a large amount of fundus image data specifically used for training and evaluating automatic retinal ganglion cell and cup area segmentation algorithms. The publicly available data includes 400 training images and 400 verification images, of which 381 training images and 379 verification images contain expert annotations. The AMD Challenge dataset focuses on age-related macular degeneration (AMD) research and contains a large number of detailed annotated fundus images. For AMD, only 400 training images are publicly available, of which 270 contain expert annotations. The GON fundus image dataset provides 3,400 training images and 400 verification images, in which the retinal ganglion cells and cup areas in each image are accurately annotated by experts. As Figure 5 As shown, (a) iChallenge-PM samples, (b) iChallenge-AMD samples, and (c) iChallenge-GON samples are given respectively. We merged the datasets together and obtained a total of 1830 valid images. These data were divided into training set and validation set in a ratio of 8:2, and the training set was further divided into training set and validation set in a ratio of 9:1. Finally, the training set includes 1318 images, the test set includes 366 images, and the validation set includes 146 images.

[0089] Data preprocessing and augmentation: In the iChallenge-GON dataset, experts have annotated the optic disc and cup separately, such as Figure 6However, since the optic cup is part of the optic disc and is located at its center, the labeled regions of the optic disc and cup need to be merged before training. We use threshold segmentation to set all pixel values ​​to 0 and display the segmented labeled image in Figure 6 middle.

[0090] In order to address the challenge of insufficient retinal image datasets, we perform data augmentation on the training data to increase the data volume and enhance the robustness of the model. The enhanced image data is as follows Figure 7 shown. Figure 7 Random operations such as random horizontal flipping and vertical flipping, random rotation (-90° to 90°), random grayscale, random addition of grid mask, random affine transformation, random addition of mask with size of 5%-10% of the image and aspect ratio between 1 and 2, and random cropping of one quarter of the image area are shown, where (a) represents the original image, b to h represent the flipped image, rotated image, grayscale image, grid mask image, affine transformation image, erased image and cropped image, respectively.

[0091] To evaluate the metrics of our data processing, we use intersection over union (IoU), accuracy (Acc), and F1 score (F1) as evaluation metrics for the experiment. IoU emphasizes the degree of overlap between the predicted and actual regions, accuracy focuses on the correctness of the overall classification, and the F1 score provides a balance between precision and recall. By combining these metrics, researchers can comprehensively evaluate and compare different image segmentation models.

[0092] In the field of deep learning-driven image segmentation, IoU is widely used as the main evaluation metric. IoU provides a quantifiable standard for measuring the degree of overlap between the predicted segmented area and the actual segmented area. Specifically, IoU is defined as the ratio of the intersection area to the union area between the predicted annotation and the actual annotation. Its mathematical expression is:

[0093] IoU = TP / (TP+FP+FN)

[0094] TP (True Positive) refers to samples that are predicted as positive and are actually positive, FP (False Positive) refers to samples that are predicted as positive but are actually negative, FN (False Negative) refers to samples that are predicted as negative but are actually positive, and TN (True Negative) refers to samples that are predicted as negative and are actually negative. The IoU value ranges from 0 to 1, and the higher the value, the better the segmentation performance.

[0095] Accuracy (ACC) is another key metric for evaluating the overall performance of the model on the entire dataset. Accuracy is defined as the ratio of the number of correctly classified pixels to the total number of pixels. The formula is as follows:

[0096]

[0097] Among them, TP represents true positive examples (that is, the number of samples predicted by the model to be positive and actually are positive examples), TN represents true negative examples (that is, the number of samples predicted by the model to be negative examples and actually are negative examples), FP represents false positive examples (that is, the number of samples predicted by the model to be positive examples but actually are negative examples), and FN represents false negative examples (that is, the number of samples predicted by the model to be negative examples but actually are positive examples).

[0098] Accuracy is an intuitive indicator used to evaluate the overall segmentation effect of the model. The higher the accuracy, the stronger the performance.

[0099] The F1 score (F1) can be considered as the harmonic mean of the model's precision and recall, taking into account both the precision and recall of the classification model. The F1 score is particularly effective in measuring the model's predictive performance for minority class samples, especially for unbalanced samples. Its calculation formula is as follows:

[0100]

[0101] Among them, precision is the proportion of samples predicted by the model as positive that are actually positive, and recall is the proportion of samples predicted by the model as positive that are actually positive. The product of these two indicators is the F1 score.

[0102] In classification tasks, the F1 score is a commonly used performance metric that can help us evaluate the predictive performance of the model. The F1 score is especially important when dealing with unbalanced datasets because it can better reflect the model's predictive performance for minority class samples. The F1 score ranges from 0 to 1, and the higher the value, the better the model performance.

[0103] Implementation details:

[0104] The experimental implementation details of this application are as follows: The computer hardware configuration includes a high-performance CPU with Intel 12th generation Core i7 and 16GB RAM, and an NVIDIA RTX 4080 GPU with 16GB VRAM. These hardware components ensure the efficiency and stability of the entire experimental process. The software configuration selects the Ubuntu 16.04 Linux operating system as the basic environment, and uses the PyTorch GPU framework, CUDA, and Python programming languages ​​to implement model training and testing. During the model training process, we use the Adam optimizer to update and optimize the model parameters to ensure the efficiency and accuracy of model training. The initial learning rate is set to 0.01 to balance the training speed and model convergence. To further improve the model performance, we reduce the learning rate to the original 0.2 in the 20th, 40th, and 80th training cycles. This helps the model achieve stable convergence in the later stages of training. In order to maximize the use of hardware resources, while taking into account the computational efficiency, training speed, and the stability and convergence of the model, we set the batch size to 8. Through these meticulous settings and optimizations, our experimental platform provides strong support for model training and ensures the reliability and effectiveness of the experimental results.

[0105] As shown in the following table, in the quantitative evaluation stage, Table 1 comprehensively summarizes the performance comparison of our proposed retinal image segmentation network SCU-Net with various U-shaped convolutional image segmentation networks (including U-Net, U-Net++, U-Net3+, AttentionU-Net) and other advanced methods (such as phase fusion PSPNet, 2C-128N-0D) using the same dataset. As can be seen from the table, our SCU-Net performs best in all performance indicators. Compared with other methods, SCU-Net has improved in key indicators such as pixel accuracy (PixAcc), intersection over union (IoU) and F1 score, which increased by 0.33%-2.99%, 0.22%-3.18% and 0.14%-2.68% respectively. In addition, we compared the performance of these algorithms on different datasets, and the specific results are shown in Tables 2, 3 and 4 respectively. The experimental results show that the SCU-Net proposed in this paper shows good performance in the optical disc segmentation task and achieves the best results on all datasets.

[0106] Table 1 - Comparative experimental results based on iChallenge-PM, iChallenge-GON and iChallenge-AMD

[0107]

[0108] Table 2 - Comparative experimental results based on iChallenge-PM

[0109]

[0110] Table 3 Comparative experiment based on iChallenge-AMD

[0111]

[0112] Table 4 Comparative experiments based on iChallenge-GON

[0113]

[0114] Table 5: Comparison of model parameters

[0115]

[0116] In the qualitative evaluation phase, we conducted an in-depth analysis of the image segmentation results of the proposed method and compared it with the U-net segmentation results mentioned in the quantitative evaluation. Figure 8 As shown in Table 1, the performance of this model is consistent with the data in Table 1 in the quantitative experiment. Through careful comparison, we can clearly see that our method shows a better performance in measuring pixel-level segmentation accuracy. Especially when dealing with the key link of the optic disc edge, SCU-Net shows a more accurate segmentation ability, and its effect is closer to the real ground truth data in actual scenes in many aspects. Figure 8 In the figure, (a) original image: the original image is the initial input data in image processing, usually an unprocessed image.

[0117] (b) Image mask. Image mask is a special type of data used in image processing and computer vision, which is usually used to specify that certain areas in an image should be processed or analyzed.

[0118] (c) Segmentation result of U-Net,U-Net is a commonly used deep learning network structure, mainly used for image segmentation tasks. It consists of an encoder (U part) and a decoder (N part), in this way, it can decompose the input image into feature maps and labels, and use this information to generate segmented images.

[0119] (d) Segmentation result of U-Net++. U-Net++ is an improved version of U-Net, which improves the performance of U-Net by adding more convolutional layers and skip connections.

[0120] (e) Segmentation result of U-Net3+. U-Net3+ is a further improved version of U-Net++, which further improves the performance by adding more convolutional layers and a more complex network structure.

[0121] (f) Segmentation result of Attention U-Net. Attention U-Net is a special U-Net network structure that introduces an attention mechanism to better handle image segmentation tasks. In Attention U-Net, the network allocates more attention to specific areas of the input image to better identify and segment these areas. The above are the meanings and contents of these six options.

[0122] (g) is the segmentation result of the SCU-Net image segmentation network of this application.

[0123] In addition, we conducted a series of ablation experiments to evaluate the specific contribution of each of our proposed modules to the model performance. As shown in Table 6, after adding the SCCNN block to the original U-Net, not only the performance is significantly improved, but also the number of parameters and GFLOPs is reduced, thereby improving the running speed. In addition, after introducing multi-layer cross-attention gates, the accuracy is significantly improved despite a slight increase in the number of GFLOPs. When the model is trained using an extended dataset, the segmentation accuracy is further enhanced. These ablation study results fully demonstrate the positive impact of each of our proposed modules in enhancing optic disc image segmentation.

[0124] Table 6: Ablation study results using different strategy combinations

[0125]

[0126] The importance of optical focus image segmentation technology in modern ophthalmology is self-evident. Faced with the problems of high computing resource consumption, insufficient feature fusion, and data scarcity in the current segmentation model, we proposed SCU-Net and invented a processing and transmission method and system for ophthalmic data based on it. The model uses efficient small kernel convolution blocks and SCCN blocks, and innovatively introduces multi-level cross-attention gates to fuse features at different levels. For scarce fundus image data, we use various image enhancement techniques to expand the dataset. Experimental results show that our proposed method outperforms other existing methods on multiple datasets, fully verifying its effectiveness. In addition, experimental studies have shown the importance of each module we proposed for fundus image segmentation.

[0127] According to another aspect of the present application, the present application also provides an eye data processing and transmission system, which is applied to an ophthalmoscope, comprising:

[0128] A communication module, used to establish a transmission relationship with a terminal device;

[0129] A receiving module, used for receiving a shooting request signal and a shooting control signal sent by a terminal device;

[0130] A control module, controlling the image sensor to acquire at least one fundus image according to the shooting request signal and the shooting control signal;

[0131] A processing module, inputting the fundus image into a processor, wherein the processor uses a preset segmentation model to perform optic disc image segmentation on the fundus image to obtain an optic disc image segmentation result;

[0132] The transmission module, in response to the shooting request signal, returns the optic disc image segmentation result and the fundus image to the terminal device.

[0133] For the specific definition of the eye data processing and transmission system, please refer to the definition of the eye data processing and transmission method above, which will not be repeated here. Each module / unit in the above-mentioned eye data processing and transmission system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules / units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0134] In one embodiment, the present application provides an ophthalmoscope, which includes: a power chip, an image sensor, a communication interface, a processor, a memory and a bus, wherein the power chip, the image sensor, the communication interface, the processor, and the memory are interconnected via a bus; and specifically also includes a shooting control module on the ophthalmoscope, which is used to generate a shooting control signal.

[0135] The memory stores computer program instructions, which, when executed, cause the processor to perform the steps of the eye data processing and transmission method.

[0136] like Fig. 9 , which is a schematic diagram of the system structure of the ophthalmoscope provided in an embodiment of the present application.

[0137] The power chip, the image sensor, the communication interface, the processor, and the memory are interconnected through a bus, the power chip is used to provide power supply, the communication interface is used to receive or send data to the terminal device, and the processor is a single-core or multi-core central processing unit, or a specific integrated circuit, or one or more integrated circuits configured to implement the embodiment of the present invention. The memory can be a random access memory (RAM) or a non-volatile memory (non-volatile memory), such as at least one hard disk memory. The memory is used to store computer execution instructions. Specifically, the computer execution instructions may include programs.

[0138] In this embodiment, when the processor calls the program, it can execute the operations of the eye data processing and transmission method, which will not be described in detail here.

[0139] Furthermore, the ophthalmoscope also includes an optical path system, such as Fig.10 As shown, the condenser is generally a plano-concave mirror. To reduce astigmatism, the flat side faces the bulb. The filament of the bulb should be located on the focal plane of the condenser so that the light forms parallel light after passing through the condenser. The aperture diameter is generally 1 mm. The function of the prism is to converge and deflect the light to illuminate the fundus. The handheld fundus ophthalmoscope places a lens at the peephole position of the original ophthalmoscope so that the image of the fundus is focused and imaged on the CCD or CMOS camera (image sensor). The image sensor sends the acquired fundus image to the processor for processing.

[0140] It should be understood that the processor provided in the above embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0141] It should also be understood that the number of processors in the computer device in the above embodiments of the present application can be one or more, and can be adjusted according to the actual application scenario. This is only an exemplary description and is not limited. The number of memories in the embodiments of the present application can be one or more, and can be adjusted according to the actual application scenario. This is only an exemplary description and is not limited.

[0142] It should also be noted that when a computer device includes a processor (or processing unit) and a memory, the processor in this application can be integrated with the memory, or the processor and the memory can be connected through an interface. It can be adjusted according to the actual application scenario and is not limited.

[0143] The present application provides a chip system, which includes a processor for supporting a computer device (client or server) to implement the functions of the controller involved in the above method, such as processing the data and / or information involved in the above method. In one possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The chip system can be composed of a chip or a chip and other discrete devices.

[0144] In another possible design, when the chip system is a chip in a user device or access network, the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit so that the chip in the client or management server may perform the steps of the common sense question answering method. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the client or management server, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0145] It should be understood that the methods and / or embodiments in the embodiments of the present application can be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the processing unit, the above functions defined in the method of the present application are executed.

[0146] It should be understood that the controller or processor mentioned in the above embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0147] It should also be understood that the number of processors or controllers in the computer device or chip system in the above embodiments of the present application can be one or more, and can be adjusted according to the actual application scenario. This is only an exemplary description and is not limited. The number of memories in the embodiments of the present application can be one or more, and can be adjusted according to the actual application scenario. This is only an exemplary description and is not limited.

[0148] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0149] In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0150] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0152] As another aspect, the embodiments of the present application further provide a computer-readable medium, which may be included in the device described in the above embodiments; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions, which may be executed by a processor to implement the steps of the methods and / or technical solutions of the above-mentioned multiple embodiments of the present application. The computer may be the above-mentioned computer device (client or server or other computer network communication device).

[0153] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.

[0154] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0155] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0156] In addition, an embodiment of the present application further provides a computer program, which is stored in a computer device, so that the computer device executes the method for controlling code execution.

[0157] It should be noted that the present application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0158] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, the terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present invention.

[0159] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the invention described herein can be implemented in an order other than those illustrated or described herein. The singular forms "a", "an", and "the" used in the embodiments of the present application are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0160] In addition, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0161] In the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. Depending on the context, the words "if" or "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination" or "in response to detection". Similarly, depending on the context, the phrases "if it is determined" or "if (stated condition or event) is detected" can be interpreted as "when determined" or "in response to determination" or "when (stated condition or event) is detected" or "in response to detection (stated condition or event)".

[0162] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing and transmitting eye data, applied to an ophthalmoscope, characterized in that: include: Establishing a transmission relationship between the ophthalmoscope and the terminal device; Receiving a shooting request signal sent by a terminal device; receiving a shooting control signal; Controlling the image sensor to acquire at least one fundus image according to the shooting request signal and the shooting control signal; The fundus image is input into a processor, and the processor uses a preset segmentation model to perform optic disc image segmentation on the fundus image to obtain an optic disc image segmentation result; In response to the photographing request signal, the optic disc image segmentation result and the fundus image are returned to the terminal device.

2. The method according to claim 1, characterized in that: The photographing request signal includes a unique identification of the patient; The processor marks the optic disc image segmentation result and the captured fundus image based on the patient unique identification and the unique device identification of the ophthalmoscope; After the optic disc image segmentation result and the fundus image are returned to the terminal device, the terminal device establishes a correspondence between the patient file and the optic disc image segmentation result and the fundus image according to the patient's unique identity, and records the unique device identifier of the ophthalmoscope that took the fundus image.

3. The method according to claim 2, characterized in that Before the processor calls the preset segmentation model to perform optic disc image segmentation on the fundus image, the method further includes: When multiple fundus images are taken, the processor is used to screen the multiple fundus images for image quality and image content, and one fundus image that meets preset requirements is selected from the fundus images as the fundus image input into the preset segmentation model; When only one fundus image is taken, the processor is used to judge the image quality and image content of the fundus image. If it does not meet the requirements, a voice prompt is issued to instruct reshooting. If it meets the requirements, the fundus image is used as the fundus image input into the preset segmentation model.

4. The method according to claim 1, characterized in that: The preset segmentation network model includes: an encoder and a decoder in a U-shaped symmetrical structure; The encoder includes five layers from top to bottom, each layer includes a SCNN block, a general convolution block following the SCNN block, and downsampling; The decoder consists of five layers from bottom to top, each layer contains a general convolutional block, and the top four layers contain multi-layer cross attention modules and perform upsampling; The top 4 layers of encoders and decoders use skip connections; The general convolutional block consists of a convolutional layer with a kernel size of 3x3, a stride of 1, and a padding of 1, and is connected to a batch normalization and ReLU activation function layer; The downsampling uses maximum pooling, with a filter window size of 2x2 and a step size of 2.

5. The method according to claim 4, characterized in that The SCNN block includes two groups of deep convolution-point convolution groups, and is arranged in sequence as follows: deep convolution layer, LN layer, point convolution layer, BN layer, SiLU layer, deep convolution layer, BN layer, GSiLU layer, point convolution layer and BN layer; the SCNN block is defined as follows: f' l =DepthwiseConv(f l-1 )+f l-1 ; "f" l =σ2(BN{PointwiseConv(LN(f' l ))}); f” l =σ1(BN{DeptwiseConv(f” l )}); f l =BN(PointwiseConv(f” l ))+f' l ; Among them, f l Represents the feature map of the Lth layer output of the SCNN block; σ1 represents the GSiLU layer, σ2 represents the SiLU layer, BN represents batch normalization, LN represents layer normalization, DepthwiseConv represents depthwise convolution, and PointwiseConv represents pointwise convolution.

6. The method according to claim 4, characterized in that The performing optic disc image segmentation on the fundus image comprises: The fundus image is input into the encoder, and each layer is subjected to feature extraction by the SCNN block to obtain a feature image, and the number of channels of the feature image is increased by the general convolution block to obtain a shallow feature image of each layer; wherein the shallow feature images of each layer of the encoder are input to the next layer of the encoder until the fifth layer after downsampling, and the shallow feature images of each layer are connected to the multi-layer cross attention module of the decoder through jump connection; The shallow feature image of the fifth layer output by the encoder is used as the input of the first layer of the decoder, and each layer of the decoder obtains a deep feature image by upsampling and inputs it into the multi-layer cross attention module of the next layer of the decoder; The multi-layer cross attention module of each layer of the decoder performs feature fusion on the shallow features of the jump-connected encoder and the upsampled deep features from the previous layer of the decoder to obtain a fused feature image and reduces the number of channels of the fused feature image through the general convolution block; The fused feature image outputted by the fifth layer of the decoder is obtained as the optic disc image segmentation result.

7. The method according to claim 6, characterized in that The first layer of the encoder is jump-connected to the fifth layer of the decoder; The second layer of the encoder is jump-connected to the fourth layer of the decoder; The third layer of the encoder is jump-connected to the third layer of the decoder; The fourth layer of the encoder is jump-connected to the second layer of the decoder.

8. The method according to claim 6, characterized in that The multi-layer cross-attention module is defined as follows: f' s =α1(BN(Conv(f s ))); f' d =f' s +α1(BN(Conv(f d ))); C-Map(d)=α2(MLP(Maxpool(f' d ))+MLP(Avgpool(f' d ))); S-Map(s)=α2(α1(BN(Conv(Concat[MaxPool(f' s );AvgPool(f' s )])))); f out =f’ s ⊙C-map(d)+f’ d ⊙S-map(s); Among them, f d represents the deep features after upsampling, f s represents the shallow features of the skip connection, f out represents the output feature; α1 represents the ReLU activation function; α2 represents the Sigmoid activation function; BN represents batch normalization; MaxPool represents maximum pooling; AvgPool represents average pooling; MLP represents a multilayer perceptron with one hidden layer, where the feature map size of the hidden layer is C / r×1×1, and r is the scaling ratio; Conv represents convolution; concat represents the connection operator; ⊙ represents the dot product operation.

9. An eye data processing and transmission system, applied to an ophthalmoscope, characterized in that: include: A communication module, used to establish a transmission relationship with a terminal device; A receiving module, used for receiving a shooting request signal and a shooting control signal sent by a terminal device; A control module, controlling the image sensor to acquire at least one fundus image according to the shooting request signal and the shooting control signal; A processing module, inputting the fundus image into a processor, wherein the processor uses a preset segmentation model to perform optic disc image segmentation on the fundus image to obtain an optic disc image segmentation result; The transmission module, in response to the shooting request signal, returns the optic disc image segmentation result and the fundus image to the terminal device.

10. An ophthalmoscope, characterized in that: The ophthalmoscope comprises: a power chip, an image sensor, a communication interface, a processor, a memory and a bus, wherein the power chip, the image sensor, the communication interface, the processor and the memory are interconnected via the bus; The memory stores computer program instructions, which, when executed, enable the processor to perform the steps of the eye data processing and transmission method according to any one of claims 1 to 8.

Citation Information

Cited By

  • Eye disease diagnosis system and method based on lesion space structure relationship

    CN120032862A