Auxiliary focusing light for ophthalmic ultrasound biomicroscopy

Abstract

The present invention discloses an auxiliary gaze lamp for ophthalmic ultrasonic biomicroscope examination, which can automatically sense the patient's adaptation to the light source. If it is detected that the current light source state is not suitable for the patient, it can timely and automatically adjust to instruct the doctor to adjust the light source state in time, so that the light source state is suitable for the patient, and the patient can quickly and stably look at the light source, thereby improving the efficiency and accuracy of eye examination. The state of the light source is adjusted in time to shorten the impact of the uncomfortable light source state on the patient, and the impact of strong light and other conditions on the patient's eyes can be reduced. In addition, according to the combination of the eyeball ultrasonic image sequence and the monitoring image sequence, the light source state information is obtained to improve the accuracy of the obtained light source state information, and the light source state suitable for the patient can be quickly adjusted to improve the efficiency and accuracy of eye examination.

Description

Technical Field

[0001] The present invention relates to the technical field of auxiliary lamps, and in particular to an auxiliary gaze lamp used for ophthalmic ultrasonic biomicroscope examination. Background Art

[0002] The function of the auxiliary gaze light used in ophthalmic examinations is to allow the patient to sense the direction in which they need to gaze so that their eyes can be fixed in that direction, thereby helping to obtain examination information of the position and range that they want to examine. For example, for an ophthalmic ultrasonic biomicroscope examination, the patient is required to turn his or her eyes according to the instructions of the auxiliary gaze light to obtain an ultrasonic image of the eye position and range that needs to be examined.

[0003] The current auxiliary gaze lights mainly use flashing or highlighting strong light to prompt patients to move their eyeballs. However, this method has the following problems: 1. When the light flashes, patients tend to blink, reducing the efficiency and accuracy of the ophthalmic ultrasound biomicroscope in collecting ultrasound images of the parts that need to be examined. 2. The highlighting strong light will also make patients afraid of light and blink or avoid light, which will also affect the efficiency and accuracy of collecting eye ultrasound images. At the same time, users need to insist on controlling their eyeballs to look at the strong light, which puts a lot of pressure on users and has a poor experience. Summary of the invention

[0004] The object of the present invention is to provide an auxiliary gaze lamp for ophthalmic ultrasonic biomicroscope examination to solve the above-mentioned problems existing in the prior art.

[0005] An embodiment of the present invention provides an auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy, comprising a microprocessor, wherein the microprocessor is used to execute the following steps of determining the state of a lamp source:

[0006] Obtain an eye ultrasound image sequence and a monitoring image sequence of a patient; the eye ultrasound image sequence includes multiple frames of eye ultrasound images, which are arranged in the order of acquisition time; the eye ultrasound images are acquired by an ophthalmic ultrasound biological microscope; the last frame of eye ultrasound image in the eye ultrasound image sequence is the current eye ultrasound image, and the acquisition time point of the current eye ultrasound image is the current time node; the acquisition time point of the previous frame of eye ultrasound image of the current eye ultrasound image is the previous time node; the monitoring image sequence includes multiple frames of eye images, which are arranged in the order of acquisition time; the eye images are acquired by a visible light camera; the last frame of eye image in the monitoring image sequence is the current eye image; the acquisition time point of the current eye image is the current time node; the acquisition time of the previous frame of eye image of the current eye image is the previous time node;

[0007] Judging whether the current light source state is suitable for the patient according to the eye ultrasound image sequence; the light source state includes brightness and color temperature;

[0008] If the current light source state is not suitable for the patient, the light source state adjustment information is obtained according to the eye ultrasound image sequence and the monitoring image sequence; the light source state adjustment information is used to instruct the doctor to adjust the light source state of the light source.

[0009] Optionally, judging whether the current light source state is suitable for the patient according to the eye ultrasound image sequence includes:

[0010] Based on the ocular ultrasound image sequence, the movement trajectory of the patient's eye reference point within a set time period is obtained and counted as the eye trajectory;

[0011] Obtaining the motion trajectory of the light source within a set time period, which is counted as the light source trajectory;

[0012] Determine whether the eye trajectory is consistent with the light source trajectory;

[0013] If the eyeball trajectory tends to be consistent with the light source trajectory, it is determined that the current light source state is suitable for the patient;

[0014] If the eyeball trajectory is not consistent with the light source trajectory, it is determined that the current light source state is not suitable for the patient.

[0015] Optionally, the movement trajectory of the patient's eye reference point within a set time period is obtained based on the eye ultrasound image sequence and counted as the eye trajectory, including:

[0016] Obtaining the middle position of the current eye ultrasound image;

[0017] The middle position is used as the eye reference point;

[0018] Identifying an eye reference point in each frame of an eye ultrasound image in an eye ultrasound image sequence, and obtaining image coordinates of the eye reference point in each frame of the eye ultrasound image;

[0019] Mapping the image coordinates of the eye reference point in each frame of the eye ultrasound image to the current eye ultrasound image to obtain the reference point trajectory coordinates;

[0020] The reference point trajectory coordinates are connected in sequence according to the arrangement order of the eye ultrasound images in the eye ultrasound image sequence to obtain the eye trajectory.

[0021] Optionally, the determining whether the eyeball trajectory is consistent with the light source trajectory includes:

[0022] Obtain a difference sequence between the eye trajectory and the light source trajectory; the difference sequence includes multiple trajectory differences, the trajectory difference is the Euclidean distance between the matching trajectory point pairs in the eye trajectory and the light source trajectory; the matching trajectory point pairs include eye trajectory points and light source trajectory points, the eye trajectory points are nodes of the eye trajectory, and the light source trajectory points are nodes of the light source trajectory; the time nodes corresponding to the eye trajectory points and the light source trajectory points of the matching trajectory point pairs are the same;

[0023] If the difference sequence tends to converge, it is determined that the eye trajectory and the light source trajectory tend to be consistent;

[0024] If the difference sequence does not tend to converge, it is determined that the eye trajectory and the light source trajectory are not consistent.

[0025] Optionally, obtaining the light source state information according to the eye ultrasound image sequence and the monitoring image sequence includes:

[0026] Obtaining the patient's eyelid state sequence according to the monitoring image sequence; the eyelid state sequence includes a plurality of eye closure indexes corresponding to a plurality of frames of eye images; the eye closure index is used to indicate whether the eye in the eye image is in an eye closure state;

[0027] The light source state information is obtained based on the eyelid state sequence and the difference sequence.

[0028] Optionally, obtaining the light source state information according to the eyelid state sequence and the difference sequence includes:

[0029] Determine the segmentation step size based on the eyelid state sequence;

[0030] Segmenting the difference sequence based on the segmentation step length to obtain a first segmentation sequence, the first segmentation sequence including a plurality of segmentation difference blocks, each segmentation difference block including one or more trajectory differences;

[0031] Obtain the sum of all trajectory differences in each segmentation difference block, and count it as the first state value; multiple segmentation difference blocks correspond to multiple first state values;

[0032] Segmenting the eyelid state sequence based on the segmentation step length to obtain a second segmentation sequence, the second segmentation sequence including a plurality of segmented eye-closing blocks, each segmented eye-closing block including one or more eye-closing indexes;

[0033] Obtaining the sum of all eye-closing indexes in each segmented eye-closing block, and calculating the sum as the second state value; multiple segmented eye-closing blocks correspond to multiple second state values;

[0034] Obtain a first judgment value based on a plurality of first state values, and obtain a second judgment value based on a plurality of second state values;

[0035] If the second judgment value is greater than the first threshold value, and the first judgment value is greater than the second threshold value, it is determined that the adjustment of the light source state information is to reduce the brightness in the light source state; if the second judgment value is greater than the first threshold value, and the first judgment value is less than or equal to the second threshold value, it is determined that the adjustment of the light source state information is to increase the color temperature in the light source state;

[0036] If the second judgment value is less than or equal to the first threshold value, and the first judgment value is greater than the second threshold value, it is determined that adjusting the light source state information is to increase the brightness in the light source state.

[0037] Optionally, obtaining the first judgment value based on a plurality of first state values, and obtaining the second judgment value based on a plurality of second state values, includes:

[0038] Using the plurality of first state values ​​as inputs of a first joint prediction network, and using the output of the first joint prediction network as a first judgment value;

[0039] Using the plurality of second state values ​​as inputs of a second joint prediction network and using the second joint prediction network as output of a second judgment value;

[0040] The structures of the first joint prediction network and the second joint prediction network are both joint time memory networks. For different training data, the parameters of the first joint prediction network and the second joint prediction network are different.

[0041] The joint temporal memory network includes multiple interconnected long short-term memory networks; the output of the previous long short-term memory network is used as the value of the input gate of the next long short-term memory network.

[0042] Optionally, the eye closure index has a value of 0 or 1. When the eye closure index has a value of 1, it indicates that the eye in the eye image is in a closed state. When the eye closure index has a value of 0, it indicates that the eye in the eye image is in an open state.

[0043] Optionally, the microprocessor is electrically connected to an ophthalmic ultrasonic biomicroscope and a visible light camera, the ophthalmic ultrasonic biomicroscope is used to collect a patient's eye ultrasonic image sequence and send the eye ultrasonic image sequence to the microprocessor; the visible light camera is used to collect a patient's monitoring image sequence and send the monitoring image sequence to the microprocessor.

[0044] Optionally, the auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy also includes a light source.

[0045] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0046] The embodiment of the present invention provides an auxiliary gaze lamp for ophthalmic ultrasonic biomicroscope examination, which can automatically sense the patient's adaptation to the light source. If it is detected that the current light source state is not suitable for the patient, it can timely and automatically adjust to instruct the doctor to adjust the light source state in time, so that the light source state is suitable for the patient, and the patient can quickly and stably look at the light source, thereby improving the efficiency and accuracy of eye examination. Timely adjustment of the light source state can shorten the impact of the uncomfortable light source state on the patient, and can reduce the impact of strong light and other conditions on the patient's eyes. In addition, according to the combination of the eye ultrasound image sequence and the monitoring image sequence, the light source state information is obtained to improve the accuracy of the obtained light source state information, and the light source state suitable for the patient can be quickly adjusted to improve the efficiency and accuracy of eye examination. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of a method for determining a light source state provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0049] Example

[0050] An embodiment of the present invention provides an auxiliary gaze lamp for ophthalmic ultrasonic biomicroscope examination, comprising a microprocessor; the microprocessor is electrically connected to the ophthalmic ultrasonic biomicroscope and a visible light camera, the ophthalmic ultrasonic biomicroscope is used to collect a patient's eye ultrasonic image sequence, and send the eye ultrasonic image sequence to the microprocessor; the visible light camera is used to collect a patient's monitoring image sequence, and send the monitoring image sequence to the microprocessor.

[0051] In an embodiment of the present invention, a microprocessor for an auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy is used to execute the following steps of a method for determining a lamp source state, such as: Figure 1 As shown, the method for determining the light source state includes the following steps:

[0052] S101: Obtain a patient's eye ultrasound image sequence and a monitoring image sequence.

[0053] Among them, the eye ultrasound image sequence includes multiple frames of eye ultrasound images, and the multiple frames of eye ultrasound images are arranged in the order of acquisition time. The eye ultrasound image is acquired by an ophthalmic ultrasound biomicroscope. The last frame of eye ultrasound image in the eye ultrasound image sequence is the current eye ultrasound image. The acquisition time point of the current eye ultrasound image is the current time node, and the acquisition time point of the previous frame of eye ultrasound image of the current eye ultrasound image is the previous time node. The monitoring image sequence includes multiple frames of eye images, and the multiple frames of eye images are arranged in the order of acquisition time. The eye image is acquired by a visible light camera. The last frame of eye image in the monitoring image sequence is the current eye image. The acquisition time point of the current eye image is the current time node, and the acquisition time of the previous frame of eye image of the current eye image is the previous time node.

[0054] S102: judging whether the current light source state is suitable for the patient according to the eye ultrasound image sequence.

[0055] The light source status includes brightness and color temperature.

[0056] S103: If the current light source state is not suitable for the patient, obtain the light source state information for adjustment according to the eye ultrasound image sequence and the monitoring image sequence.

[0057] Optionally, after S103, the method further includes sending light source adjustment state information to the doctor. The light source adjustment state information is used to instruct the doctor to adjust the light source state of the light source. Specifically, the light source adjustment state information is sent to the following display screen.

[0058] By adopting the above scheme, the auxiliary gaze lamp for ophthalmic ultrasonic biomicroscope examination provided by the embodiment of the present invention can automatically sense the patient's adaptation to the light source. If it is detected that the current light source state is not suitable for the patient, it can timely and automatically adjust to instruct the doctor to adjust the light source state in time, so that the light source state is suitable for the patient, and the patient can quickly and stably look at the light source, thereby improving the efficiency and accuracy of eye examination. Timely adjustment of the light source state can shorten the impact of the uncomfortable light source state on the patient, and can reduce the impact of strong light and other conditions on the patient's eyes.

[0059] In addition, the light source status information is obtained by combining the eye ultrasound image sequence and the monitoring image sequence, thereby improving the accuracy of obtaining the light source status information, being able to quickly adjust the light source status suitable for the patient, and improving the efficiency and accuracy of eye examination.

[0060] The auxiliary gaze lamp used for ophthalmic ultrasonic biomicroscope examination also includes a light source. After sending the doctor the light source adjustment status information, the doctor can adjust the light source status of the light source according to the light source adjustment status information, such as adjusting the brightness and color temperature of the light source.

[0061] Optionally, the auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy also includes a display screen, which is connected to the microprocessor and is used to display the state information of the light source adjustment. The doctor can see the state information of the light source adjustment from the display screen.

[0062] In an embodiment of the present invention, judging whether the current light source state is suitable for the patient according to the eye ultrasound image sequence includes:

[0063] Based on the ocular ultrasound image sequence, the movement trajectory of the patient's eye reference point within a set time period is obtained and counted as the eye trajectory. The movement trajectory of the light source within a set time period is obtained and counted as the light source trajectory. The set time period can be set to a time length of 1, 2, 3, 4, or 5 seconds.

[0064] Among them, the movement trajectory of the light source within the set time period is obtained by the light source trajectory recording module, and the light source trajectory recording module can obtain the light source trajectory according to the movement of the light source. Specifically, the light source trajectory recording module may include a processor and a camera, the processor is electrically connected to the camera, and the movement video of the light source is captured by the camera, and the movement video is sent to the processor. The processor has visual tracking technology to obtain the light source trajectory. The light source trajectory can also be obtained by obtaining the movement trajectory of the patient's eye reference point within the set time period based on the eye ultrasound image sequence as described below. The processor is electrically connected to the microprocessor, and the processor sends the light source trajectory to the microprocessor.

[0065] The movement trajectory of the patient's eye reference point within a set time period is obtained based on the eye ultrasound image sequence, which is counted as the eye trajectory, including:

[0066] The middle position of the current eye ultrasound image is obtained, and the middle position is used as the eye reference point; the middle position is the central image coordinate point of the current eye ultrasound image. For example, when the current eye ultrasound image is an N*M matrix, N and M are the length and width of the current eye ultrasound image, respectively, and the values ​​of N and M are positive integers. The horizontal coordinate of the central image coordinate point of the current eye ultrasound image is (N-1) / 2 rounded down, and the vertical coordinate of the central image coordinate point of the current eye ultrasound image is (M-1) / 2 rounded down.

[0067] An eye reference point is identified in each frame of an eye ultrasound image in an eye ultrasound image sequence, and the image coordinates of the eye reference point in each frame of an eye ultrasound image are obtained. The image coordinates of the eye reference point in each frame of an eye ultrasound image are mapped to the current eye ultrasound image to obtain the reference point trajectory coordinates. The coordinate system of each frame of an eye ultrasound image is the same, so the mapping method of mapping the image coordinates of the eye reference point in each frame of an eye ultrasound image to the current eye ultrasound image is: the eye reference point is marked in the same position coordinate point, that is, if the position of the eye reference point detected in a certain frame of an eye ultrasound image is (a1, b1), then the eye reference point is marked at the coordinate point (a1, b1) in the current eye ultrasound image, that is, the reference point trajectory coordinates are (a1, b1). The reference point trajectory coordinates are connected in sequence according to the arrangement order of the eye ultrasound images in the eye ultrasound image sequence to obtain the eye trajectory.

[0068] That is, after the positions of the eye reference points in all eye ultrasound images have been detected and mapped to the current eye ultrasound image, the eye reference points corresponding to the eye ultrasound images will be connected in sequence in the current eye ultrasound image according to the chronological order of the shooting time of the eye ultrasound images to obtain the eye trajectory.

[0069] After obtaining the eyeball trajectory and the light source trajectory, it is determined whether the eyeball trajectory is consistent with the light source trajectory. If the eyeball trajectory is consistent with the light source trajectory, it is determined that the current light source state is suitable for the patient; if the eyeball trajectory is not consistent with the light source trajectory, it is determined that the current light source state is not suitable for the patient.

[0070] If the eyeball trajectory is consistent with the light source trajectory, it means that the patient's eyeball can follow the light source, which means that the patient can quickly adapt to the light source state and can stare at the light source stably, so as to check the eyeball. If the eyeball trajectory is not consistent with the light source trajectory, it means that the patient's eyeball is in the state of the light source, and it is determined that the current light source state is not suitable for the patient. The light source needs to be adjusted so that the patient can stare at the light source stably.

[0071] As a further step, judging whether the eyeball trajectory is consistent with the light source trajectory includes:

[0072] Obtain a difference sequence between the eye trajectory and the light source trajectory. The difference sequence includes multiple trajectory differences, and the trajectory difference is the Euclidean distance between the matching trajectory point pairs in the eye trajectory and the light source trajectory. The matching trajectory point pairs include eye trajectory points and light source trajectory points. The eye trajectory points are nodes of the eye trajectory, and the light source trajectory points are nodes of the light source trajectory. The time nodes corresponding to the matching trajectory point pairs of the eye trajectory points and the light source trajectory points are the same.

[0073] If the difference sequence tends to converge, it is determined that the eye trajectory and the light source trajectory tend to be consistent; if the difference sequence does not tend to converge, it is determined that the eye trajectory and the light source trajectory do not tend to be consistent.

[0074] The difference sequence tends to converge, which can be understood as the fitting curve of the difference sequence moving towards the horizontal axis. As an example, it can be understood that the variance of the difference sequence is getting smaller and smaller.

[0075] Optionally, determining whether the eyeball trajectory is consistent with the light source trajectory includes:

[0076] Map the eyeball trajectory to the first space to obtain the eyeball mapping trajectory; map the light source trajectory to the first space to obtain the light source mapping trajectory. In an embodiment of the present invention, the first space may be in the current eye ultrasound image. Obtain a difference sequence between the eyeball mapping trajectory and the light source mapping trajectory. The difference sequence includes a plurality of trajectory differences, the trajectory difference is the Euclidean distance of the matching trajectory point pairs in the eyeball mapping trajectory and the light source mapping trajectory, the matching trajectory point pairs include eyeball trajectory points and light source trajectory points, the eyeball trajectory points are nodes of the eyeball mapping trajectory, the light source trajectory points are nodes of the light source mapping trajectory, and the matching trajectory point pairs have the same time nodes corresponding to the eyeball trajectory points and the light source trajectory points.

[0077] If the difference sequence tends to converge, it is determined that the eye trajectory and the light source trajectory tend to be consistent. If the difference sequence does not tend to converge, it is determined that the eye trajectory and the light source trajectory do not tend to be consistent.

[0078] In an embodiment of the present invention, obtaining the light source state information according to the eye ultrasound image sequence and the monitoring image sequence includes:

[0079] The eyelid state sequence of the patient is obtained according to the monitoring image sequence. The eyelid state sequence includes multiple eye closure indexes corresponding to multiple frames of eye images, and the eye closure index is used to indicate whether the eye in the eye image is in a closed eye state. In the embodiment of the present method, the eye closure index has a value of 0 or 1. When the eye closure index has a value of 1, it indicates that the eye in the eye image is in a closed eye state, and when the eye closure index has a value of 0, it indicates that the eye in the eye image is in an open eye state.

[0080] The light source state information is obtained according to the eyelid state sequence and the difference sequence. Specifically, the light source state information is obtained according to the eyelid state sequence and the difference sequence, including:

[0081] The segmentation step is determined based on the eyelid state sequence. Specifically: detect the difference between adjacent eye closure indexes in the eyelid state sequence. If the difference between two adjacent eye closure indexes is not 0, count the position of the eye closure index with an earlier time node in the two adjacent eye closure indexes as the mark position; and use the Euclidean distance between the two adjacent mark positions as the segmentation step. In an embodiment of the present invention, the unit of the segmentation step can be pieces, frames, or seconds. The unit represents the number of eye closure indexes, the unit represents the number of images, and the unit represents the time length contained in the number of images. There can be one or more segmentation steps.

[0082] The difference sequence is segmented based on the segmentation step length to obtain a first segmentation sequence, the first segmentation sequence includes a plurality of segmentation difference blocks, and each segmentation difference block includes one or more trajectory differences.

[0083] The sum of all trajectory differences in each segmented difference block is obtained and counted as the first state value; multiple segmented difference blocks correspond to multiple first state values; the eyelid state sequence is segmented based on the segmentation step to obtain a second segmentation sequence, the second segmentation sequence includes multiple segmented eye closure blocks, and each segmented eye closure block includes one or more eye closure indexes.

[0084] The sum of all eye-closing indexes in each segmented eye-closing block is obtained and counted as the second state value; multiple segmented eye-closing blocks correspond to multiple second state values.

[0085] The eyelid state sequence is segmented based on the segmentation step to obtain a second segmentation sequence, specifically: the difference between adjacent eye closure indexes in the eyelid state sequence is detected. If the difference between two adjacent eye closure indexes is not 0, the eyelid state sequence is segmented using the two adjacent eye closure indexes as segmentation boundaries. After segmentation, the two adjacent eye closure indexes are respectively located in two adjacent segmented eye closure blocks.

[0086] That is to say, the difference sequence is segmented according to the corresponding relationship in time order and the corresponding segmentation step, that is, the time nodes corresponding to the trajectory differences of adjacent ends of two adjacent segmented difference blocks after segmentation correspond to the time nodes corresponding to the two adjacent eye closure indexes in the segmented eye closure blocks corresponding in time.

[0087] A first judgment value is obtained based on a plurality of first state values, and a second judgment value is obtained based on a plurality of second state values.

[0088] If the second judgment value is greater than the first threshold and the first judgment value is greater than the second threshold, it is determined that the adjustment of the light source state information is to reduce the brightness in the light source state; if the second judgment value is greater than the first threshold and the first judgment value is less than or equal to the second threshold, it is determined that the adjustment of the light source state information is to increase the color temperature in the light source state.

[0089] If the second judgment value is less than or equal to the first threshold value, and the first judgment value is greater than the second threshold value, it is determined that adjusting the light source state information is to increase the brightness in the light source state.

[0090] Optionally, the value of the first threshold may be 2, 3, 4, or 5. The value of the second threshold may be any real number between 1 and 50, and may optionally be 5, 10, or 15.

[0091] Wherein, obtaining the first judgment value based on the plurality of first state values ​​and obtaining the second judgment value based on the plurality of second state values ​​comprises:

[0092] The plurality of first state values ​​are used as inputs of a first joint prediction network, and the output of the first joint prediction network is used as a first judgment value. The plurality of second state values ​​are used as inputs of a second joint prediction network, and the output of the second joint prediction network is used as a second judgment value.

[0093] In the embodiment of the present invention, the structures of the first joint prediction network and the second joint prediction network are both joint time memory networks, and the parameters of the first joint prediction network and the second joint prediction network obtained for different training data are different. That is, the first joint prediction network is obtained by training the joint time memory network with a training set containing training data with the same data type and data source as the first state value. The second joint prediction network is obtained by training the joint time memory network with a training set containing training data with the same data type and data source as the second state value.

[0094] The joint temporal memory network includes multiple interconnected long short-term memory networks; the output of the previous long short-term memory network is used as the value of the input gate of the next long short-term memory network.

[0095] The Long Short-Term Memory (LSTM) network includes three gating mechanisms, namely the input gate, the output gate, and the forget gate. Under this premise, for example, the joint time memory network includes multiple interconnected long short-term memory networks, which are the first long short-term memory network, the second long short-term memory network, and the third long short-term memory network in order. The first long short-term memory network, the second long short-term memory network, and the third long short-term memory network are connected to each other in sequence.

[0096] The output of the first long short time memory network is used as the input gate of the second long short time memory network, the output of the second long short time memory network is used as the input gate of the third long short time memory network, and the output of the third long short time memory network is used as the output of the joint time memory network.

[0097] When the input data is a plurality of first state values, combined with the above example, it is assumed that the plurality of first state values ​​include the first first state value, the second first state value and the third first state value in order. At this time, the first first state value, the second first state value and the third first state value are respectively used as the input of the first long short time memory network, the second long short time memory network and the third long short time memory network, and then, for the second long short time memory network and the third long short time memory network, the output of the first long short time memory network for the first first state value (counted as the first state output) is used as the input gate of the second long short time memory network. The output of the second long short time memory network (counted as the second state output) is used as the input gate of the third long short time memory network. The output of the third long short time memory network is used as the first judgment value.

[0098] When the input data is a plurality of second state values, combined with the above example, it is assumed that the plurality of second state values ​​include the first second state value, the second second state value and the third second state value in order. At this time, the first second state value, the second second state value and the third second state value are respectively used as the input of the first long short time memory network, the second long short time memory network and the third long short time memory network, and then, for the second long short time memory network and the third long short time memory network, the output of the first long short time memory network for the first second state value (counted as the first state output) is used as the input gate of the second long short time memory network. The output of the second long short time memory network (counted as the second state output) is used as the input gate of the third long short time memory network. The output of the third long short time memory network is used as the second judgment value.

[0099] That is, the structures of the second joint prediction network and the second joint prediction network are that the first long short-term memory network, the second long short-term memory network and the third long short-term memory network are connected to each other in sequence.

[0100] By adopting the above scheme, the influence of the earlier eye state (including the eyeball state and the eyelid state) on the subsequent eye state can be explored. Based on such influence, the accuracy of judging the standardized state of the eyes can be improved, thereby improving the accuracy of judging whether the eyes are adapted to the light source.

[0101] In summary, the auxiliary gaze lamp for ophthalmic ultrasonic biomicroscope examination provided by the embodiment of the present invention can automatically and accurately sense and judge the patient's adaptation to the light source, so that the state of the light source can be adjusted in time, shortening the impact of the uncomfortable light source state on the patient, and improving the efficiency and accuracy of eye examination.

Claims

1. An auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy, characterized in that: The method comprises a microprocessor, wherein the microprocessor is used to execute the following steps of determining the state of a light source: Obtain an eye ultrasound image sequence and a monitoring image sequence of a patient; the eye ultrasound image sequence includes multiple frames of eye ultrasound images, which are arranged in the order of acquisition time; the eye ultrasound images are acquired by an ophthalmic ultrasound biological microscope; the last frame of eye ultrasound image in the eye ultrasound image sequence is the current eye ultrasound image, and the acquisition time point of the current eye ultrasound image is the current time node; the acquisition time point of the previous frame of eye ultrasound image of the current eye ultrasound image is the previous time node; the monitoring image sequence includes multiple frames of eye images, which are arranged in the order of acquisition time; the eye images are acquired by a visible light camera; the last frame of eye image in the monitoring image sequence is the current eye image; the acquisition time point of the current eye image is the current time node; the acquisition time of the previous frame of eye image of the current eye image is the previous time node; Judging whether the current light source state is suitable for the patient according to the eye ultrasound image sequence; the light source state includes brightness and color temperature; If the current light source state is not suitable for the patient, obtain the light source state information according to the eye ultrasound image sequence and the monitoring image sequence; The light source adjustment state information is used to instruct the doctor to adjust the light source state of the light source; The step of judging whether the current light source state is suitable for the patient according to the eye ultrasound image sequence includes: Based on the ocular ultrasound image sequence, the movement trajectory of the patient's eye reference point within a set time period is obtained and counted as the eye trajectory; Obtaining the motion trajectory of the light source within a set time period, which is counted as the light source trajectory; Determine whether the eye trajectory is consistent with the light source trajectory; If the eyeball trajectory tends to be consistent with the light source trajectory, it is determined that the current light source state is suitable for the patient; If the eyeball trajectory is not consistent with the light source trajectory, it is determined that the current light source state is not suitable for the patient; The determining whether the eyeball trajectory is consistent with the light source trajectory includes: Obtain a difference sequence between the eye trajectory and the light source trajectory; the difference sequence includes multiple trajectory differences, the trajectory difference is the Euclidean distance between the matching trajectory point pairs in the eye trajectory and the light source trajectory; the matching trajectory point pairs include eye trajectory points and light source trajectory points, the eye trajectory points are nodes of the eye trajectory, and the light source trajectory points are nodes of the light source trajectory; the time nodes corresponding to the eye trajectory points and the light source trajectory points of the matching trajectory point pairs are the same; If the difference sequence tends to converge, it is determined that the eye trajectory and the light source trajectory tend to be consistent; If the difference sequence does not tend to converge, it is determined that the eye trajectory and the light source trajectory are not consistent; The light source state information is obtained based on the eye ultrasound image sequence and the monitoring image sequence, including: Obtaining the patient's eyelid state sequence according to the monitoring image sequence; the eyelid state sequence includes a plurality of eye closure indexes corresponding to a plurality of frames of eye images; the eye closure index is used to indicate whether the eye in the eye image is in an eye closure state; The light source state information is obtained based on the eyelid state sequence and the difference sequence.

2. The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy according to claim 1, characterized in that: The movement trajectory of the patient's eye reference point within a set time period is obtained based on the eye ultrasound image sequence, which is counted as the eye trajectory, including: Obtaining the middle position of the current eye ultrasound image; The middle position is used as the eye reference point; Identifying an eye reference point in each frame of an eye ultrasound image in an eye ultrasound image sequence, and obtaining image coordinates of the eye reference point in each frame of the eye ultrasound image; Mapping the image coordinates of the eye reference point in each frame of the eye ultrasound image to the current eye ultrasound image to obtain the reference point trajectory coordinates; The reference point trajectory coordinates are connected in sequence according to the arrangement order of the eye ultrasound images in the eye ultrasound image sequence to obtain the eye trajectory.

3. The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy according to claim 1, characterized in that: The light source state information is obtained according to the eyelid state sequence and the difference sequence, including: Determine the segmentation step size based on the eyelid state sequence; Segmenting the difference sequence based on the segmentation step length to obtain a first segmentation sequence, the first segmentation sequence including a plurality of segmentation difference blocks, each segmentation difference block including one or more trajectory differences; Obtain the sum of all trajectory differences in each segmentation difference block, and count it as the first state value; multiple segmentation difference blocks correspond to multiple first state values; Segmenting the eyelid state sequence based on the segmentation step length to obtain a second segmentation sequence, the second segmentation sequence including a plurality of segmented eye-closing blocks, each segmented eye-closing block including one or more eye-closing indexes; Obtaining the sum of all eye-closing indexes in each segmented eye-closing block, and calculating the sum as the second state value; multiple segmented eye-closing blocks correspond to multiple second state values; Obtain a first judgment value based on a plurality of first state values, and obtain a second judgment value based on a plurality of second state values; If the second judgment value is greater than the first threshold value, and the first judgment value is greater than the second threshold value, it is determined that the adjustment of the light source state information is to reduce the brightness in the light source state; if the second judgment value is greater than the first threshold value, and the first judgment value is less than or equal to the second threshold value, it is determined that the adjustment of the light source state information is to increase the color temperature in the light source state; If the second judgment value is less than or equal to the first threshold value, and the first judgment value is greater than the second threshold value, it is determined that adjusting the light source state information is to increase the brightness in the light source state.

4. The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy according to claim 3, characterized in that: Obtaining a first judgment value based on a plurality of first state values, and obtaining a second judgment value based on a plurality of second state values, comprises: Using the plurality of first state values ​​as inputs of a first joint prediction network, and using the output of the first joint prediction network as a first judgment value; Using the plurality of second state values ​​as inputs of a second joint prediction network and using the second joint prediction network as output of a second judgment value; The structures of the first joint prediction network and the second joint prediction network are both joint time memory networks. For different training data, the parameters of the first joint prediction network and the second joint prediction network are different. The joint temporal memory network includes multiple interconnected long short-term memory networks; the output of the previous long short-term memory network is used as the value of the input gate of the next long short-term memory network.

5. The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy according to claim 1, characterized in that: The value of the eye closure index is 0 or 1. When the eye closure index is 1, it indicates that the eye in the eye image is in a closed state. When the eye closure index is 0, it indicates that the eye in the eye image is in an open state.

6. The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy according to claim 1, characterized in that: The microprocessor is electrically connected to an ophthalmic ultrasound biomicroscope and a visible light camera, and the ophthalmic ultrasound biomicroscope is used to collect an eye ultrasound image sequence of a patient and send the eye ultrasound image sequence to the microprocessor; The visible light camera is used to collect monitoring image sequences of the patient and send the monitoring image sequences to the microprocessor.

7. The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy according to claim 1, characterized in that: The auxiliary gaze lamp for ophthalmic ultrasonic biomicroscopy also includes a light source.

Images