Automatic safety protection blue light therapy device

Abstract

The present invention discloses an automatic safety protection blue light therapy device, which includes a blue light therapy instrument main body and a compatible external component. It is characterized in that the compatible external component is provided with a compatible connection interface for detachably connecting with the blue light therapy instrument to realize the linkage control and power supply of blue light output. The compatible external component includes: a visual acquisition module for real-time acquisition of an image sequence of a newborn's head and eye area; an image recognition module configured to analyze the image sequence based on a pre-trained眼罩识别模型 (eyeshield recognition model) to identify the wearing state of the newborn's light therapy eyeshield; a control and alarm module signal-connected to the image recognition module and the blue light output unit of the blue light therapy instrument main body, and configured to automatically cut off the power supply of the blue light output unit and trigger an alarm when an abnormal eyeshield wearing is recognized. Ensure that the light therapy irradiation is immediately stopped when the eyeshield wearing is abnormal to prevent damage to the newborn's eyes caused by blue light. It should be noted that there is an unclear term "眼罩识别模型" in the original text which is directly retained in the translation. You may need to replace it with the correct English term if available.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology, specifically relating to an automatic safety protection blue light phototherapy device. Background Technology

[0002] Neonatal jaundice is the most common clinical problem in the neonatal period. Blue light therapy is currently the most commonly used and effective method for treating neonatal jaundice. Its principle is to promote the metabolism and excretion of bilirubin in the body by irradiating the newborn's skin with blue light, thereby reducing serum bilirubin levels and relieving jaundice symptoms.

[0003] To prevent damage to the delicate eyes of newborns, existing neonatal blue light therapy devices require newborns to wear special phototherapy goggles to cover their eyes. However, in actual clinical applications, newborns are active, turning over, or undergoing care procedures, which can easily cause the goggles to loosen or fall off. Since medical staff cannot provide 24-hour real-time monitoring, if the goggles fall off and are not detected in time, the blue light will directly shine into the newborn's eyes. Long-term or high-intensity exposure can lead to serious complications such as retinal damage and vision loss in newborns. This is the biggest safety hazard of existing blue light therapy equipment.

[0004] Currently, there is no effective automated solution to address the aforementioned safety hazards. Most hospitals can only reduce the risk by increasing the frequency of patrols by medical staff. However, this approach not only increases the workload of medical staff but also fails to fundamentally eliminate the hazards. At the same time, most existing blue light therapy devices with safety protection functions are integrated designs that cannot be adapted to the hospital's existing older blue light therapy devices. Upgrading all equipment in the hospital would require replacing the entire device, which is extremely costly and puts a significant financial burden on the hospital.

[0005] Therefore, developing a blue light therapy device that can automatically recognize the wearing status of the eye mask, trigger safety protection in a timely manner, and be compatible with older equipment and reduce upgrade costs has become an urgent technical problem to be solved in the field of neonatal medical equipment. Summary of the Invention

[0006] In view of this, the present invention proposes an automatic safety protection blue light phototherapy device. The present invention can ensure that phototherapy is stopped immediately when the eye mask is worn abnormally, so as to prevent blue light from damaging the eyes of newborns.

[0007] To achieve the above objectives, the present invention provides the following technical solution: This invention provides an automatic safety protection blue light therapy device, comprising a blue light therapy instrument body and an adapter external component. The adapter external component is provided with an adapter connection interface for detachable connection with the blue light therapy instrument, realizing linkage control and power supply for blue light output. The adapter external component includes: The visual acquisition module is used to acquire image sequences of the newborn's head and eye area in real time; The image recognition module is configured to analyze image sequences based on a pre-trained eye mask recognition model to identify the wearing status of phototherapy eye masks in newborns; The control alarm module is connected to the image recognition module and the blue light output unit of the main body of the blue light therapy device. It is configured to automatically cut off the power to the blue light output unit and trigger an alarm when abnormal wearing of the eye mask is detected.

[0008] Preferably, the image recognition module includes: The facial key point localization unit is used to locate multiple observable feature points on the face of newborns while they are wearing phototherapy eye masks. The feature points include the root of the nose, the tip of the nose, the alar of the nose, the corner of the mouth, the base of the auricle, and the angle of the mandible. The eye mask region segmentation unit is used to perform pixel-level segmentation of the acquired image and extract the precise mask of the area where the phototherapy eye mask is located; The relative position analysis unit is used to construct an eye coverage probability field based on the geometric relationship between facial key points and the eye mask area mask, and calculate the completeness of eye mask coverage on the eyes based on the probability field to infer whether the eye mask effectively covers the eye area. The eye mask edge fit detection unit is used to analyze the contact state between the eye mask edge and facial skin to determine whether there are gaps that could cause blue light leakage. The temporal state evaluation unit is used to fuse the analysis results of multiple consecutive frames of images, output the blindfold wearing status and its confidence level, and predict the trend of state change.

[0009] Preferably, the facial key point localization unit includes a deep convolutional neural network model, a shape constraint subunit, and an iterative optimization subunit; The deep convolutional neural network model adopts a network structure based on heatmap regression and is trained on a large number of newborn facial images with labeled facial key points. During the training process, a random occlusion enhancement strategy is adopted, which simulates the occlusion of the eye area by randomly generating occlusion blocks on the image, so that the network can learn to infer the location of the occluded key points from the context information of the unoccluded area of ​​the face. The shape constraint subunit is connected to the output layer of the neural network model and is used to correct the key points of the initial prediction. This subunit has a pre-stored facial shape prior model established by principal component analysis. It projects the key point coordinates output by the network onto the shape prior space and limits the range of variation of the shape parameters so that the output key points conform to the natural geometric distribution of the face. The iterative optimization sub-unit is based on the topological relationship constraints between key points. It fine-tunes the corrected key point positions through the energy minimization method. The energy function includes a data term and a prior term. The data term represents the consistency with the network output, and the prior term represents the degree of deviation from the shape prior. After iterative convergence, the final key point coordinates are output.

[0010] Preferably, the eye patch region segmentation unit employs a semantic segmentation network with an encoder-decoder structure, wherein: The encoder section introduces hollow spatial pyramid pooling to extract image features using multi-scale receptive fields; The decoder section fuses shallow detail information from each layer of the encoder via skip connections to restore spatial resolution; The network output is followed by a conditional random field as post-processing to optimize the segmentation boundary and enhance edge continuity; The network was trained on a large number of images covering different styles, colors and lighting conditions of goggles. During training, a joint loss function was used, which is a weighted sum of cross-entropy loss and Dice loss. Pixels in the edge region of the goggles were given higher weights to improve segmentation accuracy and boundary localization accuracy.

[0011] Preferably, the relative position analysis unit includes a coordinate system transformation subunit, an eye prior distribution modeling subunit, and a coverage integrity calculation subunit; The coordinate system transformation subunit is used to estimate head pose parameters based on the detected facial key points and correct the image to a standardized frontal pose through affine transformation to eliminate the influence of head pose changes. The eye prior distribution modeling subunit pre-stores a probability distribution model of eye position obtained through statistical analysis of large-sample clinical data. This includes images of newborns of different gestational ages and weights wearing phototherapy eye masks in various positions. A large sample of images was collected to document correct wearing, incorrect wearing, and eye mask dislodgement. Considering the variety of clinical eye masks, images of wearing various commonly used clinical eye masks were collected to improve recognition efficiency and accuracy. The model is represented in Gaussian mixture model form to describe the possible spatial distribution region of the eye in a standardized coordinate system. The coverage integrity calculation subunit is used to transform the probability distribution of the eyes in the standardized coordinate system back to the original image coordinates, and perform pixel-by-pixel spatial overlap analysis with the eye mask segmentation mask to calculate the coverage integrity index of the eye mask on the eyes. When the coverage integrity is lower than the preset threshold, it is determined that the eye mask is worn abnormally.

[0012] Preferably, the eye mask edge fit detection unit includes: An edge extraction subunit is used to extract the precise edge contour of the eye mask from the eye mask segmentation mask; The local 3D reconstruction subunit is used to reconstruct the local 3D surface of the edge area of ​​the goggles using multiple frames of images illuminated by multiple angle light sources and photometric stereo technology, thereby obtaining the relative height information between the edge of the goggles and the skin. The gap quantization subunit is used to sample along the normal direction of the eye mask edge, calculate the height difference and gray-scale gradient change between the eye mask and the skin, and construct an edge fit score. When the score is lower than the threshold, it is judged as loose.

[0013] Preferably, the gap quantization subunit performs equal-interval sampling along the normal direction of the eye mask edge contour, and at each sampling point extends a preset pixel distance towards the inner side of the eye mask and the outer side of the skin, respectively, to extract the pixel sequence on the normal path; Based on the pixel sequence, feature parameters for each sampling point are calculated, and the feature parameters include: Height difference feature: The height difference between the edge point of the eye patch and the adjacent skin point is obtained through local 3D reconstruction; Gray-level gradient features: Calculate the gradient rate of change of pixel gray level on the normal path, reflecting the edge sharpness at the junction of the eye mask and the skin; Texture consistency feature: Compare the texture similarity of regions on both sides of the normal path to reflect the characteristics of the transition area between the eye mask and the skin; The feature parameters are input into a pre-trained fit scoring model, which uses support vector regression or random forest regression algorithms to output a local fit score for each sampling point, with a value ranging from 0 to 1. The local fit scores of all sampling points are weighted and averaged. The weights are determined based on the eye coverage probability field value at the location of the sampling point. Areas with higher eye coverage probability are given higher weights to obtain the overall edge fit score of the eye mask. When the overall edge fit score is lower than the preset looseness threshold, the eye mask is determined to be in a loose state.

[0014] Preferably, the training process of the fit scoring model includes: A large number of clinical image samples were collected in advance, and experts marked the degree of fit at various positions on the edge of the eye patch in the images. The fit levels were marked into three categories: tight fit, slight lifting, and obvious gap. The feature parameters of each sampling point are extracted from the sample as input, and the fitting level labeled by experts is used as the output label to train the fitting score model. During training, cross-validation is used to optimize model parameters so that the model output score and expert annotations meet the preset accuracy requirements.

[0015] The present invention has achieved at least the following beneficial effects: 1. By working in tandem with the visual recognition module and the image recognition module, the wearing status of the eye mask can be automatically identified in real time 24 hours a day without the need for real-time monitoring by medical staff. Once the mask becomes loose or falls off, the blue light will be cut off immediately and an alarm will be triggered, fundamentally avoiding damage to the newborn's eyes caused by blue light exposure and solving the core safety hazard of existing equipment.

[0016] 2. Adopting a dual design of "integrated + external", it can be integrated into new blue light phototherapy devices or added to older devices through external components, without the need to replace the entire device, which greatly reduces the equipment upgrade cost for hospitals and facilitates large-scale promotion and application, especially suitable for primary hospitals.

[0017] 3. The image recognition model has been trained with a large number of samples, resulting in high recognition accuracy and adaptability to different scenarios; the control alarm module has a short response time and can trigger protective actions at the first moment of abnormality of the eye mask, minimizing the potential harm of blue light to the newborn's eyes.

[0018] 4. The equipment is highly automated, requiring no complex operations from medical personnel; only confirmation of the goggle's wearing status is needed during reset. The camera uses a high-definition infrared design, unaffected by blue light interference, and is suitable for various neonatal blue light therapy scenarios. Furthermore, the image recognition model can be upgraded online to adapt to various new phototherapy goggles currently on the market, extending the equipment's lifespan. Phototherapy irradiation time and intensity can be preset as needed.

[0019] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0020] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a schematic diagram of the structure of an automatic safety protection blue light phototherapy device according to an embodiment of the present invention. Detailed Implementation

[0021] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0022] To achieve the above objectives, the present invention provides the following technical solution: This invention provides an automatic safety protection blue light phototherapy device, referring to... Figure 1It includes the main body of the blue light therapy device and an adapter external component. The adapter external component has an adapter connection interface for detachable connection with the blue light therapy device, realizing linkage control and power supply for blue light output. The adapter external component includes: The visual acquisition module is used to acquire image sequences of the newborn's head and eye area in real time; The image recognition module is configured to analyze image sequences based on a pre-trained eye mask recognition model to identify the wearing status of phototherapy eye masks in newborns; The control alarm module is connected to the image recognition module and the blue light output unit of the main body of the blue light therapy device. It is configured to automatically cut off the power to the blue light output unit and trigger an alarm when abnormal wearing of the eye mask is detected.

[0023] In this embodiment, the main body of the blue light phototherapy device is a conventional neonatal jaundice treatment device, typically including a blue light source (such as an LED array), control circuitry, and a power supply. An adapter external component is detachably connected to the blue light phototherapy device via an adapter interface (e.g., a standard USB-C interface or a dedicated multi-pin connector) to achieve electrical and signal linkage control. This interface not only powers the external component but also transmits control signals, enabling the external component to monitor and control the blue light output in real time. The visual acquisition module uses a high-resolution, low-light CMOS camera with a resolution of at least 1920×1080 pixels and a frame rate of at least 30fps, and is equipped with an infrared supplemental light to ensure clear imaging at night or in low-light environments. The camera is mounted on a bedside stand or directly into the adapter external component box, and its angle is adjustable to cover the entire facial area. The image recognition module can be deployed on an embedded processor (such as an NVIDIA Jetson series or a Raspberry Pi Compute Module) built into the external component, or it can transmit image data to a cloud server for analysis via a wireless network. The control alarm module includes a solid-state relay or MOSFET switch to quickly cut off the power supply to the blue light output unit and trigger an audible and visual alarm (such as a buzzer and a red LED indicator) to ensure that phototherapy is stopped immediately if the eye mask is worn abnormally, thus preventing blue light from damaging the newborn's eyes.

[0024] In a preferred embodiment, the image recognition module includes: The facial key point localization unit is used to locate multiple observable feature points on the face of newborns while they are wearing phototherapy eye masks. The feature points include the root of the nose, the tip of the nose, the alar of the nose, the corner of the mouth, the base of the auricle, and the angle of the mandible. The eye mask region segmentation unit is used to perform pixel-level segmentation on the acquired image, extract the precise mask of the area where the phototherapy eye mask is located, and identify whether the eye mask position is correct under various body positions.

[0025] The relative position analysis unit is used to construct an eye coverage probability field based on the geometric relationship between facial key points and the eye mask area mask, and calculate the completeness of eye mask coverage on the eyes based on the probability field to infer whether the eye mask effectively covers the eye area. The eye mask edge fit detection unit is used to analyze the contact state between the eye mask edge and facial skin to determine whether there are gaps that could cause blue light leakage. The temporal state evaluation unit is used to fuse the analysis results of multiple consecutive frames of images, output the blindfold wearing status and its confidence level, and predict the trend of state change.

[0026] In this embodiment, the facial key point localization unit employs a deep convolutional neural network. Even when the eye mask partially covers an area (e.g., the eyes are completely covered), it can still infer the location of obscured key points based on features of unobscured areas such as the root of the nose, the tip of the nose, and the auricle. The eye mask region segmentation unit uses a semantic segmentation network to generate a precise pixel mask for the eye mask. This mask is used not only for coverage analysis but also provides a basis for edge fit detection. The relative position analysis unit constructs a probability field to represent the possible areas around the eyes by registering key points with the mask and calculates the overlap between the eye mask mask and this probability field, thereby quantifying the degree of eye mask obstruction. The eye mask edge fit detection unit further analyzes the tightness of contact between the eye mask and the skin to detect the presence of tiny gaps that may lead to blue light leakage. The temporal state evaluation unit combines the analysis results of multiple consecutive frames of images, such as through Kalman filtering or hidden Markov models, to smooth and predict the wearing state, reduce false detections in a single frame, and provide early warnings when the eye mask gradually slips off.

[0027] In a preferred embodiment, the facial key point localization unit includes a deep convolutional neural network model, a shape constraint subunit, and an iterative optimization subunit; The deep convolutional neural network model adopts a network structure based on heatmap regression and is trained on a large number of newborn facial images with labeled facial key points. During the training process, a random occlusion enhancement strategy is adopted, which simulates the occlusion of the eye area by randomly generating occlusion blocks on the image, so that the network can learn to infer the location of the occluded key points from the context information of the unoccluded area of ​​the face. The shape constraint subunit is connected to the output layer of the neural network model and is used to correct the key points of the initial prediction. This subunit has a pre-stored facial shape prior model established by principal component analysis. It projects the key point coordinates output by the network onto the shape prior space and limits the range of variation of the shape parameters so that the output key points conform to the natural geometric distribution of the face. The iterative optimization sub-unit is based on the topological relationship constraints between key points. It fine-tunes the corrected key point positions through the energy minimization method. The energy function includes a data term and a prior term. The data term represents the consistency with the network output, and the prior term represents the degree of deviation from the shape prior. After iterative convergence, the final key point coordinates are output.

[0028] In this embodiment, the heatmap regression network adopts an architecture similar to StackedHourglass or HRNet, outputting a two-dimensional Gaussian heatmap corresponding to each keypoint. The peak value of the heatmap represents the keypoint coordinates. The training dataset contains over 100,000 images of newborn faces, each manually labeled with 15 facial keypoints (including the aforementioned feature points). A random occlusion enhancement strategy randomly generates black rectangular blocks of varying sizes in the eye region of the image during training, simulating eye patch occlusion. This forces the network to infer the location of eye keypoints using contextual information from areas such as the bridge of the nose and cheeks, thereby improving the model's robustness in real-world occlusion scenarios. The shape constraint subunit reduces the dimensionality of the keypoint coordinates in the training set through Principal Component Analysis (PCA), obtaining shape parameters and principal component feature vectors. The keypoint coordinates output by the network are first projected into the PCA space, then the shape parameters are constrained to within the mean ± 3 standard deviations, and finally backprojected into the image space. This process removes obviously abnormal outliers. The iterative optimization subunit uses the energy function E = E_data + λE_prior, where E_data is the sum of squared Euclidean distances between the current keypoint and the peak value of the network's output heatmap, E_prior is the Mahalanobis distance between the shape parameter and the mean, and λ is the weighting coefficient (usually set to 0.1). Iterative optimization is performed using gradient descent until convergence. After this process, the keypoint localization accuracy can reach the sub-pixel level, providing a reliable foundation for subsequent analysis.

[0029] In a preferred embodiment, the eye patch region segmentation unit employs a semantic segmentation network with an encoder-decoder structure, wherein: The encoder section introduces hollow spatial pyramid pooling to extract image features using multi-scale receptive fields; The decoder section fuses shallow detail information from each layer of the encoder via skip connections to restore spatial resolution; The network output is followed by a conditional random field as post-processing to optimize the segmentation boundary and enhance edge continuity; The network was trained on a large number of images covering different styles, colors and lighting conditions of goggles. During training, a joint loss function was used, which is a weighted sum of cross-entropy loss and Dice loss. Pixels in the edge region of the goggles were given higher weights to improve segmentation accuracy and boundary localization accuracy.

[0030] In this embodiment, the semantic segmentation network uses a variant of DeepLabV3+ or U-Net. The encoder uses ResNet-50 as the backbone network and introduces a dilated spatial pyramid pooling (ASPP) module. This module contains four parallel branches, employing dilated convolutions with dilation rates of 1, 6, 12, and 18, respectively, and a global average pooling branch, thereby capturing contextual information at different scales. The decoder fuses shallow features from the encoder (such as the first and second layer outputs of ResNet) with ASPP output features through skip connections to recover high-resolution details. The network output is a probability map of each pixel belonging to the eye mask category, with a threshold of 0.5 to obtain a binary mask. Subsequently, a Conditional Random Field (CRF) is applied as post-processing. CRF considers the color similarity and spatial proximity between pixels to fine-tune the probability map, making the segmentation boundary more closely match the actual edge of the eye mask. The training dataset contains 50,000 images of newborns wearing eye masks, covering eye masks of different brands, colors (blue, green, pink, etc.), materials (non-woven fabric, silicone), and different lighting conditions (daytime, nighttime, supplemental lighting). The joint loss function is L = αL_ce + βL_dice, where L_ce is the pixel-level cross-entropy loss and L_dice is the Dice loss (1 - Dice coefficient). α and β are typically set to a 1:1 ratio. Furthermore, pixels in the eye patch edge region (the boundary band extracted through morphological dilation) are additionally multiplied by a weight factor of 2 in the cross-entropy loss to enhance edge learning.

[0031] In a preferred embodiment, the relative position analysis unit includes a coordinate system transformation subunit, an eye prior distribution modeling subunit, and a coverage integrity calculation subunit; The coordinate system transformation subunit is used to estimate head pose parameters based on the detected facial key points and correct the image to a standardized frontal pose through affine transformation to eliminate the influence of head pose changes. The eye prior distribution modeling subunit pre-stores a probability distribution model of eye position obtained through statistical analysis of large-sample clinical data. This includes images of newborns of different gestational ages and weights wearing phototherapy eye masks in various positions. A large sample of images was collected to document correct wearing, incorrect wearing, and eye mask dislodgement. Considering the variety of clinical eye masks, images of wearing various commonly used clinical eye masks were collected to improve recognition efficiency and accuracy. The model is represented in Gaussian mixture model form to describe the possible spatial distribution region of the eye in a standardized coordinate system. The coverage integrity calculation subunit is used to transform the probability distribution of the eyes in the standardized coordinate system back to the original image coordinates, and perform pixel-by-pixel spatial overlap analysis with the eye mask segmentation mask to calculate the coverage integrity index of the eye mask on the eyes. When the coverage integrity is lower than the preset threshold, it is determined that the eye mask is worn abnormally.

[0032] In this embodiment, the coordinate system transformation subunit first estimates the rotation and translation parameters of the head pose using facial key points (such as the root of the nose, the tip of the nose, and the base points of the left and right auricles). Specifically, the transformation matrix from the camera coordinate system to the world coordinate system is solved using the PnP algorithm, and then an affine transformation is applied to map the original image to a standard frontal pose (eyes horizontal, face center aligned). The transformed image size is normalized to 256×256 pixels. The eye prior distribution modeling subunit, based on 5000 neonatal clinical images, manually labels the left and right eye regions (including eyelids and eyeballs) in a standardized coordinate system, counts the frequency of each pixel belonging to the eye, and constructs a two-dimensional probability map. Due to individual differences, the eye positions exhibit a certain distribution. A Gaussian mixture model (GMM, with a mixture number of 2, corresponding to the left and right eyes respectively) is used to fit the probability map, obtaining the mean and covariance matrices, which are stored as a prior model. The coverage completeness calculation subunit maps the eye probability distribution in the standard coordinate system back to the original image coordinates through an inverse transformation, obtaining the probability value of each original pixel belonging to the eye. Then, a mask is created to separate it from the eye mask. (Values ​​are 0 or 1) Perform pixel-by-pixel multiplication to calculate coverage completeness:

[0033] That is, the ratio of the sum of the eye probabilities covered by the eye mask to the total eye probability. The value range is from 0 to 1. This means the eye mask completely covers all possible eye areas. This indicates no coverage at all. (Preset threshold) Set to 0.95, when This threshold is considered an abnormality in eye patch wearing (insufficient coverage). This threshold can be adjusted according to clinical requirements; for example, it can be set to 0.98 under stricter safety standards.

[0034] In a preferred embodiment, the eye mask edge fit detection unit includes: An edge extraction subunit is used to extract the precise edge contour of the eye mask from the eye mask segmentation mask; The local 3D reconstruction subunit is used to reconstruct the local 3D surface of the edge area of ​​the goggles using multiple frames of images illuminated by multiple angle light sources and photometric stereo technology, thereby obtaining the relative height information between the edge of the goggles and the skin. The gap quantization subunit is used to sample along the normal direction of the eye mask edge, calculate the height difference and gray-scale gradient change between the eye mask and the skin, and construct an edge fit score. When the score is lower than the threshold, it is judged as loose.

[0035] In this embodiment, the edge extraction subunit extracts the contour directly from the segmentation mask using the Canny edge detection algorithm or by extracting the contour, obtaining the closed curve of the goggle edge, represented as an ordered set of pixels. The local 3D reconstruction subunit utilizes multiple LED light sources (e.g., one on each side and one above) surrounding the camera on an external component to sequentially illuminate light sources in different directions and simultaneously acquire images, obtaining multiple frames of the same scene under different lighting directions. Based on photometric stereo technology, according to the reflection model of the surface normal and the lighting direction (assuming Lambertian reflection), the surface normal vector of each pixel is calculated, and then the local relative height map is reconstructed through integration. The reconstruction area is concentrated in a 10-pixel-wide strip around the edge of the goggle. The height map can quantify the height difference Δh between the edge of the goggle and the adjacent skin. The gap quantization subunit performs dense sampling along the edge normal direction, with a sampling interval of 1 pixel. At each sampling point, feature parameters are calculated, including height difference, grayscale gradient, texture consistency, etc., and then input into a machine learning model to obtain a local fit score. Finally, a weighted average is used to obtain the overall score, and if it is lower than a threshold (e.g., 0.8), it is judged as loose.

[0036] In a preferred embodiment, the gap quantization subunit performs equal-interval sampling along the normal direction of the eye mask edge contour, and extends a preset pixel distance at each sampling point in the direction of the inner side of the eye mask and the outer side of the skin, respectively, to extract the pixel sequence on the normal path; Equal-interval sampling is performed along the normal direction of the eye mask, with an interval of 2-5 pixels between adjacent sampling points. At each sampling point, a preset pixel distance is extended towards the inside of the eye mask and the outside of the skin, respectively, and the pixel sequence on the normal path is extracted. Based on the pixel sequence, feature parameters for each sampling point are calculated, and the feature parameters include: Height difference feature: The height value of the goggles at the sampling point is obtained by reconstructing the three-dimensional surface using photometric stereo technology. Height values ​​of adjacent skin areas Calculate the height difference ; Gray-level gradient characteristics: Calculate the gray-level change curve along the normal direction and find the maximum gradient value. and gradient rate of change ; Texture consistency feature: Calculate the texture consistency on both sides of the normal. The local binary pattern histogram of a pixel region measures the similarity of textures on both sides using Bach distance. ; The feature parameters are mapped to a local fit score using the Sigmoid function. :

[0037] in, For the Sigmoid function, , , For feature weights, For the maximum permissible height difference, For gradient threshold, , For adjustment coefficients, The value ranges from 0 to 1; The local fit scores of all sampling points are weighted and averaged, with each sampling point having a weight. The probability value of eye coverage based on its location Sure:

[0038] Calculate the overall score of edge fit :

[0039] when When, it is judged as a normal fit; when When it is determined to be slightly loose, an alert is triggered; when When the system detects a significant looseness, it triggers an alarm and cuts off the blue light. This is the normal fit threshold. This is the loosening threshold.

[0040] In this embodiment, the normal direction sampling extends 10 pixels inward (towards the eye patch side) and 10 pixels outward (towards the skin side) from the edge point, forming a path of 21 pixels in length. The height difference feature is the maximum height difference within ±2 pixels from the edge point on the path. The grayscale gradient feature calculates the maximum value of the first derivative of grayscale on the path, reflecting the sharpness of the edge; the gradient is usually gentle at the gaps. The texture consistency feature calculates the chi-square distance between the local binary pattern (LBP) histograms of 5×5 pixel blocks on both sides of the path; the smaller the distance, the more similar the textures, and the larger the texture difference between the two sides at the gaps. The fit rating model uses random forest regression, and the training data comes from 5000 expert-annotated sampling points, with annotation levels of tight fit (score 1.0), slight lifting (0.5), and obvious gap (0.0). Five-fold cross-validation is used during model training, and the final mean absolute error on the test set is less than 0.1. When calculating the overall score, the weights are determined by the eye coverage probability field, meaning that points closer to the eye have a higher weight (e.g., areas with a probability field value greater than 0.5 have a weight of 2, while the rest have a weight of 1), ensuring that the fit near the eye contributes more to the overall score. The loosening threshold is set to 0.7, and an alarm is triggered when the overall score falls below 0.7.

[0041] In a preferred embodiment, the temporal state assessment unit uses a Kalman filter to fuse the coverage integrity and edge fit scores of multiple consecutive frames to estimate the current state and predict future trends; at the same time, it uses a multi-frame voting mechanism to improve confidence.

[0042] In this embodiment, the temporal state evaluation unit receives the coverage integrity C_t output from the relative position analysis unit and the comprehensive score S_t (t being the frame number) output from the eye patch edge fit detection unit. Two independent Kalman filters are used to smooth C_t and S_t respectively, with the filter state including the current value and rate of change. The observation noise covariance is adaptively adjusted according to image quality (e.g., increasing noise in low light). Based on the filtered estimated values ​​C_t and S_t, the wearing status is comprehensively determined: if C_t < 0.95 or S_t < 0.7, it is considered abnormal; otherwise, it is considered normal. Simultaneously, the filter is used to predict the value of the next frame; if the predicted value is about to fall below a threshold, an early warning is issued. Furthermore, a multi-frame voting mechanism is employed: if 4 out of 5 consecutive frames are considered abnormal, the final abnormal status is output to avoid false detections in a single frame. The confidence level is determined by the trace of the covariance matrix of the Kalman filter; the smaller the covariance, the higher the confidence level.

[0043] In a preferred embodiment, the control alarm module includes a microcontroller, a relay drive circuit, an audible and visual alarm, and a communication interface. The microcontroller receives the judgment result output by the image recognition module. When an abnormal signal is received, it immediately cuts off the power supply to the blue light output unit through the relay and simultaneously activates the audible and visual alarm. When a signal indicating that the system has returned to normal is received, the power supply is reconnected after a 5-second delay to avoid frequent on / off cycles.

[0044] In this embodiment, the microcontroller uses an STM32 series low-power chip and communicates with the image recognition module via UART or I2C. Solid-state relays are used, with a response time of less than 10ms, ensuring rapid power cut-off. The audible and visual alarm includes a high-decibel buzzer (>85dB) and a high-brightness red LED, continuously sounding the alarm until manually confirmed. The communication interface can be either Wi-Fi or Bluetooth, used to send alarm information to the nurse station or a parent's mobile app. A 5-second delay before reconnecting power is to prevent repeated on / off switching due to temporary loosening of the goggles (such as a newborn turning over), and to ensure the goggles are securely worn before re-irradiation.

[0045] In a preferred embodiment, the adaptable external component also includes a data storage module for locally caching image sequences and analysis results, and uploading them to the cloud when the network is available for doctors to review later.

[0046] In this embodiment, the data storage module uses an eMMC storage chip with a capacity of no less than 32GB, capable of cyclically storing monitoring data from the most recent 72 hours. Image data is stored after compression (H.264 encoding), while also saving the timestamp and status tag of the analysis results. When the network is available, data is uploaded to a cloud server via the MQTT protocol. Doctors can view historical videos and abnormal events through a web interface, facilitating clinical analysis and improvement of treatment plans.

[0047] In a preferred embodiment, the units in the image recognition module can be processed in parallel, and a pipelined architecture is adopted to improve the processing speed and ensure that the real-time performance is within 30ms per frame.

[0048] In this embodiment, a multi-threaded pipeline design is adopted: thread 1 is responsible for image acquisition and preprocessing, thread 2 runs facial key point localization and eye mask segmentation (which can be parallelized on two GPU cores), thread 3 performs relative position analysis and edge fit detection, and thread 4 performs temporal fusion and decision-making. The overall processing time is controlled within 25ms per frame, meeting the requirements of real-time monitoring.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0050] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. An automatic safety protection blue light phototherapy device, comprising a blue light phototherapy instrument body and an adaptive external component, characterized in that, The adaptable external component is provided with an adaptable connection interface for detachably connecting with a blue light phototherapy device to achieve the linkage control and power supply of blue light output. The adaptable external component includes: A visual acquisition module for real-time acquisition of an image sequence of a neonate's head and eye area; An image recognition module configured to analyze the image sequence based on a pre-trained eye mask recognition model to identify the wearing state of the neonate's phototherapy eye mask; A control and alarm module, which is signal-connected to the image recognition module and the blue light output unit of the main body of the blue light phototherapy device, and is configured to automatically cut off the power supply of the blue light output unit and trigger an alarm when an abnormal wearing of the eye mask is recognized.

2. The automatic safety protection blue light phototherapy device according to claim 1, characterized in that, The image recognition module includes: A facial key point localization unit for locating multiple observable feature points on the face of a neonate in the state of wearing a phototherapy eye mask. The feature points include the nasion, the tip of the nose, the alar points, the oral commissure points, the auricular base points and the mandibular angle points; An eye mask area segmentation unit for pixel-level segmentation of the acquired image to extract an accurate mask of the area where the phototherapy eye mask is located; A relative position analysis unit for constructing an eye coverage probability field based on the geometric relationship between the facial key points and the eye mask area mask, and calculating the coverage integrity of the eye mask for the eyes based on this probability field to infer whether the eye mask effectively blocks the eye area; An eye mask edge fitting degree detection unit for analyzing the contact state between the edge of the eye mask and the facial skin to determine whether there is a gap causing blue light leakage; A timing state evaluation unit for fusing the analysis results of multiple consecutive frames of images, outputting the wearing state of the eye mask and its confidence level, and predicting the state change trend.

3. The automatic safety protection blue light phototherapy device according to claim 2, characterized in that, The facial key point localization unit includes a deep convolutional neural network model, a shape constraint sub-unit and an iterative optimization sub-unit; The deep convolutional neural network model adopts a network structure based on heat map regression and is trained with a large number of neonatal facial images marked with facial key points; During the training process, a random occlusion augmentation strategy is adopted. By randomly generating occlusion blocks on the image to simulate the occlusion of the eye area by the eye mask, the network is made to learn to infer the positions of the occluded key points from the context information of the unoccluded areas of the face; The shape constraint sub-unit is connected to the output layer of the neural network model and is used to correct the initially predicted key points. The sub-unit pre-stores a facial shape prior model established through principal component analysis, projects the key point coordinates output by the network into the shape prior space, and makes the output key points conform to the natural geometric distribution of the human face by restricting the change range of the shape parameters; The iterative optimization sub-unit fine-tunes the positions of the corrected key points through an energy minimization method based on the topological relationship constraint between the key points. The energy function includes a data term and a prior term. The data term represents the consistency with the network output, and the prior term represents the degree of deviation from the shape prior. After iterative convergence, the final key point coordinates are output.

4. The automatic safety protection blue light phototherapy device according to claim 2, characterized in that, The eye mask area segmentation unit adopts a semantic segmentation network with an encoder-decoder structure, where: The encoder part introduces dilated spatial pyramid pooling to extract image features with multi-scale receptive fields; The decoder part restores the spatial resolution by fusing the shallow detail information of each layer of the encoder through skip connections; The network output is followed by a conditional random field as post-processing to optimize the segmentation boundary and enhance edge continuity; The network was trained on a large number of images covering different styles, colors and lighting conditions of goggles. During training, a joint loss function was used, which is a weighted sum of cross-entropy loss and Dice loss. Pixels in the edge region of the goggles were given higher weights to improve segmentation accuracy and boundary localization accuracy.

5. The automatic safety protection blue light phototherapy device according to claim 2, characterized in that, The relative position analysis unit includes a coordinate system transformation sub-unit, an eye prior distribution modeling sub-unit, and a coverage integrity calculation sub-unit; The coordinate system transformation subunit is used to estimate head pose parameters based on the detected facial key points and correct the image to a standardized frontal pose through affine transformation to eliminate the influence of head pose changes. The eye prior distribution modeling subunit has a pre-stored eye position probability distribution model obtained through large sample clinical data statistics. The model is represented in the form of a Gaussian mixture model and is used to describe the possible spatial distribution area of ​​the eye in the standardized coordinate system. The coverage integrity calculation subunit is used to transform the probability distribution of the eyes in the standardized coordinate system back to the original image coordinates, and perform pixel-by-pixel spatial overlap analysis with the eye mask segmentation mask to calculate the coverage integrity index of the eye mask on the eyes. When the coverage integrity is lower than the preset threshold, it is determined that the eye mask is worn abnormally.

6. The automatic safety protection blue light phototherapy device according to claim 2, characterized in that, The eye mask edge fit detection unit includes: An edge extraction subunit is used to extract the precise edge contour of the eye mask from the eye mask segmentation mask; The local 3D reconstruction subunit is used to reconstruct the local 3D surface of the edge area of ​​the goggles using multiple frames of images illuminated by multiple angle light sources and photometric stereo technology, thereby obtaining the relative height information between the edge of the goggles and the skin. The gap quantization subunit is used to sample along the normal direction of the eye mask edge, calculate the height difference and gray-scale gradient change between the eye mask and the skin, and construct an edge fit score. When the score is lower than the threshold, it is judged as loose.

7. The automatic safety protection blue light phototherapy device according to claim 6, characterized in that, The gap quantization subunit samples at equal intervals along the normal direction of the eye mask edge contour. At each sampling point, it extends a preset pixel distance towards the inside of the eye mask and the outside of the skin, respectively, and extracts the pixel sequence on the normal path. Based on the pixel sequence, feature parameters for each sampling point are calculated, and the feature parameters include: Height difference feature: The height difference between the edge point of the eye patch and the adjacent skin point is obtained through local 3D reconstruction; Gray-level gradient features: Calculate the gradient rate of change of pixel gray level on the normal path, reflecting the edge sharpness at the junction of the eye mask and the skin; Texture consistency feature: Compare the texture similarity of regions on both sides of the normal path to reflect the characteristics of the transition area between the eye mask and the skin; The feature parameters are input into a pre-trained fit scoring model, which uses support vector regression or random forest regression algorithms to output a local fit score for each sampling point, with a value ranging from 0 to 1. The local fit scores of all sampling points are weighted and averaged. The weights are determined based on the eye coverage probability field value at the location of the sampling point. Areas with higher eye coverage probability are given higher weights to obtain the overall edge fit score of the eye mask. When the overall edge fit score is lower than the preset looseness threshold, the eye mask is determined to be in a loose state.

8. The automatic safety protection blue light phototherapy device according to claim 7, characterized in that, The training process of the fit rating model includes: A large number of clinical image samples were collected in advance, and experts marked the degree of fit at various positions on the edge of the eye patch in the images. The fit levels were marked into three categories: tight fit, slight lifting, and obvious gap. The feature parameters of each sampling point are extracted from the sample as input, and the fitting level labeled by experts is used as the output label to train the fitting score model. During training, cross-validation is used to optimize model parameters so that the model output score and expert annotations meet the preset accuracy requirements.

Images