Wearing identification method and device for eye protection device of near-infrared light treatment equipment

Abstract

The invention provides a wearing recognition method and device for an eye protection device of near-infrared light treatment equipment, and the method comprises the steps: continuously collecting a sequence of images through a wide-angle image collection device disposed at the inner side of the front part of a head cap, and carrying out the recognition of the image sequence under the condition that the head cap is properly worn on the head of an object, the image acquisition device forms an oblique overlook view angle relative to the head of the object; and based on at least one recent image in the acquired image sequence, identifying, by at least one processor, a wearing condition of the eye protection device by using a trained deep learning network, specifically comprising: if the at least one recent image contains at least one of a plurality of angulated crossing patterns on the eye protection device, identifying the wearing condition of the eye protection device by using the trained deep learning network, and if the at least one recent image contains at least one of a plurality of angulated crossing patterns on the eye protection device, identifying the wearing condition of the eye protection device. If yes, the wearing condition of the eye protection device is recognized as wearing in place, illumination operation of the near-infrared light treatment equipment is allowed, and if not, the wearing condition of the eye protection device is recognized as wearing non-in-place, and illumination operation of the near-infrared light treatment equipment is forbidden.

Description

Technical Field

[0001] The present disclosure relates to the technical field of near-infrared light therapy, and in particular to a wearing identification method and device for an eye protection device of a near-infrared light therapy device. Background Art

[0002] When using near-infrared light for light therapy of the brain, the near-infrared light needs to be able to penetrate the skull and enter the user's brain tissue, so the near-infrared light during treatment has a strong optical power. During the light therapy process, if the eye protection device is not worn correctly, it is easy to cause damage to the user's eyes.

[0003] In the related art, in order to determine whether the user is wearing eye protection correctly, a deep learning network can be used for recognition. Taking eye protection as protective glasses as an example, the existing glasses recognition model cannot recognize the specific type of glasses, so when the user is wearing glasses or sunglasses, it is easy to be misidentified, and the recognition accuracy needs to be improved. In addition, there is also a problem of insufficient recognition accuracy for whether the protective glasses are in an acceptable wearing state. Summary of the invention

[0004] The present application is proposed to solve the above technical problems in the prior art. The present application aims to provide a method and device for identifying the wearing of an eye protection device of a near-infrared light therapy device, which can identify the recent images collected by the image acquisition device in the near-infrared light therapy device through a trained deep learning network, and determine that the wearing status of the eye protection device is in place when at least one of the multiple angled cross patterns on the eye protection device is identified, thereby ensuring the eye safety of the user during the light therapy process.

[0005] According to the first scheme of the present application, a method for identifying the wearing of an eye protection device of a near-infrared light therapy device is provided, wherein the near-infrared light therapy device comprises a head cap worn on the head of an object and a plurality of illumination units for emitting near-infrared light to the head of the object, and the wearing identification method comprises the following steps: using a wide-angle image acquisition device disposed on the inner side of the front of the head cap to continuously acquire a sequence of images, and when the head cap is properly worn on the head of the object, the image acquisition device is at an angle of view relative to the head of the object. Based on at least one recent image in the acquired image sequence, at least one processor uses a trained deep learning network to identify the wearing status of the eye protection device, specifically comprising: if the at least one recent image contains at least one of the plurality of angled cross patterns on the eye protection device, then the wearing status of the eye protection device is identified as being worn in place, and the illumination operation of the near-infrared light therapy device is allowed; otherwise, the wearing status of the eye protection device is identified as being not worn in place, and the illumination operation of the near-infrared light therapy device is prohibited.

[0006] According to a second scheme of the present application, a wearing identification device for an eye protection device of a near-infrared light therapy device is provided, the wearing identification device comprising a processor, and the processor is configured to at least execute the steps of the wearing identification method for an eye protection device of a near-infrared light therapy device described in each embodiment of the present application.

[0007] According to the third scheme of the present application, a light therapy system is provided, which includes a head cap, a plurality of illumination units, a wide-angle image acquisition device, and a processor. The head cap is worn on the head of the subject. The illumination unit is used to emit near-infrared light to the head of the subject. The image acquisition device is arranged on the inner front side of the head cap, and is used to continuously acquire an image sequence. The processor is configured to: use a trained deep learning network to identify the wearing condition of the eye protection device in at least one recent image in the image sequence, specifically including: if the at least one recent image contains at least one of the plurality of angled cross patterns on the eye protection device, then the wearing condition of the eye protection device is identified as being worn in place, and the illumination operation of the near-infrared light therapy device is allowed; otherwise, the wearing condition of the eye protection device is identified as not being worn in place, and the illumination operation of the near-infrared light therapy device is prohibited.

[0008] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0009] According to a fourth embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the wearing identification method of the eye protection device of the near-infrared light therapy equipment described in each embodiment of the present application are executed.

[0010] Compared with the prior art, the beneficial effects of the embodiments of the present application are: The wearing identification method of the eye protection device of the near-infrared light therapy device provided in the embodiment of the present application can continuously collect image sequences by using a wide-angle image acquisition device arranged on the front side of the head cap inside the near-infrared light therapy device, and continuously perform wearing identification of the eye protection device through a trained deep learning network, and when the wearing condition of the eye protection device is identified as being worn in place, the illumination operation of the near-infrared light therapy device is allowed, otherwise the illumination operation of the near-infrared light therapy device is prohibited, thereby ensuring the eye safety of the user during the light therapy process. Among them, the eye protection device contains a plurality of angular cross patterns, which are difficult to be repeated with the content in the background area of ​​the image and other existing eye devices (such as glasses). Therefore, the eye protection device identification performed on the angular cross patterns in the eye protection device using a deep learning network can accurately identify the special eye protection device, and is not easily interfered by other eye devices or other objects in the background, and has high recognition accuracy and strong anti-interference ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0012] Figure 1 A flowchart showing a method for identifying the wearing of an eye protection device of a near-infrared light therapy device provided by some embodiments of the present disclosure is shown; Figure 2 A schematic diagram of a near-infrared light therapy device in a wearing identification method of an eye protection device of a near-infrared light therapy device provided in some embodiments of the present disclosure is shown; Figure 3 A three-dimensional schematic diagram of an eye protection device at different angles in a wearing identification method of an eye protection device of a near-infrared light therapy device provided in some embodiments of the present disclosure is shown; Figure 4 A schematic diagram of the image preprocessing process in the wearing identification method of the eye protection device of the near-infrared light therapy equipment provided in some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present application.

[0014] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0015] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined here. Techniques and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the techniques and equipment should be considered as part of the specification.

[0016] See also Figure 1 As shown, it is a flow chart of a method for wearing and identifying an eye protection device of a near-infrared light therapy device provided in an embodiment of the present disclosure, the method includes S101 to S102, wherein: S101: Utilize a wide-angle image acquisition device disposed on the inner front side of the head cap to continuously acquire a sequence of images. When the head cap is properly worn on the subject's head, the image acquisition device is at an inclined downward viewing angle relative to the subject's head.

[0017] S102: Based on at least one recent image in the acquired image sequence, at least one processor uses a trained deep learning network to identify the wearing condition of the eye protection device, specifically including: if the at least one recent image contains at least one of the multiple angled cross patterns on the eye protection device, then the wearing condition of the eye protection device is identified as being worn in place, and the illumination operation of the near-infrared light therapy device is allowed; otherwise, the wearing condition of the eye protection device is identified as being not worn in place, and the illumination operation of the near-infrared light therapy device is prohibited.

[0018] The following is a detailed description of the above steps.

[0019] With respect to S101, the wide-angle image acquisition device may be, for example, a fisheye camera, an eagle-eye camera, etc., and the field of view of the image acquisition device may be 80°, 130°, 160°, 170°, 180°, 210°, 220°, 360°, etc. When acquiring a sequence of images, the image acquisition device may acquire images according to a set number of image frames, which may be 20 frames, 25 frames, 30 frames, 60 frames, etc.

[0020] Exemplarily, the schematic diagram of the head cap of the near-infrared light therapy device can be as follows: Figure 2 As shown, the device may also include a host electrically connected to the head cap (in Figure 2 (not shown), the host may include a light unit driving circuit, a main processor, etc. Figure 2 As shown, the head cap is provided with a plurality of illumination units (not shown in the figure) for emitting near-infrared light to the head of the subject, and a wide-angle image acquisition device is provided on the inner front side of the head cap, and the image acquisition device is at an inclined downward viewing angle relative to the head of the subject.

[0021] In this way, by using a wide-angle image acquisition device that is arranged at an inclined downward viewing angle relative to the subject's head, the image of the subject's head inside the head cap of the near-infrared light therapy device can be completely acquired, thereby enabling stable identification of the wearing of the eye protection device; on the other hand, compared to the image acquisition device arranged outside the near-infrared light therapy device, by arranging the image acquisition device on the front inner side of the head cap, it is possible to avoid obstruction of the subject's head during image acquisition, thereby ensuring that the subject's head can be continuously photographed during the light therapy process, thereby ensuring that the wearing identification mechanism of the eye protection device can continue to operate during the light therapy process, thereby improving safety during the light therapy process.

[0022] For S102, the network type of the deep learning network may be a convolutional neural network (CNN), a Transformer model, etc. The eye protection device may be a device with an eye protection function such as protective glasses and protective goggles. The eye protection device may be provided with a plurality of angularly crossed patterns, such as a "cross" shaped image. The processor may be a general / dedicated processor disposed inside a near-infrared light therapy device, such as a processor in a head cap, or a processor in a host, etc., or a processor disposed in a cloud server in the cloud. The crossed patterns can not only be significantly distinguished from life or equipment patterns in the background, such as spots and stripes on clothes, chassis and wires, but also maintain good recognition in wide-angle photographs under various wearing postures of the eye protection device, for example, even if the angle size changes, the crossed lines and sharp angles are still maintained, so that it is easier to be identified by a machine learning algorithm and is not easily confused with other interference patterns.

[0023] Among them, the image on the eye protection device can be processed by laser or other means to obtain an angled cross pattern that is different from the explicit physical properties of the eye protection device body. For example, the eye protection device body can be made of black matte material, and the area where the angled cross pattern is located can be made of white mirror material.

[0024] Exemplarily, the schematic diagram of the eye protection device can be as follows: Figure 3 As shown in (a) and (b) of the figure, the overall shape of the eye protection device is a pair of glasses that can be worn on the head of the subject, and the eye protection device is provided with angled cross patterns on the upper side, front side, left side, and right side, respectively, so that Figure 2 The image capture device configured as shown is capable of capturing images containing an angled cross pattern when the eye protection device is properly worn.

[0025] In this way, the angularly crossed pattern has significantly different explicit physical properties from other content captured by the image acquisition device, especially the background pattern. The deep learning network trained on this basis can be used specifically to identify eye protection devices containing angularly crossed patterns, thereby achieving high-precision identification of the wearing of eye protection devices.

[0026] In some embodiments, the deep learning network can be configured to perform recognition based on an integrated feature map, wherein the integrated feature map is extracted according to the following steps A1-A4: A1: Perform initial convolution processing on the input image to obtain the first feature map.

[0027] Here, when performing initial convolution processing on the input image, convolution processing, batch normalization processing, and activation processing can be performed on the input image in sequence to complete the initial convolution processing on the input image and obtain the first feature map.

[0028] Exemplarily, the convolution kernel used in the convolution processing can be 3×3, the step size used can be 2, and the activation function used in the activation processing can be the SiLU activation function, so as to achieve preliminary extraction of image features in the input image, facilitating subsequent execution of target detection, image classification, semantic segmentation and other tasks.

[0029] A2: Perform multi-level feature extraction and weighted fusion on the first feature map in sequence to obtain a second feature map representing local features.

[0030] Here, the local features can be understood as features that represent the shallow layer and pay more attention to the local content of the image, rather than the features corresponding to the local image obtained by only extracting features from the local image, due to the small number of feature extraction times and the feature extraction method being more local-oriented.

[0031] In some embodiments, when performing multi-level feature extraction and weighted fusion on the first feature map in sequence, a convolution module and a residual connection module including a residual network structure can be used to perform local feature extraction and weighted fusion processing on the first feature map in sequence to obtain a second feature map representing local features.

[0032] Among them, the residual network structure can extract richer gradient flow information, and can achieve deep fusion of features at different levels through weighted fusion. The weight parameters used in the weighted fusion processing can be learnable parameters. By adjusting the learnable parameters (i.e., weights) in real time during the training process of the deep learning network, a deep learning network that is more suitable for identifying eye protection devices can be obtained.

[0033] A3: Perform multi-level feature extraction and weighted fusion on the second feature map to obtain a third feature map representing global features.

[0034] Here, when performing multi-level feature extraction and weighted fusion on the second feature map, the second feature map can be sequentially subjected to convolution processing and multiple pooling processing, and the feature map obtained after each pooling processing can be weighted fused to obtain a third feature map that can characterize global features.

[0035] The pooling process may be a maximum pooling process, and the size of the pooling window used in the pooling process may be set to 5 to improve the ability to extract global features.

[0036] A4: Integrate the third feature map using a feature pyramid.

[0037] Here, the type of the feature pyramid can be FPN, Path Aggregation Network with Feature Pyramid Network (PAN-FPN), etc. Since the FPN in PAN-FPN is used to realize top-down semantic information transmission, and PAN is used to realize bottom-up path aggregation, and the positioning information of the low-level feature map is passed to the high-level feature map, PAN-FPN has stronger positioning capability and target detection performance than FPN.

[0038] In some embodiments, when the third feature map is integrated using a feature pyramid, the third feature map may be integrated using a target intermediate feature map obtained in a process of generating the third feature map.

[0039] Among them, the target intermediate feature map can be the second feature map and / or the intermediate feature map obtained in the process of generating the second feature map.

[0040] In this way, combined with the high-performance feature integration capability of PAN-FPN, the global features and local features in the image can be deeply integrated, thereby improving the recognition accuracy of the deep learning network.

[0041] In some embodiments, when using a trained deep learning network to identify the wearing condition of the eye protection device, the following steps B1-B2 may be performed: B1: performing image preprocessing on the at least one recent image to obtain an image to be identified; wherein the image preprocessing includes at least one of cropping the image size to a predetermined size, adjusting the image resolution to a predetermined resolution, and performing image mask processing.

[0042] In some embodiments, when the at least one recent image is preprocessed to obtain the image to be recognized, Figure 4 As shown, you can do it through the following steps B11~B13: B11: Perform image size cropping on the at least one recent image to crop the image size of the at least one recent image to a predetermined size to obtain a first intermediate image.

[0043] Exemplarily, taking the image resolution of the at least one recent image as 640px×360px as an example, the long half can be taken as the center and the half of the wide side can be retained on the left and right sides for image cropping to obtain a first intermediate image of 360px×360px.

[0044] B12: Adjust the resolution of the first intermediate image to a predetermined resolution to obtain a second intermediate image.

[0045] Here, the predetermined resolution (ie, the resolution of the second intermediate image) is smaller than the resolution of the first intermediate image.

[0046] In some embodiments, when adjusting the image resolution, the image resolution can be adjusted by means of a linear interpolation algorithm or the like to reduce the resolution of the first intermediate image, thereby reducing the real-time computing resources required for single image recognition and improving the stability of the near-infrared light therapy device during continuous wearing recognition during light therapy.

[0047] Continuing with the above example, the first intermediate image with a resolution of 360px×360px may be adjusted to a second intermediate image with a resolution of 320px×320px by a linear interpolation algorithm.

[0048] In this way, by compressing the image resolution, the wearing recognition efficiency can be improved, and the degree of compression does not affect the accuracy of the recognition result. That is, the image can be compressed as much as possible while ensuring accurate recognition, taking into account both recognition accuracy and recognition speed.

[0049] B13: Perform mask processing on the background area on the side opposite to the head of the object in the second intermediate image to obtain an image to be recognized with simplified background image information.

[0050] Here, when masking the background area, the RGB value of each pixel in the background area can be set to zero, so that the color of the pixel is black, and it can be ignored when the subsequent deep learning network performs recognition.

[0051] For example, Figure 4 As shown, the head of the object is located in the lower half of the image, and the background area is located in the upper half of the image. Therefore, the upper half of the image can be masked to obtain an image to be identified with simplified background image information.

[0052] In some embodiments, the normal activity area can be determined based on the pre-tested movable range of the subject's head when the head cap is properly worn, and the background image area that needs to be masked can be determined based on the normal activity area.

[0053] Exemplarily, the area above the following coordinate points (320, 150), (250, 140), (200, 130), (160, 125), (100, 130), (50, 140), and (0, 150) can be used as the background image area.

[0054] In this way, by performing mask processing on the second intermediate image, the real-time computing resources required for single image recognition can be further reduced, thereby improving the stability of the near-infrared light therapy device when it is continuously worn and recognized during the light therapy process.

[0055] B2: Using the trained deep learning network, identify the eye protection device being worn on the image to be identified.

[0056] Here, image cropping, image compression, and image masking are performed in sequence through the above steps to obtain an image to be identified that is suitable for continuous identification of scenes during phototherapy, thereby greatly reducing the real-time computing resources required for single image recognition and improving the stability of the near-infrared light therapy device during continuous wear identification during phototherapy.

[0057] In actual applications, since the subject's head remains relatively fixed for most of the time during light therapy, continuously using a deep learning network for wearing recognition may consume more computing resources. Therefore, eye protection device wearing recognition can be performed only when specific conditions are met.

[0058] In some embodiments, before using a trained deep learning network to identify the wearing condition of the eye protection device, it is also possible to determine whether to use the deep learning network to perform wearing identification on the at least one image based on the similarity between the at least one recent image and the first image of the worn-in-place reference image.

[0059] If the first image similarity is greater than a first predetermined image similarity, the deep learning network is not used and the wearing condition of the eye protection device is directly identified as being worn in place; otherwise, the deep learning network is used to perform wearing recognition on the at least one image.

[0060] Among them, the first predetermined image similarity can be 95%, 98%, etc. If the first image similarity is greater than the first predetermined image similarity, it indicates that the similarity between the currently recognized image and the worn-in-place reference image is very high, so it can be considered that the current image is also worn in place.

[0061] In some embodiments, the wearing position reference image may include at least one of the following: 1. The first image of the current subject with the eye protection device in place.

[0062] Here, the first image may be determined after being recognized by the deep learning network, or may be collected in advance for the current object and manually confirmed.

[0063] For example, training can be performed before starting light therapy to allow the current subject to correctly wear eye protection and headgear. At this time, image acquisition can be performed, and manual (or deep learning network) confirmation of wearing in place can be performed, and the acquired image can be used as the first image.

[0064] 2. A second image of the reference subject corresponding to the eye protection device being worn in place.

[0065] Here, the reference object may be an object that matches the attributes of the current object, and the attributes may include gender, head shape, head size, age, etc.

[0066] For example, taking the current object as an elderly male with a square head shape, a reference object matching the attributes of the current object can be found in the preset reference object library, and a second image corresponding to the reference object with eye protection devices worn in place can be used for subsequent image matching.

[0067] 3. A third image formed by merging the first image and the second image.

[0068] Here, when performing image fusion, a predetermined image fusion algorithm may be used to perform the fusion, so that the third image contains facial features of both the current object and the reference object.

[0069] When using the third image to perform image similarity judgment, if the currently recognized image has a high similarity with the third image, it means that its movement amplitude is small and meets the wearing requirements of the eye protection device, that is, the standard for similarity judgment using the third image is more stringent.

[0070] In some embodiments, the training sample images of the deep learning network can be obtained by following the steps C1 to C3: C1: Obtain a sample video of a sample subject performing phototherapy using the phototherapy device; wherein the sample subject wears the eye protection device, and the sample subject performs at least one set action during the phototherapy, and the set action includes wearing a head cap and eye protection device at the same time during the phototherapy, and being in a standard posture for phototherapy.

[0071] Here, the sample objects may include subjects, human models, etc. with different attributes.

[0072] In some embodiments, the setting action includes at least one of the following: 1. Rotate the subject's head horizontally within the first set angle range.

[0073] Here, the first setting angle range may be from -30 degrees to 30 degrees, that is, no more than 30 degrees to the left and no more than 30 degrees to the right.

[0074] 2. Rotate the subject's head in the vertical direction within a second set angle range.

[0075] Here, the second set angle range may be from -15 degrees to 15 degrees, that is, no more than 15 degrees upwards and no more than 15 degrees downwards.

[0076] 3. Within the third set angle range, the subject leans forward or backward.

[0077] Here, the third setting angle may range from -10 degrees to 10 degrees, that is, the forward inclination does not exceed 10 degrees and the backward inclination does not exceed 10 degrees.

[0078] In this way, these set actions can simulate the possible positions of the subject's head during phototherapy and the status of the eye protection device in the corresponding position. The training sample images constructed on this basis can accurately identify whether the subject is wearing eye protection devices normally in the phototherapy scenario.

[0079] C2: Sampling the sample video to obtain a sample image to be labeled.

[0080] In some embodiments, when the sample video is sampled to obtain the sample image to be labeled, the following steps C21 to C23 may be performed: C21: Sampling the sample video according to a predetermined sampling interval to obtain multiple frames of images to be screened Here, in order to avoid the image similarity being too high due to a too short sampling interval, which makes it easy for the deep learning network to overfit during training, and the sampling interval being too long resulting in longer sample videos and higher costs, the sampling interval can be set to 1s.

[0081] C22: For each frame of the image to be filtered, filtering is performed based on the second image similarity between the current frame of the image to be filtered and its adjacent frame of the image to be filtered, to obtain a target image group and a similar image group; wherein, the image similarity between the similar images in the similar image group and at least one target image in the target image group is greater than the second predetermined image similarity.

[0082] Here, the adjacent frame of the image to be filtered of the current frame of the image to be filtered may be the image to be filtered of the previous frame or the image to be filtered of the subsequent frame.

[0083] In some embodiments, when filtering based on the second image similarity between the current frame image to be filtered and its adjacent frame image to be filtered to obtain a target image group and a similar image group, starting from the second frame image to be filtered, the current frame image to be filtered and the previous frame image to be filtered can be used to calculate the image similarity, and when the image similarity is greater than the second predetermined image similarity, the previous frame image to be filtered is placed in the similar image group, and the current frame image to be filtered is placed in the target image group.

[0084] C23: the target image in the target image is used as the sample image to be labeled, while the similar images in the similar image group are not used as the sample image to be labeled.

[0085] In this way, by calculating the image similarity to screen the images to be screened, the overfitting problem caused by using a large number of similar sample images to train the deep learning network can be avoided, thereby ensuring the recognition accuracy and generalization performance of the deep learning network.

[0086] C3: Obtaining a labeling result of a detection frame of an eye protection device of the sample image to be labeled. The sample image to be labeled and its corresponding labeling result are used to form a training sample image.

[0087] Here, the shape of the detection frame may be a rectangle, such as a rectangle including the eye protection device; or, the shape of the detection frame may be an irregular shape formed by the edge of the eye protection device.

[0088] In some embodiments, the labeling result corresponding to the sample image to be labeled can be obtained according to the following steps C31-C32: C31: For each sample image to be labeled, use the trained eye protection device recognition model to identify the detection frame of the eye placement device.

[0089] Here, the eye protection device recognition model may be a model for performing a target detection task or a semantic segmentation task.

[0090] In some embodiments, when the eye protection device is protective glasses, the eye protection device recognition model may be a pre-trained glasses recognition model for recognizing glasses.

[0091] In some embodiments, when the eye protection device is a protective eye mask, the eye protection device recognition model may be a pre-trained eye mask recognition model for recognizing an eye mask.

[0092] C32: when the current sample image to be labeled contains the corner intersection pattern, the detection frame is corrected according to the position information of the corner intersection pattern in the current training sample image, and the corrected detection frame is used as the labeling result corresponding to the sample image.

[0093] Here, when the detection frame is corrected according to the position information of the angular intersection pattern in the current training sample image, the detection frame can be adjusted in response to the boundary correction operation on the detection frame according to the boundary correction operation; or, the detection frame in the sample image to be labeled can be automatically adjusted according to the target detection frame corresponding to the angular intersection pattern and a preset adjustment algorithm, so that the target detection frame is included in the detection frame, that is, the angular intersection pattern is included in the current detection frame, thereby realizing automatic adjustment of the detection frame and improving image labeling efficiency.

[0094] In some embodiments, when the deep learning network is used to identify the wearing condition of the eye protection device, any one of the following methods may be used: Method 1: Identify the relative position relationship between the eye protection device and the subject's face in the image.

[0095] Here, the deep learning network may be used to recognize a first detection frame of the eye protection device and a second detection frame of the subject's face in the at least one recent image.

[0096] Specifically, if the relative position relationship between a first detection frame corresponding to the eye protection device in at least one of the recent images and a second detection frame corresponding to the face of the object satisfies a predetermined eye coverage condition, the wearing condition of the eye protection device is identified as being in place, and the illumination operation of the near-infrared light therapy device is allowed; otherwise, the wearing condition of the eye protection device is identified as not being in place, and the illumination operation of the near-infrared light therapy device is prohibited.

[0097] In some embodiments, the eye coverage condition may be obtained based on statistical data.

[0098] When performing data statistics, based on the images of the eye protection device being correctly worn captured by the image acquisition device in the near-infrared light therapy equipment, the relative position relationship between the first detection frame corresponding to the eye protection device and the second detection frame corresponding to the subject's face when the eye protection device is correctly worn can be obtained, so that the corresponding eye coverage conditions can be further obtained.

[0099] Method 2: Identify whether the object's eyes exist in the image.

[0100] Here, the deep learning network is used to classify whether the at least one recent image contains the eyes of the object. If it contains glasses, pupils, and other contents that should not be recognized when the eye protection device is normally worn, it means that the eye protection device is not worn in place.

[0101] Further, if the classification result of the at least one recent image is that it contains the subject's eyes, the wearing condition of the eye protection device is identified as not being worn properly, and the illumination operation of the near-infrared light therapy device is prohibited.

[0102] In some other embodiments, a wearing identification device for an eye protection device of a near-infrared light therapy device is provided, and the wearing identification device includes a processor, and the processor is configured to at least execute the steps of the wearing identification method for the eye protection device of the near-infrared light therapy device described in each embodiment of the present application. Please note that the processor can be a processor built into the head cap, a processor in a host electrically connected to the head cap, or a processor in the cloud, which will not be repeated here.

[0103] In some other embodiments, a light therapy system is also provided, which includes a head cap, a plurality of illumination units, a wide-angle image acquisition device, and a processor. The head cap is worn on the head of the subject. The illumination unit is used to emit near-infrared light to the head of the subject. The image acquisition device is arranged on the inner front side of the head cap, and is used to continuously acquire an image sequence. The processor is configured to: use a trained deep learning network to identify the wearing condition of the eye protection device in at least one recent image in the image sequence, specifically including: if the at least one recent image contains at least one of the plurality of angled cross patterns on the eye protection device, then the wearing condition of the eye protection device is identified as being worn in place, and the illumination operation of the near-infrared light therapy device is allowed; otherwise, the wearing condition of the eye protection device is identified as not being worn in place, and the illumination operation of the near-infrared light therapy device is prohibited.

[0104] The processor may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0105] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0106] The exemplary methods described in this application may be implemented at least in part by a machine or computer. In some embodiments, a computer-readable storage medium stores computer program instructions, which, when executed by a processor, cause the processor to perform the wearing identification method of the eye protection device of the near-infrared light therapy device as described in various embodiments of the present application.

[0107] The implementation of such a method may include software code, such as microcode, assembly language code, high-level language code, etc. Various software programming techniques may be used to create various programs or program modules. For example, a program portion or program module may be designed with or by means of Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules may be integrated into a computer system and / or a computer-readable medium. Such software code may include computer-readable instructions for executing various methods. The software code may form a part of a computer program product or a computer program module. In addition, in an example, the software code may be tangibly stored on one or more volatile, non-temporary or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (such as optical disks and digital video disks), cassettes, memory cards or memory sticks, random access memories (RAMs), read-only memories (ROMs), etc.

[0108] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the execution of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the claims and the full scope of their equivalents.

[0109] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein as examples or embodiments in the specific embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0110] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for identifying the wearing of an eye protection device of a near-infrared light therapy device, wherein the near-infrared light therapy device comprises a head cap worn on a subject's head and a plurality of illumination units for emitting near-infrared light to the subject's head, characterized in that: The steps include: Continuously capturing a sequence of images using a wide-angle image acquisition device disposed on the inner side of the front portion of the head cap, wherein the image acquisition device is at an oblique downward viewing angle relative to the subject's head when the head cap is properly worn on the subject's head; Based on at least one recent image in the acquired image sequence, at least one processor uses a trained deep learning network to identify the wearing condition of the eye protection device, specifically including: if the at least one recent image contains at least one of the multiple angled cross patterns on the eye protection device, then the wearing condition of the eye protection device is identified as being worn in place, and the lighting operation of the near-infrared light therapy device is allowed; otherwise, the wearing condition of the eye protection device is identified as being not worn in place, and the lighting operation of the near-infrared light therapy device is prohibited.

2. The wearing identification method according to claim 1, characterized in that: The deep learning network is configured to perform recognition based on an integrated feature map, which is extracted according to the following process: Perform initial convolution processing on the input image to obtain the first feature map; Sequentially performing multi-level feature extraction and weighted fusion on the first feature map to obtain a second feature map representing local features; Performing multi-level feature extraction and weighted fusion on the second feature map to obtain a third feature map representing global features; The third feature map is integrated using a feature pyramid.

3. The wearing identification method according to claim 1 or 2, characterized in that: The method of using a trained deep learning network to identify the wearing status of eye protection devices includes: Performing image preprocessing on the at least one recent image to obtain an image to be identified; wherein the image preprocessing includes at least one of cropping the image size to a predetermined size, adjusting the image resolution to a predetermined resolution, and performing image mask processing; The trained deep learning network is used to identify the eye protection device being worn on the image to be identified.

4. The wearing identification method according to claim 3, characterized in that: Performing image preprocessing on the at least one recent image to obtain an image to be recognized includes: Performing image size cropping on the at least one recent image to crop the image size of the at least one recent image to a predetermined size, thereby obtaining a first intermediate image; adjusting the resolution of the first intermediate image to a predetermined resolution to obtain a second intermediate image; Mask processing is performed on the background area on the side opposite to the head of the object in the second intermediate image to obtain an image to be recognized with simplified background image information.

5. The wearing identification method according to claim 1, characterized in that: Prior to using the trained deep learning network to identify the wearing status of the eye protection device, the processor also includes: Based on the first image similarity between the at least one recent image and the worn-in-place reference image, determine whether to use the deep learning network to perform wearing recognition on the at least one image; if the first image similarity is greater than a first predetermined image similarity, the deep learning network is not used and the wearing condition of the eye protection device is directly identified as worn-in-place; otherwise, the deep learning network is used to perform wearing recognition on the at least one image.

6. The wearing identification method according to claim 5, characterized in that: The wearing position reference image includes at least one of the following: A first image of the current subject with the eye protection device in place, a second image of the reference subject with the eye protection device in place, and a third image formed by the fusion of the first image and the second image.

7. The wearing identification method according to claim 1, characterized in that: The wearing recognition method further comprises obtaining a training sample image of the deep learning network according to the following steps: Obtaining a sample video of a sample subject using the phototherapy device for phototherapy; wherein the sample subject wears the eye protection device, and the sample subject performs at least one set action during the phototherapy process, wherein the set action includes simultaneously wearing a head cap and an eye protection device during the phototherapy process, and being in a standard posture for phototherapy; Performing sampling processing on the sample video to obtain a sample image to be labeled; Obtain a labeling result of a detection frame of the eye protection device of the sample image to be labeled, and the sample image to be labeled and its corresponding labeling result are used to form a training sample image.

8. The wearing identification method according to claim 7, characterized in that: The setting action includes at least one of the following: Rotate the subject's head in a horizontal direction within a first set angle range; within a second set angle range, rotating the subject's head in a vertical direction; Within the third set angle range, the subject's body leans forward or backward.

9. The wearing identification method according to claim 7 or 8, characterized in that: The sampling process of the sample video to obtain a sample image to be labeled includes: Sampling the sample video according to a predetermined sampling interval to obtain multiple frames of images to be screened; For each frame of the image to be screened, screening is performed according to the second image similarity between the current frame of the image to be screened and its adjacent frame of the image to be screened, to obtain a target image group and a similar image group; wherein the image similarity between the similar images in the similar image group and at least one target image in the target image group is greater than the second predetermined image similarity; The target image in the target image group is used as the sample image to be labeled, while the similar image in the similar image group is not used as the sample image to be labeled.

10. The wearing identification method according to claim 7 or 8, characterized in that: The wearing recognition method further comprises obtaining a labeling result corresponding to the sample image to be labeled according to the following steps: For each sample image to be labeled, use the trained eye protection device recognition model to identify the detection frame of the eye protection device; In the case that the current sample image to be labeled contains the corner intersection pattern, the detection frame is corrected according to the position information of the corner intersection pattern in the current training sample image, and the corrected detection frame is used as the labeling result corresponding to the sample image.

11. The wearing identification method according to claim 1, characterized in that: Also includes: The deep learning network is used to identify a first detection frame of the eye protection device and a second detection frame of the subject's face in the at least one recent image; if the relative positional relationship between the first detection frame corresponding to the eye protection device in the at least one recent image and the second detection frame corresponding to the subject's face satisfies a predetermined eye coverage condition, the wearing condition of the eye protection device is identified as being in place, and the illumination operation of the near-infrared light therapy device is allowed; otherwise, the wearing condition of the eye protection device is identified as not being in place, and the illumination operation of the near-infrared light therapy device is prohibited.

12. The wearing identification method according to claim 1, characterized in that: It also includes: using the deep learning network to classify whether the at least one recent image contains the object's eyes. If the classification result of the at least one recent image is that it contains the object's eyes, identifying the wearing status of the eye protection device as not being worn properly, and prohibiting the lighting operation of the near-infrared light therapy device.

13. A wearing identification device for eye protection device of near infrared light therapy equipment, characterized in that: The device comprises a processor configured to at least execute the steps of the wearing identification method of the eye protection device of the near-infrared light therapy device according to any one of claims 1 to 12.

14. A light therapy system, characterized in that: include: a head cap, the head cap being worn on the subject's head; a plurality of illumination units, the illumination units being used to emit near infrared light toward the head of the subject; A wide-angle image acquisition device, which is disposed inside the front of the head cap and is used to continuously acquire an image sequence; A processor, wherein the processor is configured to: use a trained deep learning network to identify the wearing condition of an eye protection device in at least one recent image in an image sequence, specifically including: if the at least one recent image contains at least one of a plurality of angled cross patterns on the eye protection device, then identifying the wearing condition of the eye protection device as being in place, and allowing the illumination operation of a near-infrared light therapy device; otherwise, identifying the wearing condition of the eye protection device as being not in place, and prohibiting the illumination operation of the near-infrared light therapy device.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the wearing identification method of the eye protection device of the near-infrared light therapy device as claimed in any one of claims 1 to 12.

Images