Attention assistance system and control method thereof

Abstract

An attention assisting system and a control method thereof, the attention assisting system includes an image sensor, a processing circuit, a magnetic pulse generator, and a magnetic pulse control module. The image sensor is used to sense a head frame. The processing circuit is used to determine a stimulation position from the head frame. The magnetic pulse generator is used to generate a magnetic pulse. The magnetic pulse control module is used to adjust the propagation direction of the magnetic pulse so that the magnetic pulse is incident on the stimulation position.

Description

Technical Field

[0001] This disclosure relates to an attention assist system, and more particularly to an attention assist system for enhancing attention and its control method. Background Technology

[0002] In current transcranial stimulation (TCS) techniques, the electrodes for transcranial microcurrent stimulation must be attached to the user's head, and the transcranial magnetic stimulation (TMS) device must be aligned with a designated location on the user's head to provide a changing magnetic field to the user's attentional nerves. This generates a potential difference within the attentional nerves, promoting neuronal charge transfer and activating the brain. However, in typical use, the user's head is fixed in place to ensure that the magnetic field provided by the TMS device is precisely directed to the brain region to be stimulated, increasing inconvenience. Therefore, providing a device and control method to solve these problems is an important issue in this field. Summary of the Invention

[0003] This disclosure provides an attention assist system, which includes an image sensor, a processing circuit, a magnetic pulse generator, and a magnetic pulse control module. The image sensor senses a head frame. The processing circuit determines the location to be stimulated from the head frame. The magnetic pulse generator generates magnetic pulses. The magnetic pulse control module adjusts the propagation direction of the magnetic pulses to direct them to the stimulation location.

[0004] This disclosure provides an attention assist system, which includes a first magnetic pulse generator, a first magnetic pulse control module, a second magnetic pulse generator, and a second magnetic pulse control module. The first magnetic pulse generator generates a first magnetic pulse. The first magnetic pulse control module adjusts the propagation direction of the first magnetic pulse to direct it to a first location to be stimulated. The second magnetic pulse generator generates a second magnetic pulse, wherein the first and second magnetic pulses are asynchronous. The second magnetic pulse control module adjusts the propagation direction of the second magnetic pulse to direct it to a second location to be stimulated.

[0005] This disclosure provides a control method for an attention assist system, comprising the following steps: An alert system senses a facial frame and determines a physical state based on the facial frame. If the physical state is determined to be fatigue, the alert system sends an activation signal to a magnetic pulse control system. The magnetic pulse control system measures a head frame and determines at least one location to be stimulated based on the head frame. The magnetic pulse control system generates and controls the propagation direction of at least one magnetic pulse so that the at least one magnetic pulse is incident on the at least one location to be stimulated.

[0006] In summary, this disclosure utilizes a magnetic pulse generator that identifies the driver's physical state to generate magnetic pulses, and a magnetic pulse control module adjusts the propagation direction of the magnetic pulses to direct them towards the stimulation target on the driver's head. Thus, the magnetic pulses generated by the magnetic pulse generator, via transcranial magnetic stimulation, can stimulate the driver's attentional neural network from the left frontal lobe to the right parietal lobe, thereby enhancing the driver's attention. Attached Figure Description

[0007] To make the above and other objects, features, advantages and embodiments of this disclosure more apparent and understandable, the accompanying drawings are described below:

[0008] Figure 1 This is a schematic diagram of an attention assist system 100 according to some embodiments of this disclosure.

[0009] Figure 2 This is a functional block diagram of an attention assist system 100 shown according to some embodiments of this disclosure.

[0010] Figure 3 This is a flowchart illustrating a control method according to some embodiments of this disclosure.

[0011] Figure 4 This is a schematic diagram of a head frame, head features, and head outline shown according to some embodiments of this disclosure.

[0012] Figures 5A to 5C This is a schematic diagram of a magnetic pulse shown according to some embodiments of this disclosure.

[0013] Explanation of reference numerals in the attached figures:

[0014] To make the above and other objects, features, advantages and embodiments of this disclosure more apparent and understandable, the appended symbols are explained as follows:

[0015] 100: Attention Assistance System

[0016] 110: Magnetic Pulse Control System

[0017] 111, 121: Processing circuit

[0018] 112, 122, 122a, 122b: Image sensors

[0019] 113: Magnetic pulse control device

[0020] 114, 115: Magnetic pulse generator

[0021] 116, 117: Magnetic pulse control module

[0022] 120: Alert System

[0023] 124: Emotion Recognition Neural Network

[0024] 200: Control Method

[0025] STP1, STP2: Locations to be stimulated

[0026] Ff: Face frame

[0027] Fh: Header frame

[0028] Fe1, Fe2: Characteristics

[0029] Fe3,FP: Head contour

[0030] MP1, MP2: Magnetic pulses

[0031] S210, S220, S230, S240: Steps Detailed Implementation

[0032] The following is a detailed description of embodiments with reference to the accompanying drawings. However, the provided embodiments are not intended to limit the scope of this disclosure, and the description of the structure's operation is not intended to limit its execution order. Any structure resulting from the recombination of elements and producing an apparatus with equivalent technical effects is within the scope of this disclosure. Furthermore, the illustrations are for illustrative purposes only and are not drawn to their original dimensions. For ease of understanding, the same or similar elements will be designated with the same symbols in the following description.

[0033] Unless otherwise specified, the terms used throughout the specification and claims generally have their ordinary meaning in the context of the art, the disclosure, and the specific content.

[0034] Furthermore, the terms "comprising," "including," "having," "containing," etc., used in this document are all open-ended terms, meaning "including but not limited to." Additionally, the term "and / or" as used in this document includes any one or more of the related listed items and all combinations thereof.

[0035] In this document, when a component is referred to as “coupled” or “coupled,” it may mean “electrically coupled” or “electrically coupled.” “Coupled” or “coupled” can also be used to indicate that two or more components operate or interact with each other. Furthermore, although terms such as “first,” “second,” etc., are used in this document to describe different components, these terms are only used to distinguish components or operations described using the same technical terms.

[0036] Please see Figure 1 , Figure 1This is a schematic diagram of an attention assist system 100 according to some embodiments of this disclosure. Figure 1 As shown, the attention assist system 100 includes image sensors 112, 122a, and 122b and a magnetopulse control device 113. In some embodiments, image sensors 112, 122a, and 122b are implemented by an image sensor, which may be a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, a CCD (Charge-Coupled Device) image sensor, other light-sensing components, or light-sensing devices.

[0037] Image sensors 122a and 122b are positioned in front of the driver's seat, with their lenses facing the driver's seat to capture the driver's face and generate facial frames. The facial frames captured by image sensors 122a and / or 122b are analyzed by a neural network model to assess the driver's physical / mental state (e.g., focused, calm, drowsy, absent-minded, tired) through micro-expressions / facial features.

[0038] In some embodiments, the image sensor 122a is disposed in the rearview mirror, windshield, or other locations. The image sensor 122a is positioned at a height close to the driver's face height, thereby capturing the driver's facial expressions from both top-down and horizontal perspectives, and generating a facial frame. In some embodiments, the field of view of the image sensor 122a includes the driver's face, thereby capturing the driver's face and the driver's body movements (e.g., rubbing eyes). In some embodiments, the image sensor 122a captures close-ups of the driver's eye muscle changes from both top-down and horizontal perspectives.

[0039] In some embodiments, the image sensor 122b is embedded / installed in the dashboard, center console, or other location. The image sensor 122b is positioned below the driver's face level to capture the driver's facial expressions from a low angle and generate a facial frame. In some embodiments, the field of view of the image sensor 122b includes the driver's face, thereby capturing the driver's face and the driver's body movements (e.g., covering the mouth). In some embodiments, the low-angle shooting can provide close-ups of the driver's mouth muscle movements.

[0040] In some embodiments, the image sensor 112 is disposed above the driver's seat and is used to capture images of the top of the driver's head to generate a head (top) frame. In some embodiments, the image sensor 112 is disposed on the inside of the roof, facing the driver's seat. The head frame captured by the image sensor 112 is analyzed by another neural network model, and the prefrontal cortex and other brain locations associated with attention can be located by the outer contour of the driver's head.

[0041] Please see Figure 2 , Figure 2 This is a functional block diagram of an attention assist system 100 shown according to some embodiments of this disclosure. For example... Figure 2 As shown, the attention assist system 100 includes a magnetopulse control system 110 and an alert system 120. The attention assist system 100 is electrically / communicationally coupled to the magnetopulse control system 110 and the alert system 120 to transmit commands, data, or information.

[0042] The magnetic pulse control system 110 includes a processing circuit 111, an image sensor 112, and a magnetic pulse control device 113. The warning system 120 includes a processing circuit 121, an image sensor 122, and a storage device 123.

[0043] Processing circuits 111 and 121 may be a central processing unit, microprocessor, graphics processing unit, field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other hardware device suitable for retrieving or executing instructions stored in memory. In some embodiments, processing circuit 111 is disposed on a microcontroller that includes memory (not shown), and processing circuit 121 and storage device 123 are disposed on another microcontroller.

[0044] Storage device 123 may be implemented by electrical, magnetic, optical storage devices or other storage devices for storing instructions or data. In some embodiments, memory 120 may be implemented by volatile memory or non-volatile memory. In some embodiments, memory 120 may be implemented by random access memory (RAM), dynamic random access memory (DRAM), magnetoresistive random access memory (MRAM), phase-change random access memory (PCRAM) or other storage devices.

[0045] In some embodiments, Figure 2 The image sensor 112 corresponds to Figure 1 The image sensor 112 in, and Figure 2 The image sensor 122 corresponds to Figure 1 Image sensor 122a and / or image sensor 122b.

[0046] Please refer to the following: Figure 2 as well as Figure 3 . Figure 3 This is a flowchart illustrating a control method 200 according to some embodiments of this disclosure. The control method 200 includes steps S210 to S240.

[0047] In step S210, the warning system 120 captures the driver's face to generate a facial image frame Ff, and determines the driver's physical condition based on the facial image frame Ff. In some embodiments, the processing circuit 121 of the warning system 120 is communicatively / electrically connected to the image sensor 122. The image sensor 122 captures the driver's face to generate a facial image frame Ff, and transmits the facial image frame Ff to the processing circuit 121.

[0048] In some embodiments, the processing circuit 121 of the warning system 120 is electrically coupled to the storage device 123. The processing circuit 121 extracts relevant instructions and data from the storage device 123 to run the emotion recognition neural network 124, thereby using the emotion recognition neural network 124 to analyze the driver's facial expressions (e.g., changes in eye size / shape caused by muscle activity around the eyes, changes in mouth shape caused by muscle movement of the mouth) and / or body movements (e.g., the driver rubbing his eyes or covering his mouth) in the facial frame Ff and generate analysis results (e.g., whether the driver is tired, distracted, or has negative emotions), in order to determine the driver's physical / psychological state (e.g., focused, calm, drowsy, absent-minded, tired).

[0049] In some embodiments, the emotion recognition neural network 124 may be implemented by a supervised neural network architecture (e.g., a convolutional neural network architecture, a deep neural network architecture, a deep convolutional neural network architecture), an unsupervised neural network architecture (e.g., an adversarial generative network architecture), a hybrid neural network architecture, or other neural network architectures. Therefore, this disclosure is not limited thereto.

[0050] In step S220, if the driver's physical state is determined to be fatigued, the warning system 120 sends an activation signal to the magnetic pulse control system 110. If the processing circuit 121 obtains that the driver's physical state is fatigued by calculating the facial frame Ff using the emotion recognition neural network 124 in step S210, the processing circuit 121 sends an activation signal to the processing circuit 111 of the magnetic pulse control system 110.

[0051] In step S230, the magnetic pulse control system 110 captures the driver's head to generate a head frame Fh, and determines at least one location to be stimulated from the head frame Fh. Specifically, the processing circuit 111 of the magnetic pulse control system 110 responds to the start signal transmitted by the processing circuit 121 of the warning system 120, and controls / instructs the image sensor 112 to capture / sense the driver's head (top of the head) and transmit the sensed image back to the processing circuit 111 to generate the head frame Fh.

[0052] The processing circuit 111 of the magnetic pulse control system 110 retrieves relevant instructions and data for running the contour recognition neural network from a storage device (not shown), thereby using the contour recognition neural network to analyze the position coordinates corresponding to the top of the driver's head in the head frame Fh. In some embodiments, the contour recognition neural network may be implemented by a supervised neural network architecture (e.g., a convolutional neural network architecture, a deep neural network architecture, a deep convolutional neural network architecture), an unsupervised neural network architecture (e.g., an adversarial generative network architecture), a hybrid neural network architecture, or other neural network architectures. Therefore, this disclosure is not limited thereto.

[0053] For a better explanation, please refer to the following: Figure 4 . Figure 4This is a schematic diagram illustrating a head frame Fh, head features Fes, head contour Fe3, and FP according to some embodiments of this disclosure. Processing circuit 111 uses a contour recognition neural network to extract the driver's nose tip position as feature Fe1 from the head frame Fh, extracts the driver's facial contour as feature Fe2 based on the nose tip position, and extracts the driver's head contour Fe3 (corresponding to head contour FP) based on features Fe1 and Fe2. Processing circuit 111 uses the contour recognition neural network to obtain the corresponding position coordinates (e.g., positions A1, A2, O1, O2, T3-T6, P3-P4, Pz, C3-C4, Cz, F3-F4, F7-F8, Fz, Fp1, and Fp2) based on the driver's head contour FP, and extracts at least one of positions F3 and P4 as stimulation positions STP1 and STP2. In some embodiments, position coordinate F3 corresponds to the left frontal lobe region of the human brain, and position P4 corresponds to the right parietal lobe region of the human brain.

[0054] In step S240, the magnetic pulse control system 110 generates and controls the propagation direction of at least one magnetic pulse so that the at least one magnetic pulse is incident on at least one position to be stimulated. The processing circuit 111 instructs the magnetic pulse control device 113 to generate magnetic pulses Mp1 and Mp2 to the positions STP1 to STP2 on the driver's head to be stimulated. Specifically, the magnetic pulse control device 113 includes magnetic pulse generators 114 and 115 and magnetic pulse control modules 116 and 117. The magnetic pulse generators 114 and 115 are used to generate a magnetic field in the vertical direction, and the magnetic pulse control modules 116 and 117 are composed of liquid crystals doped with multiple magnetic nanoparticles. In some embodiments, the magnetic nanoparticles may be superparamagnetic nano-iron oxide (Fe3O4).

[0055] In some embodiments, the processing circuit 111 directs the magnetic nanoparticles in the magnetic pulse control modules 116 and 117 of the magnetic pulse control device 113 to change the propagation direction of the magnetic pulses Mp1 and Mp2 according to the stimulation positions STP1 and STP2, so that the magnetic pulses Mp1 and Mp2 are directed toward the stimulation positions STP1 and / or STP2 on the driver's head.

[0056] Please see Figures 5A to 5C . Figures 5A to 5C This is a schematic diagram illustrating magnetic pulses Mp1 and Mp2 according to some embodiments of this disclosure. Figures 5A to 5CAs shown, the waveforms of magnetic pulses Mp1 and Mp2 are out of phase. In other words, the phase difference between magnetic pulses Mp1 and Mp2 is half a period. In some embodiments, the frequencies of magnetic pulses Mp1 and Mp2 are values ​​within the range of 10Hz, 20Hz, or 5-30Hz, and the frequencies of magnetic pulses Mp1 and Mp2 are set to the same frequency. Thus, the magnetic pulses Mp1 and Mp2 generated by the magnetic pulse control device 113 can stimulate the driver's frontal-to-right parietal attention network via repetitive transcranial magnetic stimulation (rTMS), thereby enhancing the driver's attention.

[0057] It is worth noting that, in Figures 5A to 5C In some embodiments, magnetic pulse Mp1 and magnetic pulse Mp2 are asynchronous pulses. In other embodiments, magnetic pulse Mp1 and magnetic pulse Mp2 can be synchronous pulses; therefore, this disclosure is not limited thereto.

[0058] In summary, the alert system 120 of the attention assist system 100 of this disclosure uses the emotion recognition neural network 124 to analyze whether the driver is fatigued. If the alert system 120 determines that the driver is fatigued, the alert system 120 communicates with the magnetic pulse control system 110 through an electrical / communication connection, so that the magnetic pulse control system 110 uses a contour neural network to recognize the contour of the driver's head and analyze the location of the driver's brain regions, thereby extracting the positions of the top of the driver's head corresponding to the left frontal lobe and the right parietal lobe as the stimulation positions STP1 and STP2.

[0059] Furthermore, the magnetic pulse generators 114 and 115 of the magnetic pulse control device 113 in the magnetic pulse control system 110 generate magnetic pulses MP1 and MP2. Magnetic pulses MP1 and MP2 are adjusted / controlled in their propagation direction via magnetic pulse control modules 116 and 117 to the stimulation positions STP1 and STP2 on the driver's head. Thus, the magnetic pulses MP1 and MP2 generated by the magnetic pulse control device 113 can stimulate the driver's attentional neural network from the left frontal lobe to the right parietal lobe via transcranial magnetic stimulation, thereby enhancing the driver's attention.

[0060] Although this disclosure has been described above with reference to embodiments, it is not intended to limit this disclosure. Any person skilled in the art can make various changes and modifications without departing from the concept and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be determined by the claims.

Claims

1. An attention assisting system, comprising: an image sensor configured to sense a head frame; a processing circuit configured to determine a stimulation position from the head frame; a magnetic pulse generator configured to generate a magnetic pulse; and a magnetic pulse control module configured to adjust a propagation direction of the magnetic pulse such that the magnetic pulse is incident on the stimulation position, wherein the stimulation position corresponds to a left frontal lobe or a right parietal lobe of a human brain, thereby improving attention of a driver, wherein the magnetic pulse generator is configured to generate a magnetic field along a vertical direction, wherein the magnetic pulse control module comprises liquid crystal doped with magnetic particles, wherein the processing circuit is configured to instruct the magnetic particles in the magnetic pulse control module to turn based on the stimulation position, thereby changing the propagation direction of the magnetic pulse such that the magnetic pulse is directed toward the stimulation position.

2. The attention assisting system of claim 1, wherein the processing circuit is configured to construct a head contour feature from the head frame using a neural network model, and the processing circuit is configured to determine the stimulation position based on the head contour feature.

3. The attention assisting system of claim 1, wherein the magnetic pulse has a frequency in a range of 10-30 Hz.

4. The attention assisting system of claim 1, wherein the magnetic particles have superparamagnetic properties.

5. An attention assisting system, comprising: a first magnetic pulse generator configured to generate a first magnetic pulse; a first magnetic pulse control module configured to adjust a propagation direction of the first magnetic pulse such that the first magnetic pulse is incident on a first stimulation position; a second magnetic pulse generator configured to generate a second magnetic pulse, wherein the first magnetic pulse and the second magnetic pulse are not synchronized; and a second magnetic pulse control module configured to adjust a propagation direction of the second magnetic pulse such that the second magnetic pulse is incident on a second stimulation position, wherein the first stimulation position and the second stimulation position correspond to a left frontal lobe and a right parietal lobe of a human brain, respectively, thereby improving attention of a driver, wherein the first magnetic pulse generator and the second magnetic pulse generator are configured to generate a magnetic field along a vertical direction, wherein the first magnetic pulse control module and the second magnetic pulse control module comprise liquid crystal doped with magnetic particles, wherein a processing circuit is configured to instruct the magnetic particles in the first magnetic pulse control module and the second magnetic pulse control module to turn based on the first stimulation position and the second stimulation position, thereby changing the propagation directions of the first magnetic pulse and the second magnetic pulse such that the first magnetic pulse and the second magnetic pulse are directed toward the first stimulation position and the second stimulation position, respectively.

6. The attention assisting system of claim 5, wherein the first magnetic pulse and the second magnetic pulse have a same frequency, wherein the frequency is set in a range of 10-30 Hz, and wherein the first magnetic pulse and the second magnetic pulse have a phase difference of one-half period.

7. A control method, comprising: sensing, by an alert system, a face frame and determining a physical state based on the face frame; if the physical state is determined to be fatigue, transmitting, by the alert system, an activation signal to a magnetic pulse control system; measuring a head frame by the magnetic pulse control system, and determining at least one stimulation position from the head frame; and generating and controlling a propagation direction of at least one magnetic pulse by the magnetic pulse control system so that the at least one magnetic pulse is incident to the at least one stimulation position, wherein the at least one stimulation position corresponds to a left frontal lobe or a right parietal lobe of a human brain, thereby improving the driver's attention, wherein a magnetic pulse generator of the magnetic pulse control system generates a magnetic field along a vertical direction, wherein a magnetic pulse control module of the magnetic pulse control system comprises liquid crystals with magnetic particles, wherein the magnetic pulse control system indicates the magnetic particles in the magnetic pulse control module to turn according to the stimulation position, so as to change the propagation direction of the magnetic pulse, and make the magnetic pulse shoot at the stimulation position.

Images