Intelligent interaction system and method for medical self-service machine for the elderly based on BP neural network

Abstract

The application provides an old person medical self-service machine intelligent interaction system and method based on a BP neural network, binocular cameras collect near-infrared and visible light dual-mode image data of a user's face, a first sub-network calculates an old person identity similarity P1; the relationship between the old person identity similarity P1 and a preset threshold T1 is judged, if P1 >= T1, it is considered that an old person is operating, an infrared sensor is triggered to collect hand behavior data; hand behavior features are extracted according to the hand behavior data collected by the infrared sensor, the hand behavior features are used as inputs of a second sub-network, and a behavior compliance degree P2 is calculated; the relationship between the behavior compliance degree P2 and a preset threshold T4 is judged, if P2 < T4, and the hovering duration >= T2 or the click frequency <= T3, it is determined that an operation difficulty state exists; an interaction adaptation and multi-level early warning mechanism is triggered. The application has the characteristics of high intelligence, multi-modal data fusion, active intelligent adaptation, three-level early warning service closed loop, adaptive parameter configuration and small sample learning ability.

Description

Technical Field

[0001] This invention relates to the field of medical equipment technology, and in particular to an intelligent interactive system and method for a self-service medical machine for the elderly based on a BP neural network. Background Technology

[0002] With the advancement of medical informatization, hospitals at all levels across China have widely deployed self-service medical terminals to facilitate functions such as registration, payment, and printing of test reports. The widespread adoption of these machines has effectively alleviated queuing pressure at hospital counters and improved medical efficiency. However, with the arrival of an aging society and the continued rise in the proportion of the population aged 60 and above, the difficulties faced by the elderly when using self-service medical terminals are becoming increasingly prominent.

[0003] In terms of age-friendly renovations, existing technologies have gone through the following development stages:

[0004] (1) First stage: Physical modification stage - simple physical improvements such as increasing button size, increasing screen brightness, and equipping with reading glasses;

[0005] (2) Second stage: Manual adaptation stage - After the user manually selects "elderly mode", the system switches to the large font interface;

[0006] (3) Third stage: Intelligent recognition stage - start to try to recognize the user's age through the camera and automatically switch the interface mode.

[0007] Deficiencies and shortcomings of existing technologies

[0008] (1) Limited recognition accuracy: It relies solely on simple behavioral image recognition or age information for judgment, without using deep learning algorithms for accurate multi-dimensional feature analysis;

[0009] (2) Single data collection dimension: It only relies on a monocular camera to collect image information and does not introduce hardware devices such as infrared sensors, so it cannot accurately obtain refined behavioral characteristics such as hand hovering time, click frequency, and swipe speed;

[0010] (3) Lack of a complete service loop: A multi-level early warning mechanism has not been established, and a complete service process from local voice reminders to remote operation and maintenance assistance cannot be realized. When elderly people encounter difficulties, they still find it difficult to get timely help.

[0011] (4) No dual-mode fusion is used: Relying solely on a single-spectrum camera to acquire images, the recognition accuracy drops significantly in complex lighting environments such as low light and backlight;

[0012] (5) High sample dependence: Traditional machine learning and deep learning solutions require a large number of labeled samples for training, but it is difficult to collect facial and behavioral samples of the elderly in medical institutions, which restricts the application of the technology.

[0013] In view of this, it is necessary to improve the existing interactive systems for the elderly in order to solve the above problems. Summary of the Invention

[0014] The technical problem to be solved by the present invention is: in order to overcome the shortcomings of the prior art, the present invention provides an intelligent interactive system and method for elderly medical self-service machines based on BP neural networks.

[0015] The technical solution adopted by this invention to solve its technical problem is: an intelligent interactive system for elderly medical self-service machines based on BP neural networks, comprising a data acquisition module, a central processing module, and an interactive output module, wherein...

[0016] The data acquisition module includes a binocular camera and an infrared sensor. The binocular camera is used to simultaneously acquire near-infrared and visible light dual-mode image data of the user's face. The infrared sensor is used to non-contactly acquire user hand operation behavior data within 10cm in front of the touch screen display unit. The user hand operation behavior data includes one or more of the following: hover position, hover duration, click action, swipe direction, and swipe speed.

[0017] The central processing module has a built-in BP neural network model, which includes two sub-networks: a first sub-network and a second sub-network. The first sub-network uses a convolutional neural network (CNN) structure to process near-infrared and visible light dual-mode image data of the user's face captured by the binocular camera; the second sub-network uses a twin network structure to process user hand operation behavior data.

[0018] The interactive output module includes a touch screen display unit, an audio output unit, and an indicator light unit. The touch screen display unit performs interface adaptation operations, the audio output unit performs volume boost operations, and the indicator light unit performs multi-level warning status indication.

[0019] Further preferably, the binocular camera adopts an integrated dual-mode face recognition TCF261 device, installed directly above the touch screen display unit. It has a built-in near-infrared (850nm) and visible light dual-camera module with consistent baselines, a maximum resolution of 1920x1080 pixels, and a sampling frequency of 30 frames / second. It supports automatic white balance, automatic exposure, and face focusing, and is equipped with an 850nm infrared fill light, with an operating illuminance of up to 0.01Lux, for simultaneously acquiring near-infrared and visible light dual-mode image data of the user's face.

[0020] Further preferably, the infrared sensor adopts a matrix infrared ranging sensor array (8x8 units), with a detection range of 3-50cm and a sampling frequency of 20Hz, and is installed in front of the touch screen display unit.

[0021] Furthermore, to achieve the acquisition of near-infrared and visible light dual-mode image data of the user's face, the first sub-network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, a feature comparison layer, and an output layer. The input layer has 512 nodes, corresponding to the 512-dimensional pre-trained face model feature vector extracted from the dual-mode image data of the binocular camera. The convolutional layer consists of three layers: the first layer has 32 3×3 convolutional kernels, the second layer has 64 3×3 convolutional kernels, and the third layer has 128 3×3 convolutional kernels. The 3×3 kernel size indicates that the convolutional kernel covers a spatial dimension of 3 rows × 3 columns of pixels. The pooling layer consists of three layers, each pooling layer is placed after the corresponding convolutional layer, i.e., the order is: first convolutional layer → first pooling layer → second convolutional layer → second pooling layer → third convolutional layer → third pooling layer. All pooling layers use 2×2 max pooling with a stride of 2 pixels to perform pooling on the feature maps output by the convolutional layers. Downsampling reduces the feature map space size to half of its original size while retaining the most salient features, thereby reducing data dimensionality and enhancing the translation invariance of features. Two fully connected layers are used: a first fully connected layer with 256 nodes and a second fully connected layer with 128 nodes, for outputting feature vectors. The feature comparison layer compares the feature vectors output by the fully connected layers with template vectors in a pre-stored elderly registration template library, calculates the Euclidean distance d1, and converts it to a similarity P1 = 1 / (1+d1) through normalization. The output layer outputs the elderly person's identity similarity P1; when P1 ≥ a preset threshold T1, the user is determined to be an elderly person. The first sub-network employs a transfer learning strategy, using a pre-trained face recognition model as a foundation, freezing the parameters of the convolutional and pooling layers (i.e., feature extraction layers), and only fine-tuning the parameters of the last two fully connected layers, reducing the required number of samples from 5000 to 200-500.

[0022] Furthermore, to collect hand behavior data, the second sub-network adopts a Siamese network structure to process hand operation behavior data, including an input layer, a hidden layer, a feature distance calculation layer, and an output layer. The input layer includes 8 nodes, corresponding to 8 behavioral feature parameters, specifically: hand hovering duration, hovering position X-coordinate, hovering position Y-coordinate, number of clicks, average click interval, sliding direction angle, average sliding speed, and the percentage of time the hand is present within the detection area. The hidden layer includes two Siamese branches with shared weights, namely Siamese branch A and Siamese branch B. The twin branch A receives real-time 8-dimensional behavioral feature vectors, and the twin branch B receives template vectors from the behavioral prototype template library. Both branches pass through three hidden layers, with 32, 16, and 8 nodes respectively. The outputs of the two branches calculate the Euclidean distance d2 through the feature distance calculation layer and convert it into a similarity P2=1 / (1+d2) through normalization. The output layer outputs the behavioral compliance P2. When P2 is lower than the preset threshold T4, it is judged as a behavioral abnormality. The second sub-network is trained with a small number of samples. Only 5-10 labeled samples are needed to establish a prototype template for each hand operation behavior category.

[0023] Preferably, the interface adaptation includes enlarging the font, button size, and icon size, simplifying the interface layout, and enhancing the display contrast, wherein the font enlargement factor is 1.5-2.5 times.

[0024] Preferably, the audio output unit has a volume boost factor of 1.2-1.5 times; the multi-level warning status indications are, in order, local voice and text reminders, manual assistance prompts, and remote operation and maintenance assistance.

[0025] Furthermore, it also includes a host unit. The touchscreen display unit is located on the front side of the host unit. The infrared sensor and binocular camera are located on the host unit above the touchscreen display unit. The host unit below the touchscreen display unit is also equipped with a receipt printing outlet, a medical insurance card reader, a bank card reader, and a laser printer. The receipt printing outlet is used to print registration receipts and payment vouchers. The medical insurance card reader is used to read medical insurance card information to complete identity verification and medical insurance settlement. The bank card reader is used to read bank card information to complete payment. The laser printer is used to print inspection and testing reports.

[0026] A method for intelligent interaction in a self-service medical machine for the elderly based on a BP neural network, employing the aforementioned intelligent interaction system for a self-service medical machine for the elderly based on a BP neural network, further includes the following steps:

[0027] S1: The binocular camera acquires near-infrared and visible light dual-mode image data of the user's face, and detects facial information through the MTCNN algorithm. MTCNN is short for Multi-task Cascaded Convolutional Networks, which is a classic and influential face detection algorithm. The detected facial information is then used as input information into the first sub-network to calculate the similarity P1 of the elderly person's identity.

[0028] S2: Determine the relationship between the elderly person's identity similarity P1 and the preset probability threshold T1. If P1 ≥ T1, it is considered to be an elderly person's operation, triggering the infrared sensor to collect hand behavior data and proceeding to step S3; if P1 < T1, continue monitoring and return to step S1 to collect data again; preferably, the probability threshold T1 is in the range of T1 ≥ 0.6, and more preferably, the probability threshold T1 is in the range of 0.6-0.8.

[0029] S3: Extract hand behavior features from the hand behavior data collected by the infrared sensor, use the hand behavior features as the input of the second sub-network, and calculate the behavior compliance P2.

[0030] S4: Determine the relationship between the compliance of behavior P2 and the preset probability threshold T4, as well as the relationship between the actual hovering time and the preset hovering time threshold T2, and the actual click frequency and the preset click frequency threshold T3. If P2 < T4, and simultaneously satisfies: actual hovering time ≥ T2 or actual click frequency ≤ T3, then it is determined to be an operation difficulty state, and proceed to step S5; otherwise, it is determined to be an operation normal, and return to step S1 to continue monitoring.

[0031] S5: Trigger interactive adaptation and multi-level early warning mechanism.

[0032] Furthermore, step S5 specifically includes the following steps:

[0033] S51: The touchscreen display unit performs interface adaptation, enlarging the interface font to 1.5-2.5 times the original size, enlarging the operation button size to 1.5-2 times the original size, simplifying the interface layout to a single-column vertical arrangement, and enhancing the display contrast to more than 1.5 times; the audio output unit increases the voice prompt volume to 1.2-1.5 times the original volume and plays the voice prompt "It has been detected that you may need help. Do you need assistance?";

[0034] S52: If the user does not respond effectively within 30-60 seconds, the first-level warning will be activated, the yellow indicator light will be illuminated, and the text prompt "For assistance, please contact the medical guide!" and a one-click call button will be displayed on the touch screen display unit; the effective response includes clicking the screen button, completing an operation process, or actively clicking the "No help needed" button;

[0035] S53: If the user does not respond effectively within 60-120 seconds after the first-level warning, the second-level warning will be activated, the red indicator light will be illuminated, and the one-click call button will be highlighted on the touch screen.

[0036] S54: If no effective response is received after a cumulative wait of 60-180 seconds, a Level 3 warning will be issued. The central processing module will send a help signal to the hospital service center server through the network interface. The signal will include the self-service machine number, geographical coordinates, user facial feature snapshot, behavior compliance, and timestamp. After receiving the signal, the server will pop up an alarm window on the management interface to notify the medical guide to provide assistance.

[0037] The beneficial effects of this invention are:

[0038] (1) High level of intelligence: The BP neural network is used for dual-path parallel judgment. The first sub-network accurately identifies the attributes of the crowd based on facial images, and the second sub-network analyzes the operation status based on behavioral data. The dual judgment is more accurate and reliable. The recognition accuracy rate for the elderly is ≥95%, and the recognition accuracy rate for those with operational difficulties is ≥90%.

[0039] (2) Multimodal data fusion: Near-infrared and visible light dual-mode image information are collected simultaneously by binocular cameras to achieve cross-spectral fusion recognition. It can still accurately identify in a low-light environment of 0.01 Lux. Combined with infrared sensors to accurately collect 6-dimensional behavioral data of the hand, multimodal information fusion of image and behavior is achieved, and the judgment basis is more comprehensive.

[0040] (3) Proactive intelligent adaptation: The system automatically recognizes the user's identity and status, optimizes the interface font and voice prompts in real time, and provides assistance proactively, reducing the threshold for elderly users.

[0041] (4) Three-level early warning service closed loop: Establish a complete service chain of local voice and text reminders → manual assistance prompts → remote operation and maintenance assistance to ensure that the elderly can receive timely assistance in a tiered manner when they encounter problems that they cannot solve.

[0042] (5) Adaptive parameter configuration: The key thresholds are set in a range (judgment threshold 0.6-0.8, hovering threshold 20-60 seconds, etc.), which can be optimized according to different hospital scenarios and different elderly population characteristics, and have a wide range of applications.

[0043] (6) Small sample learning ability: The combination strategy of transfer learning and Siamese network greatly reduces the sample requirement (the requirement for facial samples is reduced by more than 90%, and the requirement for behavioral samples is reduced from 10,000 to 5-10 per category), which is particularly suitable for the actual situation of difficult sample collection in medical institutions. Attached Figure Description

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] Figure 1 This is the overall structural block diagram and reasoning logic flow of the intelligent interactive system for elderly medical self-service machines based on BP neural networks according to the present invention.

[0046] Figure 2 This is a schematic block diagram of the intelligent interactive system for elderly medical self-service machines based on BP neural networks, according to the present invention.

[0047] Figure 3 This invention relates to a flowchart of the intelligent interactive BP neural network model structure and method for a self-service medical machine for the elderly based on a BP neural network.

[0048] Figure 4 This is a schematic diagram of the self-service interface of the intelligent interaction method for the elderly medical self-service machine based on BP neural network according to the present invention.

[0049] In the diagram: 1. Main body, 2. Touch screen display unit, 3. Infrared sensor, 4. Binocular camera, 5. Receipt printing outlet, 6. Medical insurance card reader port, 7. Bank card reader port, 8. Laser printer port. Detailed Implementation

[0050] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0052] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. Furthermore, the technical features involved in the different embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0053] like Figure 1 and Figure 2 As shown, this invention discloses an intelligent interactive system for a self-service medical machine for the elderly based on a BP neural network. The system includes a host, a data acquisition module, a central processing module, and an interactive output module. All three modules are located on the host. Specifically, the touchscreen display unit is located on the front of the host. The infrared sensor and binocular camera are located on the host above the touchscreen display unit. Below the touchscreen display unit, the host also has a receipt printing port 5 for printing registration receipts and payment vouchers; a medical insurance card reader port 6 for reading medical insurance card information to complete identity verification and medical insurance settlement; a bank card reader port 7 for reading bank card information to complete payment; and a laser printer port 8 for printing examination reports. Figure 1 The system's overall structure diagram and reasoning logic flow are shown, illustrating the connections and data flow between the data acquisition module (dual-lens camera + infrared sensor), the central processing module (BP neural network), and the interactive output module (display + audio + indicator lights).

[0054] The data acquisition module includes a binocular camera and an infrared sensor. The binocular camera is used to simultaneously acquire near-infrared and visible light dual-mode image data of the user's face. The infrared sensor is used for non-contact acquisition of user hand operation behavior data within a 10cm range in front of the touch screen display unit. The user hand operation behavior data includes one or more of the following: hover position, hover duration, click action, swipe direction, and swipe speed. The binocular camera adopts an integrated dual-mode face recognition TCF261 device, installed directly above the touch screen display unit. It has a built-in near-infrared (850nm) and visible light dual-camera module with consistent baselines, a maximum resolution of 1920x1080 pixels, and an acquisition frequency of 30 frames / second. It supports automatic white balance, automatic exposure, and face focusing, and is equipped with an 850nm infrared fill light, with an operating illuminance of up to 0.01Lux, used for simultaneously acquiring near-infrared and visible light dual-mode image data of the user's face. More preferably, the infrared sensor adopts a matrix infrared ranging sensor array (8x8 units), with a detection range of 3-50cm, an acquisition frequency of 20Hz, and is installed in front of the touch screen display unit.

[0055] like Figure 1 and Figure 3 As shown, where, Figure 1 The reasoning and judgment logic flowchart shows the complete processing flow from face detection, elderly recognition, behavior collection, difficulty judgment to multi-level early warning. Figure 3 This document presents the structure diagram and flowchart of the BP neural network model, illustrating the hierarchical structure, number of nodes, connection relationships, and input / output dimensions of the first sub-network (CNN) and the second sub-network (Siamese network). The central processing module incorporates a BP neural network model, comprising two sub-networks: the first sub-network, which uses a Convolutional Neural Network (CNN) structure to process near-infrared and visible light dual-mode image data of the user's face captured by the binocular camera; and the second sub-network, which uses a Siamese network structure to process user hand gesture data.

[0056] The first sub-network includes an input layer, convolutional layers, pooling layers, fully connected layers, a feature comparison layer, and an output layer. The input layer has 512 nodes, corresponding to a 512-dimensional pre-trained face model feature vector. The convolutional layers consist of three layers: a first layer with 32 3×3 convolutional kernels, a second layer with 64 3×3 convolutional kernels, and a third layer with 128 3×3 convolutional kernels. The 3×3 kernel size indicates that the kernel covers a spatial dimension of 3 rows × 3 columns of pixels. The pooling layers also consist of three layers, each following a corresponding convolutional layer, in the following order: first convolutional layer → first pooling layer → second convolutional layer → second pooling layer → third convolutional layer → third pooling layer. All pooling layers use 2×2 max pooling with a stride of 2 pixels to downsample the feature maps output by the convolutional layers, reducing the spatial size of the feature maps to their original size while retaining the most salient features. The data dimensionality is reduced by half, thereby enhancing the translation invariance of features. There are two fully connected layers: a first fully connected layer with 256 nodes and a second fully connected layer with 128 nodes, used to output feature vectors. The feature comparison layer compares the feature vectors output by the fully connected layers with the template vectors in the pre-stored elderly registration template library, calculates the Euclidean distance d1, and converts it into a similarity P1=1 / (1+d1) through normalization. The output layer outputs the elderly identity similarity P1. When P1≥ a preset threshold T1, the user is determined to be an elderly person. The first sub-network adopts a transfer learning strategy, using a pre-trained face recognition model as a base, freezing the parameters of the convolutional layers and pooling layers (i.e., the feature extraction layers), and only fine-tuning the parameters of the last two fully connected layers (i.e., the first and second fully connected layers). The sample requirement is reduced from 5000 images to 200-500 images.The second sub-network adopts a Siamese network structure to process hand operation behavior data, including an input layer, a hidden layer, a feature distance calculation layer, and an output layer. The input layer includes 8 nodes, corresponding to 8 behavioral feature parameters, specifically: hand hovering duration (seconds), hovering position X-coordinate (pixels), hovering position Y-coordinate (pixels), number of clicks (times), average click interval (seconds), swipe direction angle (degrees), average swipe speed (pixels / second), and percentage of time the hand is present within the detection area (percentage). The hidden layer includes two Siamese networks with shared weights. The network consists of two branches, twin branch A and twin branch B. Twin branch A receives a real-time 8-dimensional behavioral feature vector, while twin branch B receives a template vector from a behavioral prototype template library. Both branches pass through three hidden layers with 32, 16, and 8 nodes respectively. The outputs of the two branches are used to calculate the Euclidean distance d2 through a feature distance calculation layer, and then normalized to a similarity P2 = 1 / (1+d2). The output layer outputs the behavioral compliance P2. When P2 is lower than a preset threshold T4, the behavior is considered abnormal. The second sub-network is trained with a small number of samples. Only 5-10 labeled samples are needed to establish a prototype template for each hand operation behavior category.

[0057] The interactive output module includes a touch screen display unit, an audio output unit, and an indicator light unit. The touch screen display unit performs interface adaptation operations, including but not limited to font enlargement, button size enlargement, icon size enlargement, interface layout simplification, and display contrast enhancement, wherein the font enlargement factor is 1.5-2.5 times. The audio output unit performs volume boosting operations, with a boost factor of 1.2-1.5 times. The indicator light unit performs multi-level warning status indication, which includes local voice and text reminders, manual assistance prompts, and remote operation and maintenance assistance, in that order.

[0058] like Figures 1-3 As shown, a method for intelligent interaction in a self-service medical machine for the elderly based on a BP neural network, employing the aforementioned intelligent interaction system for a self-service medical machine for the elderly based on a BP neural network, further includes the following steps:

[0059] S1: The binocular camera acquires near-infrared and visible light dual-mode image data of the user's face, and detects facial information through the MTCNN algorithm. MTCNN is short for Multi-task Cascaded Convolutional Networks, which is a classic and influential face detection algorithm. The detected facial information is then used as input information into the first sub-network to calculate the similarity P1 of the elderly person's identity.

[0060] S2: Determine the relationship between the elderly person's identity similarity P1 and the preset probability threshold T1. If P1 ≥ T1, it is considered to be an elderly person's operation, triggering the infrared sensor to collect hand behavior data and proceeding to step S3; if P1 < T1, continue monitoring and return to step S1 to collect data again; preferably, the probability threshold T1 is in the range of T1 ≥ 0.6, and more preferably, the probability threshold T1 is in the range of 0.6-0.8.

[0061] S3: Extract hand behavior features from the hand behavior data collected by the infrared sensor, use the hand behavior features as the input of the second sub-network, and calculate the behavior compliance P2.

[0062] S4: Determine the relationship between the compliance of behavior P2 and the preset probability threshold T4, as well as the relationship between the actual hover duration and the preset hover duration threshold T2, and the actual click frequency and the preset click frequency threshold T3. If P2 < T4, and simultaneously satisfies: actual hover duration ≥ T2 or actual click frequency ≤ T3, then it is determined to be an operation difficulty state, and proceed to step S5; otherwise, it is determined to be an operation normal, and return to step S1 to continue monitoring; preferably, the hover duration threshold T2 is in the range of 20-60 seconds, the click frequency threshold T3 is in the range of 1-5 times / minute, and the probability threshold T4 is in the range of 0.5-0.7.

[0063] S5: As Figure 4 As shown, this triggers interactive adaptation and multi-level early warning mechanisms.

[0064] S51: The touchscreen display unit performs interface adaptation, enlarging the interface font to 1.5-2.5 times the original size, enlarging the operation button size to 1.5-2 times the original size, simplifying the interface layout to a single-column vertical arrangement, and enhancing the display contrast to more than 1.5 times; the audio output unit increases the voice prompt volume to 1.2-1.5 times the original volume and plays the voice prompt "It has been detected that you may need help. Do you need assistance?";

[0065] S52: If the user does not respond effectively within 30-60 seconds, the first-level warning will be activated, the yellow indicator light will be illuminated, and the text prompt "For assistance, please contact the medical guide!" and a one-click call button will be displayed on the touch screen; the effective response includes clicking the screen button, completing an operation process, or actively clicking the "No help needed" button;

[0066] S53: If the user does not respond effectively within 60-120 seconds after the first-level warning, the second-level warning will be activated, the red indicator light will be illuminated, and the one-click call button will be highlighted on the touch screen.

[0067] S54: If no effective response is received after a cumulative wait of 60-180 seconds, a Level 3 warning will be issued. The central processing module will send a help signal to the hospital service center server through the network interface. The signal will include the self-service machine number, geographical coordinates, user facial feature snapshot, behavior compliance, and timestamp. After receiving the signal, the server will pop up an alarm window on the management interface to notify the medical guide to provide assistance.

[0068] [Implementation Scenario]: In the outpatient hall of a tertiary hospital, 10 medical self-service machines integrating the system of this invention are installed, serving an average of about 500 elderly users per day.

[0069] [Hardware Configuration]:

[0070] • Binocular camera: TCF261 integrated dual-mode device, consistent baseline, near-infrared 850nm + visible light dual-mode, 1920x1080@30fps, infrared fill light 0.01Lux;

[0071] • Infrared sensor: 8x8 matrix ranging array, detection range 3-50cm, sampling frequency 20Hz, RS485 interface Modbus RTU protocol;

[0072] • Central Processing Unit: ARM Cortex-A76 high-performance embedded processor, 4GB RAM, 32GB storage, USB 3.0 + Ethernet interface;

[0073] • Interactive output: 21.5-inch 1920x1080 capacitive touchscreen, 5W speaker, tri-color LED indicator.

[0074] [Model Training]:

[0075] • First sub-network: Based on the FaceNet pre-trained model, it was fine-tuned using 300 facial samples of elderly people collected by the hospital through transfer learning, with a learning rate of 0.0003, trained for 40 epochs, and an accuracy of 92%.

[0076] • Second sub-network: Using a twin network structure, a prototype template was trained based on 500 normal operation logs and 500 difficult operation logs from the hospital's history. Eight labeled samples were used to establish a baseline for each category, and the accuracy of difficult operation identification was 91%.

[0077] [Threshold Configuration]:

[0078] • The threshold for judging the elderly is T1=0.7, the threshold for hovering time is T2=40 seconds, the threshold for click frequency is T3=3 times / minute, and the threshold for difficulty in operation is T4=0.6.

[0079] [Running Results]:

[0080] • On average, the elderly identification function is triggered approximately 450 times per day, and the operation difficulty warning is triggered approximately 80 times per day;

[0081] • The success rate for elderly users increased from 65% to 88%;

[0082] • The on-site response time for staff to request assistance has been reduced from an average of 15 minutes to less than 5 minutes;

[0083] • User satisfaction rating improved from 3.2 out of 5 to 4.5 out of 5.

[0084] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the scope of the present invention. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A smart interactive system for a self-service medical machine for the elderly based on a BP neural network, characterized in that: It includes a data acquisition module, a central processing module, and an interactive output module, among which, The data acquisition module includes a binocular camera and an infrared sensor. The binocular camera is used to simultaneously acquire near-infrared and visible light dual-mode image data of the user's face. The infrared sensor is used to non-contactly acquire user hand operation behavior data within 10cm in front of the touch screen display unit. The user hand operation behavior data includes one or more of the following: hover position, hover duration, click action, swipe direction, and swipe speed. The central processing module has a built-in BP neural network model, which includes two sub-networks: a first sub-network and a second sub-network. The first sub-network uses a convolutional neural network structure to process near-infrared and visible light dual-mode image data of the user's face captured by the binocular camera; the second sub-network uses a twin network structure to process user hand operation behavior data. The interactive output module includes a touch screen display unit, an audio output unit, and an indicator light unit. The touch screen display unit performs interface adaptation operations, the audio output unit performs volume boost operations, and the indicator light unit performs multi-level warning status indication.

2. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: The binocular camera uses an integrated dual-mode face recognition TCF261 device, with a built-in near-infrared and visible light dual-camera module with consistent baselines, a maximum resolution of 1920x1080 pixels, and a sampling frequency of 30 frames / second; it supports automatic white balance, automatic exposure, and face focusing, and is equipped with an 850nm infrared fill light, with an operating illuminance of up to 0.01Lux.

3. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: The infrared sensor uses a matrix infrared ranging sensor array with a detection range of 3-50cm and a sampling frequency of 20Hz.

4. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: The first sub-network includes an input layer, convolutional layers, pooling layers, fully connected layers, feature comparison layers, and an output layer. The input layer has 512 nodes, corresponding to the 512-dimensional pre-trained face model feature vector extracted from dual-mode image data from a binocular camera. The convolutional layers consist of three layers: a first layer with 32 3×3 convolutional kernels, a second layer with 64 3×3 convolutional kernels, and a third layer with 128 3×3 convolutional kernels. The 3×3 kernel size indicates that the kernel covers a spatial dimension of 3 rows × 3 columns of pixels. The pooling layers also consist of three layers, each following a corresponding convolutional layer, in the following order: first convolutional layer → first pooling layer → second convolutional layer → second pooling layer → third convolutional layer → third pooling layer. All pooling layers use 2×2 max pooling with a stride of 2 pixels to downsample the feature maps output by the convolutional layers, preserving the most salient features while... The feature map space size is reduced to half of its original size, thereby reducing data dimensionality and enhancing the translation invariance of features. Two fully connected layers are used: a first fully connected layer with 256 nodes and a second fully connected layer with 128 nodes, for outputting feature vectors. The feature comparison layer compares the feature vectors output by the fully connected layers with template vectors in a pre-stored elderly registration template library, calculates the Euclidean distance d1, and converts it to a similarity P1 = 1 / (1+d1) through normalization. The output layer outputs the elderly person's identity similarity P1; when P1 ≥ a preset threshold T1, the user is determined to be an elderly person. The first sub-network adopts a transfer learning strategy, using a pre-trained face recognition model as a foundation, freezing the parameters of the convolutional and pooling layers (i.e., feature extraction layers), and only fine-tuning the parameters of the last two fully connected layers, reducing the required number of samples from 5000 to 200-500.

5. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: The second sub-network adopts a Siamese network structure to process hand operation behavior data, including an input layer, a hidden layer, a feature distance calculation layer, and an output layer. The input layer includes 8 nodes, corresponding to 8 behavioral feature parameters, specifically: hand hovering duration, hovering position X-coordinate, hovering position Y-coordinate, number of clicks, average click interval, sliding direction angle, average sliding speed, and the percentage of time the hand is present within the detection area. The hidden layer includes two Siamese branches with shared weights, namely Siamese branch A and Siamese branch B. Siamese branch A receives real-time 8D... The behavior feature vector is generated by the twin branch B receiving template vectors from the behavior prototype template library. Both branches pass through three hidden layers with 32, 16, and 8 nodes respectively. The outputs of the two branches are converted into Euclidean distance d2 by the feature distance calculation layer and then normalized to a similarity P2=1 / (1+d2). The output layer outputs the behavior compliance P2. When P2 is lower than the preset threshold T4, the behavior is judged as abnormal. The second sub-network is trained with a small number of samples. Only 5-10 labeled samples are needed to build a prototype template for each hand operation behavior category.

6. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: The interface adaptation includes enlarging the font, button size, and icon size, simplifying the interface layout, and enhancing the display contrast, with the font enlargement factor being 1.5-2.5 times.

7. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: The audio output unit has a volume boost factor of 1.2-1.5 times; the multi-level warning status indications are, in order, local voice and text reminders, manual assistance prompts, and remote operation and maintenance assistance.

8. The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in claim 1, characterized in that: It also includes a main unit, with the touchscreen display unit located on the front side of the main unit. The infrared sensor and binocular camera are located on the main unit above the touchscreen display unit. The main unit below the touchscreen display unit is also equipped with a receipt printing outlet, a medical insurance card reader, a bank card reader, and a laser printer. The receipt printing outlet is used to print registration receipts and payment vouchers; the medical insurance card reader is used to read medical insurance card information to complete identity verification and medical insurance settlement; the bank card reader is used to read bank card information to complete payment; and the laser printer is used to print inspection and testing reports.

9. A method for intelligent interaction in a self-service medical machine for the elderly based on a BP neural network, characterized in that: The intelligent interactive system for elderly medical self-service machines based on BP neural networks as described in any one of claims 1-8 further includes the following steps: S1: The binocular camera collects near-infrared and visible light dual-mode image data of the user's face, and detects facial information through the MTCNN algorithm. Then, the detected facial information is used as input information into the first sub-network to calculate the elderly person's identity similarity P1. S2: Determine the relationship between the elderly person's identity similarity P1 and the preset probability threshold T1. If P1 ≥ T1, it is considered to be an elderly person's operation, triggering the infrared sensor to collect hand behavior data and proceeding to step S3; if P1 < T1, continue monitoring and return to step S1 to collect data again. S3: Extract hand behavior features from the hand behavior data collected by the infrared sensor, use the hand behavior features as the input of the second sub-network, and calculate the behavior compliance P2. S4: Determine the relationship between the compliance of behavior P2 and the preset probability threshold T4, as well as the relationship between the actual hovering time and the preset hovering time threshold T2, and the actual click frequency and the preset click frequency threshold T3. If P2 < T4, and simultaneously satisfies: actual hovering time ≥ T2 or actual click frequency ≤ T3, then it is determined to be an operation difficulty state, and proceed to step S5; otherwise, it is determined to be an operation normal, and return to step S1 to continue monitoring. The preset hovering time threshold T2 ranges from 20 to 60 seconds, and the preset click frequency threshold T3 ranges from 1 to 5 times / minute. S5: Trigger interactive adaptation and multi-level early warning mechanism.

10. The intelligent interaction method for elderly medical self-service machines based on BP neural networks as described in claim 9, characterized in that: Step S5 specifically includes the following steps: S51: The touchscreen display unit performs interface adaptation, enlarging the interface font to 1.5-2.5 times the original size, enlarging the operation button size to 1.5-2 times the original size, simplifying the interface layout to a single-column vertical arrangement, and enhancing the display contrast to more than 1.5 times; the audio output unit increases the voice prompt volume to 1.2-1.5 times the original volume and plays the voice prompt "It has been detected that you may need help. Do you need assistance?"; S52: If the user does not respond effectively within 30-60 seconds, the first-level warning will be activated, the yellow indicator light will be illuminated, and the text prompt "For assistance, please contact the medical guide!" and a one-click call button will be displayed on the touch screen display unit; the effective response includes clicking the screen button, completing an operation process, or actively clicking the "No help needed" button; S53: If the user does not respond effectively within 60-120 seconds after the first-level warning, the second-level warning will be activated, the red indicator light will be illuminated, and the one-click call button will be highlighted on the touch screen. S54: If no effective response is received after a cumulative wait of 60-180 seconds, a Level 3 warning will be issued. The central processing module will send a help signal to the hospital service center server through the network interface. The signal will include the self-service machine number, geographical coordinates, user facial feature snapshot, behavior compliance, and timestamp. After receiving the signal, the server will pop up an alarm window on the management interface to notify the medical guide to provide assistance.

Images