An intelligent panel control method based on DNN noise reduction technology

Abstract

The present invention relates to the field of smart panel technology, and discloses a smart panel control method based on DNN noise reduction technology, including: using a pre-trained first convolutional neural network model to perform noise reduction processing on voice interaction data to obtain noise-reduced voice interaction data; analyzing image interaction data and noise-reduced voice interaction data to obtain user instruction data; adjusting the display content data and interface layout data of the smart panel to be controlled based on the user instruction data and context labels; and controlling the display interaction interface of the smart panel to be controlled based on the adjusted display content data and adjusted interface layout data. By implementing the present invention, multi-source data, including images, voices, environmental data, and personnel behavior data, are integrated, so that the smart panel can respond to the user's operation more accurately, solving the problem that the smart panel control method has obvious deficiencies in data processing and context adaptability, and is difficult to meet the high requirements of modern hospitals for intelligent management and control.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent panel control, and in particular to an intelligent panel control method based on DNN noise reduction technology. Background Art

[0002] With the rapid development of intelligent medical systems, intelligent management and control within hospitals have become an important means to improve the quality and efficiency of medical services. Traditional intelligent panel control methods mainly rely on simple button operations or limited touch interactions, which have obvious deficiencies in user experience, comprehensive information acquisition, and environmental adaptability. When receiving user instructions, traditional intelligent panels often only use a single interaction channel such as buttons or touch screens, ignoring more natural and efficient interaction methods such as voice and images. This results in users needing specific operating environments and conditions when interacting with intelligent panels, limiting the flexibility and convenience of interaction. Especially in a complex and changeable environment such as a hospital, medical staff or patients may find it difficult to effectively operate the intelligent panel through traditional methods due to emergency situations or physical reasons. Traditional intelligent panels lack effective noise reduction and data analysis mechanisms when processing interactive data. The hospital environment is often accompanied by various background noises and interference factors, such as the operating sound of medical equipment and the sound of patients talking, which will seriously affect the accuracy and clarity of voice interaction.

[0003] Related technologies also make use of image interaction data, but they are limited to basic image recognition and fail to deeply explore the contextual information and user intentions behind the image. In the control process, user instructions are often processed in isolation, ignoring the importance of the overall hospital environment data and personnel behavior data. These data are crucial for optimizing the display content and interface layout of smart panels and providing more personalized and intelligent services. However, due to the lack of an effective data integration and analysis mechanism, it is difficult for smart panels in related technologies to make adaptive adjustments based on real-time environmental changes and user needs.

[0004] Therefore, the intelligent panel control method in the relevant technology has obvious deficiencies in data processing and situational adaptability, and it is difficult to meet the high requirements of modern hospitals for intelligent management and control. Summary of the invention

[0005] In view of this, the present invention provides an intelligent panel control method based on DNN noise reduction technology to solve the problem that the intelligent panel control method in the related technology has obvious deficiencies in data processing and situational adaptability, and is difficult to meet the high requirements of modern hospitals for intelligent management and control.

[0006] The present invention provides a smart panel control method based on DNN noise reduction technology, the method comprising: in response to detecting that a smart panel to be controlled receives a user's wake-up operation, obtaining first control data and second control data, the first control data being voice interaction data and image interaction data collected by the smart panel to be controlled, and the second control data being environmental data and personnel behavior data of the entire hospital area; performing noise reduction processing on the voice interaction data using a pre-trained first convolutional neural network model to obtain noise-reduced voice interaction data; analyzing the image interaction data and the noise-reduced voice interaction data to obtain user instruction data; obtaining a context label based on the image interaction data, the noise-reduced voice interaction data, the second control data and the pre-trained second convolutional neural network model; adjusting the display content data and the interface layout data of the smart panel to be controlled based on the user instruction data and the context label; and controlling the display interaction interface of the smart panel to be controlled based on the adjusted display content data and the adjusted interface layout data.

[0007] The smart panel control method based on DNN noise reduction technology provided in this embodiment, first, by instantly responding to the user's wake-up operation, ensures that the smart panel can quickly enter the standby state and prepare to receive the user's instructions. In this way, the response speed of the system and the immediacy of user interaction are improved. Secondly, by introducing the pre-trained first convolutional neural network model, the voice interaction data is deeply denoised. In this way, the background noise in the hospital environment is effectively removed, the accuracy and clarity of voice recognition are improved, thereby enhancing the reliability and satisfaction of user voice interaction. By comprehensively analyzing the two interactive data of image and voice, the user's intentions and needs can be understood more comprehensively. In this way, not only the accuracy of user instruction recognition is improved, but also the intelligence level of the system is enhanced, so that the smart panel can respond to the user's operation more accurately. Then, by fusing multi-source data, including images, voices, environmental data and personnel behavior data, and with the help of the pre-trained second convolutional neural network model, context analysis and label generation are performed, and the change data of the hospital environment and the personnel behavior pattern data can be used in real time to provide accurate contextual basis for subsequent display content and interface layout adjustment. Afterwards, by combining user instructions and context labels, the display content and interface layout of the smart panel are adjusted individually. In this way, not only the user's immediate needs are met, but also a more intimate and convenient service experience is provided according to the current context. Finally, by controlling the smart panel to be controlled to display the adjusted interactive interface, a complete closed loop from user wake-up to command execution to interface display is achieved. In this way, the smoothness and efficiency of the entire interaction process are ensured, providing users with a more intelligent and convenient hospital service experience, and solving the problem that the smart panel control method in the relevant technology has obvious deficiencies in data processing and context adaptability, and it is difficult to meet the high requirements of modern hospitals for intelligent management and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 is a flow chart of a smart panel control method based on DNN noise reduction technology according to an embodiment of the present invention. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0011] With the rapid development of intelligent medical systems, intelligent management and control within hospitals have become an important means to improve the quality and efficiency of medical services. Traditional intelligent panel control methods mainly rely on simple button operations or limited touch interactions, which have obvious deficiencies in user experience, comprehensive information acquisition, and environmental adaptability. When receiving user instructions, traditional intelligent panels often only use a single interaction channel such as buttons or touch screens, ignoring more natural and efficient interaction methods such as voice and images. This requires users to interact with the intelligent panel in a specific operating environment and conditions, limiting the flexibility and convenience of the interaction. Especially in a complex and changeable environment such as a hospital, medical staff or patients may find it difficult to effectively operate the intelligent panel in traditional ways due to emergency situations or physical reasons. Traditional intelligent panels lack effective noise reduction and data analysis mechanisms when processing interactive data. The hospital environment is often accompanied by various background noises and interference factors, such as the operating sound of medical equipment and the sound of patients talking, which will seriously affect the accuracy and clarity of voice interaction.

[0012] Related technologies also make use of image interaction data, but they are limited to basic image recognition and fail to deeply explore the contextual information and user intentions behind the image. In the control process, user instructions are often processed in isolation, ignoring the importance of the overall hospital environment data and personnel behavior data. These data are crucial for optimizing the display content and interface layout of smart panels and providing more personalized and intelligent services. However, due to the lack of an effective data integration and analysis mechanism, it is difficult for smart panels in related technologies to make adaptive adjustments based on real-time environmental changes and user needs.

[0013] Therefore, the intelligent panel control method in the relevant technology has obvious deficiencies in data processing and situational adaptability, and it is difficult to meet the high requirements of modern hospitals for intelligent management and control.

[0014] The smart panel control method based on DNN noise reduction technology provided in this embodiment, first, by instantly responding to the user's wake-up operation, ensures that the smart panel can quickly enter the standby state and prepare to receive the user's instructions. In this way, the response speed of the system and the immediacy of user interaction are improved. Secondly, by introducing the pre-trained first convolutional neural network model, the voice interaction data is deeply denoised. In this way, the background noise in the hospital environment is effectively removed, the accuracy and clarity of voice recognition are improved, thereby enhancing the reliability and satisfaction of user voice interaction. By comprehensively analyzing the two interactive data of image and voice, the user's intentions and needs can be understood more comprehensively. In this way, not only the accuracy of user instruction recognition is improved, but also the intelligence level of the system is enhanced, so that the smart panel can respond to the user's operation more accurately. Then, by fusing multi-source data, including images, voices, environmental data and personnel behavior data, and with the help of the pre-trained second convolutional neural network model, context analysis and label generation are performed, and the change data of the hospital environment and the personnel behavior pattern data can be used in real time to provide accurate contextual basis for subsequent display content and interface layout adjustment. Afterwards, by combining user instructions and context labels, the display content and interface layout of the smart panel are adjusted individually. In this way, not only the user's immediate needs are met, but also a more intimate and convenient service experience is provided according to the current context. Finally, by controlling the smart panel to be controlled to display the adjusted interactive interface, a complete closed loop from user wake-up to command execution to interface display is achieved. In this way, the smoothness and efficiency of the entire interaction process are ensured, providing users with a more intelligent and convenient hospital service experience, and solving the problem that the smart panel control method in the relevant technology has obvious deficiencies in data processing and context adaptability, and it is difficult to meet the high requirements of modern hospitals for intelligent management and control.

[0015] According to an embodiment of the present invention, an embodiment of an intelligent panel control method based on DNN noise reduction technology is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0016] In this embodiment, a smart panel control method based on DNN noise reduction technology is provided. Figure 1 is a flow chart of a smart panel control method based on DNN noise reduction technology according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0017] Step S101, in response to detecting that the smart panel to be controlled receives a wake-up operation from the user, obtains first control data and second control data, the first control data being the voice interaction data and image interaction data collected by the smart panel to be controlled, and the second control data being the environmental data and personnel behavior data of the entire hospital area.

[0018] Specifically, the smart panel to be controlled is an intelligent device that integrates a microphone sensor, a camera sensor, and an interactive interface. It is installed in areas that require remote control, such as hospital wards, operating rooms, corridors, or halls. It can respond to user instructions, perform a series of control operations, and collect relevant information. The user activates the smart panel to be controlled in a preset way, such as a voice command "Hello, smart panel" or touching a specific button on the screen, so that it enters the working state from the standby or dormant state and is ready to receive further instructions or data from the user. The first control data includes two types of data collected by the smart panel to be controlled: voice interaction data is the command or request input by the user through voice, which is captured by the microphone of the smart panel and converted into a digital signal for subsequent processing and analysis. Image interaction data is an image or video captured by the camera on the smart panel, including the user's gestures, facial expressions, or real-time images of the surrounding environment, which is used to enhance the interactive experience or perform visual recognition. The second control data is a broader data set related to the entire hospital area, which is not directly collected through the smart panel to be controlled. The environmental data of the hospital area, such as temperature, humidity, light intensity, air quality, etc., is collected by a sensor network distributed throughout the hospital. Personnel behavior data includes the movement trajectory, stay time, interaction pattern, etc. of patients and medical staff, which are obtained through RFID tags, Wi-Fi positioning, video surveillance, etc.

[0019] Step S102: Use the pre-trained first convolutional neural network model to perform noise reduction processing on the voice interaction data to obtain noise-reduced voice interaction data.

[0020] Specifically, the pre-trained first convolutional neural network model includes: a convolutional layer, an activation function and a decoder, and step S102 includes:

[0021] Step S1021, performing short-time Fourier transform on the voice interaction data to obtain a set of voice interaction data feature matrices.

[0022] Specifically, the first convolutional neural network model pre-trained is a DNN model, which uses multiple types of data for training. The first is a speech data set, including speech data in a hospital environment. These speech data contain common speech interaction scenarios in hospitals, such as conversations between medical staff, communication between patients and medical staff, and alarms of medical equipment. These data should contain various noise backgrounds, such as the operation sound of medical equipment, the conversation sound of patients, and the footsteps in the corridor, to simulate the real hospital environment. And clean speech data, in order to train the model for effective noise reduction, a batch of clean speech data without noise interference is used as a reference or target. These clean speech data can be combined with noise data to generate training samples with noise. Noise data set: Secondly, there is noise data unique to hospitals, such as the operation sound of medical equipment such as electrocardiographs, ventilators, and ultrasound machines, as well as the conversation sound of patients and medical staff, and the footsteps in the corridor. These data are used to simulate various noise sources in the hospital environment to help the model learn how to extract clear speech signals under these noise backgrounds. In order to increase the diversity and quantity of training data, a synthetic method is also used to generate training data. Clean speech data is mixed with noise data in a certain proportion to generate speech samples with noise. The intensity and type of noise can be flexibly controlled to meet training needs. In addition to the training data, independent validation and test data sets are also used to evaluate the performance of the model. These data sets should contain speech samples and noise backgrounds that are different from the training data to ensure that the model can generalize to unseen scenarios. Short-Time Fourier Transform (STFT) is a method for analyzing non-stationary signals (local spectral characteristics. The signal is divided into frames and Fourier transforms each frame to obtain a representation of the signal's spectrum changing over time. The original voice interaction data is processed by STFT, converted from the time domain to the frequency domain, and a set of spectrum matrices containing multiple time frames is obtained, that is, a set of voice interaction data feature matrices.

[0023] Step S1022, using a convolution layer to perform a convolution operation on the voice interaction data feature matrix set to obtain a voice interaction data feature matrix graph set, and using an activation function to process the voice interaction data feature matrix graph set to obtain a nonlinear activation feature graph set.

[0024] Specifically, the obtained voice interaction data feature matrix set is input into the convolution layer of the convolutional neural network. The convolution layer slides on the feature matrix through multiple convolution kernels, performs convolution operations, extracts higher-level features, and forms a voice interaction data feature matrix graph set. The feature matrix graph set output by the convolution layer is processed through an activation function (such as ReLU), introducing nonlinear factors so that the network can learn complex data relationships and obtain a nonlinear activation feature graph set. The formula used is as follows:

[0025]

[0026] Among them, y[n,f] represents the value of the voice interaction data feature matrix output by the convolution layer at the position (n,f). The voice interaction data feature matrix is ​​a two-dimensional array, where n represents the time frame, f represents the frequency index band, k[i,j] represents the value of the convolution kernel in the convolution layer at the position (i,j), the convolution kernel is a two-dimensional array, the height and width of the convolution kernel are hyperparameters, i and j represent the index variables when the convolution kernel slides on the voice interaction data features, b represents the bias term, and ReLU is the activation function.

[0027] Step S1023, using a decoder to process the nonlinear activation feature map set to obtain noise-reduced speech interaction data.

[0028] Specifically, a set of nonlinear activated feature maps is input into the decoder. The decoder reconstructs the denoised voice interaction data by performing deconvolution or other appropriate inverse transformation operations on these feature maps. Through the above steps, the pre-trained convolutional neural network model is used to perform denoising on the voice interaction data, which can effectively reduce the interference of noise on the voice signal and improve the clarity and recognizability of the voice interaction.

[0029] Step S103: Analyze the image interaction data and the noise reduction voice interaction data to obtain user instruction data.

[0030] Specifically, step S103 includes:

[0031] Step S1031, determining user identity data and instruction content data based on the image interaction data and the noise reduction voice interaction data.

[0032] Specifically, a face recognition algorithm or a voiceprint recognition algorithm is used to extract user identity features from image or voice data, and compared with known user identity information to determine the user identity type. The user identity data includes user identity type data, and the user identity types include medical type and patient type. A natural language processing algorithm or an image recognition algorithm is used to analyze the noise reduction voice interaction data or image interaction data to extract the user's instruction intention and specific content.

[0033] Step S1032: determining the user authority data based on the user identity data and the mapping relationship between the user identity data and the user authority data.

[0034] Specifically, a mapping relationship table between user identity and authority is predefined, and the corresponding authority data is determined according to the user identity data by looking up the table.

[0035] Step S1033, obtaining user instruction data based on the user identity data, instruction content data and user authority data.

[0036] Specifically, the user identity and authority are first determined, and then combined with the content of the instruction issued by the user to form user instruction data containing complete information.

[0037] Step S104, obtaining a context label based on the image interaction data, the noise reduction voice interaction data, the second control data and the pre-trained second convolutional neural network model.

[0038] Specifically, step S104 includes:

[0039] Step S1041, extracting facial features, action features and object visual features from the image interaction data.

[0040] Specifically, facial features are specific information about a person's face extracted from image interaction data, such as facial contours, expressions, and identity tags. HOG features are combined with SVM classifiers to detect and extract facial features. Action features are features that describe the movement or posture of a person or object in an image, such as walking, running, standing, lying down, etc. Actions are detected and tracked using the optical flow method. Object visual features are identifiable features such as shape, size, color, and texture of objects in an image. The YOLO detection algorithm is used to identify objects in an image and extract their visual features.

[0041] Step S1042: extracting emotional features and intonation features from the noise-reduced speech interaction data.

[0042] Specifically, an analysis algorithm based on speech prosody, sound quality, and vocabulary content is used to identify information about the speaker's emotion and intonation from the noise-reduced speech interaction data, and perform feature extraction.

[0043] Step S1043, extracting environmental data features and personnel behavior data features from the second control data.

[0044] Specifically, the collection of environmental data and personnel behavior data in the hospital area includes temperature, humidity, lighting, access control records, patient movement trajectories, etc. Data is collected through temperature sensors, humidity sensors, access control systems, RFID tracking systems, etc. in the hospital. The collected raw data is preprocessed, and then the time series analysis algorithm is used to extract the characteristics of environmental data and personnel behavior data.

[0045] Step S1044, face features, action features, object visual features, emotion features, intonation features, environmental data features and personnel behavior data features are integrated according to timestamps to obtain event stream data.

[0046] Specifically, ensure that the timestamps of all data sources are synchronized, and then combine the extracted features into event stream data in chronological order.

[0047] Step S1045, input the event stream data into the pre-trained second convolutional neural network model to obtain a context label.

[0048] Specifically, the second convolutional neural network model is a CNN model, which is trained using a data set containing multiple context labels. After the training is completed, the model can receive event stream data as input and output corresponding context labels. Context labels include "infusion context", "rest context", "anesthesia surgery context", etc.

[0049] Step S105: adjusting the display content data and interface layout data of the smart panel to be controlled based on the user instruction data and the context label.

[0050] Specifically, step S105 includes:

[0051] Step S1051, determining whether the instruction content data and the user authority data meet preset conditions.

[0052] Specifically, the preset condition is that the instruction content data is included in the user authority data.

[0053] Step S1052, if the instruction content data and the user authority data meet the preset conditions, the first display content data and the second display content data are adjusted based on the user identity data, the instruction content data, the user authority data and the context label, and the display content corresponding to the first display content data is used to adjust the medical device parameters, and the display content corresponding to the second display content data is used to adjust the environmental parameters.

[0054] Specifically, for instruction content data related to the adjustment of medical device parameters, according to user permissions, user identity, instruction content and context labels, medical device parameters closely related to emergency treatment or current treatment needs are displayed and adjusted first. For example, in the "emergency treatment" scenario, medical device parameters related to emergency treatment, such as heart rate monitoring, ventilator settings, etc., will be highlighted, and these parameters will be adjusted to a position that is easy to access and operate. If the user enters the instruction "Increase the infusion pump flow rate to 100ml / h" and the current context label is "emergency treatment", the first display content area on the smart panel will highlight the parameter settings related to the infusion pump, and allow the user to adjust the flow rate to 100ml / h directly through the panel, and the corresponding medical device parameter adjustment interface will be added to the first display content data. For the instruction content data related to the adjustment of environmental parameters, the display layout and operability of the environmental parameters are optimized based on user permissions, user identity, instruction content and context labels. If the user enters the instruction "adjust the ward temperature to 24°C" and the current context label is "rest", the second display content area on the smart panel will display the environmental parameter control interface, allowing the user to adjust the ward temperature to 24°C. At the same time, some other environmental parameter suggestions related to rest may also be displayed, such as appropriate humidity and light intensity, and the corresponding temperature parameter adjustment interface will be added to the first display content data.

[0055] Step S1053: adjusting the interface layout data based on the adjusted first display content data and the adjusted second display content data.

[0056] Specifically, the interface layout of the smart panel is updated according to the adjusted first and second display content data, including rearranging display elements, adjusting font size, changing color scheme, etc., to ensure that users can quickly find and operate key parameters in a specific situation.

[0057] In some optional implementations, if the instruction content data and the user authority data do not meet the preset conditions, the first display content data is adjusted based on the user identity data and the user instruction data, and the second display content data is adjusted based on the user instruction data and the context label; based on the adjusted first display content data and the adjusted second display content data, the interface layout data is adjusted. The user identity data includes user identity type data, and the user identity type includes medical type and patient type. If the instruction content data and the user authority data do not meet the preset conditions, the user identity type is determined based on the user identity data; if the user identity type is medical type, the first display content data is adjusted to permission opening request operation interface data based on the user identity data and the user instruction data, and the second display content data is adjusted to medical type environment operation interface data based on the user authority data and the context label; if the user identity type is patient type, the first display content data is adjusted to unauthorized interface data based on the user identity data and the user instruction data, and the second display content data is adjusted to patient type environment operation interface data based on the user authority data and the context label.

[0058] Specifically, if the user identity type is medical type, based on the user identity data and user instruction data, the first display content data is adjusted to the permission opening request operation interface data. This interface may include a reason input box for permission application, a submit button, etc., so that medical staff can request additional operation permissions from the system. For example, medical staff try to adjust the parameters of a certain advanced medical device, but the current permissions are insufficient. At this time, the first display area of ​​the smart panel will display a permission application interface, requiring the medical staff to fill in the application reason and submit it. Based on the user permission data and the situational label, the second display content data is adjusted to the medical environment operation interface data. This interface contains environmental control options commonly used by medical staff, such as adjusting the temperature and humidity of the ward, etc., to ensure that medical staff can still effectively control the environmental parameters when they do not have the operation permissions of specific medical equipment. For example, in the "surgery preparation" scenario, even if medical staff cannot directly operate the surgical equipment, they can still adjust the environmental parameters of the operating room through the smart panel, such as lowering the temperature to reduce bacterial growth. If the user identity type is patient type, based on the user identity data and user instruction data, the first display content data is adjusted to the unauthorized interface data. This interface includes a friendly prompt message to inform the patient that the current operation authority is insufficient, and may provide guidance on getting help or contacting medical staff. For example, if the patient tries to adjust the parameters of medical equipment that is not related to his treatment, such as infusion pumps, ventilators, monitors, etc., the first display area of ​​the smart panel will display a message such as "You do not have permission to adjust this setting, please contact medical staff for help." Based on the user permission data and situational tags, the second display content data is adjusted to the patient-type environmental operation interface data. This interface contains environmental control options that patients may need, such as adjusting the brightness of the bedside lamp, calling a nurse, etc., to ensure that patients can easily control their treatment environment. For example, in the "rest and recuperate" scenario, patients can easily adjust the lights and curtains in the ward through the smart panel to create a more comfortable rest environment.

[0059] Step S106, controlling the smart panel to be controlled to display an interactive interface based on the adjusted display content data and the adjusted interface layout data.

[0060] Specifically, after adjusting the display content data and interface layout data, the two parts of data are integrated to ensure that the display content can be correctly displayed on the smart panel according to the new layout requirements. Assume that the system has adjusted the first display content data to the surgical equipment parameter monitoring interface and the second display content data to the operating room environment control interface according to the user identity (and context label), and re-arranged the size and position of these two areas.

[0061] In some optional embodiments, the method also includes: performing keyword monitoring on the instruction content data, and in response to monitoring that the instruction content data contains preset keywords, controlling the smart panel to be controlled to communicate with the department smart panel of the department corresponding to the smart panel to be controlled in real time, and displaying image interaction data and noise reduction voice interaction data on the smart panel to be controlled and the department smart panel.

[0062] Specifically, a set of preset keywords are predefined, and the keywords are related to emergency situations, specific operation requests, or scenarios that require cross-device collaboration. For example, keywords may include "emergency call", "collaborative operation", "transfer of patients", etc. Whenever a user enters an instruction through the smart panel to be controlled, the system will perform real-time keyword monitoring on the instruction content data. If any preset keyword is contained in the instruction content data, the system will immediately trigger a series of response actions. The system controls the smart panel to be controlled to establish a real-time communication connection with the department smart panel of the corresponding department. This is usually achieved through the network to ensure the instant transmission of information. After the communication connection is established, the system will synchronize the image interaction data and noise reduction voice interaction data on the smart panel to be controlled to the department smart panel in real time. At the same time, relevant data can also be received from the department smart panel and displayed on the smart panel to be controlled to achieve two-way communication. The display interface of the smart panel to be controlled and the department smart panel will be updated in real time according to the synchronized data. For example, if the keyword is "emergency call", the icon of the emergency call, the name of the called medical staff, and the related voice call interface will be displayed on both panels. To ensure that important information can be highlighted, the interface layout data is adjusted according to the new display content, and urgent or important information is placed in a more prominent position. For example, in a hospital's operating room, the surgeon inputs instructions through the operating room's smart panel to request that the patient be transferred to the ICU (intensive care unit). When inputting the instructions, the doctor used the keyword "transfer patient".

[0063] The system detected that the command contained the keyword "transfer patient". The system immediately controlled the operating room's smart panel to be controlled and established a real-time communication connection with the ICU's department smart panel. Images in the operating room, such as the patient's current vital signs monitoring screen and noise-reduced voice call data, such as the communication content between doctors and ICU medical staff, are synchronized to the ICU's department smart panel in real time. At the same time, ICU medical staff can also view the situation in the operating room through their smart panels and send back relevant information. The interfaces of the two smart panels have been adjusted accordingly, placing relevant information about emergency transfer (such as the patient's basic information, estimated arrival time, ICU's reception preparation, etc.) in the most prominent position of the interface so that medical staff can see it at a glance.

[0064] In some optional embodiments, the method also includes: in response to detecting wake-up behavior of at least two users for the same smart panel to be controlled, determining priority sorting data of at least two users, and determining a target user based on the priority sorting data, wherein the target user is the user who wakes up the smart panel to be controlled.

[0065] Specifically, the priority ranking data of each user is calculated based on the preset priority rules and the collected user data. The priority rules are weighted by factors such as user role (e.g., medical staff have higher priority than patients or family members), distance from the smart panel (users who are closer have higher priority), and historical usage records (users who use the smart panel more frequently may get higher priority). Based on the priority ranking data, the system determines the user with the highest priority as the target user. The target user is regarded as an effective wake-up person, and his subsequent operation instructions will be responded to and executed by the system.

[0066] The smart panel control method based on DNN noise reduction technology provided in this embodiment, first, by instantly responding to the user's wake-up operation, ensures that the smart panel can quickly enter the standby state and prepare to receive the user's instructions. In this way, the response speed of the system and the immediacy of user interaction are improved. Secondly, by introducing the pre-trained first convolutional neural network model, the voice interaction data is deeply denoised. In this way, the background noise in the hospital environment is effectively removed, the accuracy and clarity of voice recognition are improved, thereby enhancing the reliability and satisfaction of user voice interaction. By comprehensively analyzing the two interactive data of image and voice, the user's intentions and needs can be understood more comprehensively. In this way, not only the accuracy of user instruction recognition is improved, but also the intelligence level of the system is enhanced, so that the smart panel can respond to the user's operation more accurately. Then, by fusing multi-source data, including images, voices, environmental data and personnel behavior data, and with the help of the pre-trained second convolutional neural network model, context analysis and label generation are performed, and the change data of the hospital environment and the personnel behavior pattern data can be used in real time to provide accurate contextual basis for subsequent display content and interface layout adjustment. Afterwards, by combining user instructions and context labels, the display content and interface layout of the smart panel are adjusted individually. In this way, not only the user's immediate needs are met, but also a more intimate and convenient service experience is provided according to the current context. Finally, by controlling the smart panel to be controlled to display the adjusted interactive interface, a complete closed loop from user wake-up to command execution to interface display is achieved. In this way, the smoothness and efficiency of the entire interaction process are ensured, providing users with a more intelligent and convenient hospital service experience, and solving the problem that the smart panel control method in the relevant technology has obvious deficiencies in data processing and context adaptability, and it is difficult to meet the high requirements of modern hospitals for intelligent management and control.

[0067] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A smart panel control method based on DNN noise reduction technology, characterized in that: The method comprises: In response to detecting that the smart panel to be controlled receives a wake-up operation from the user, first control data and second control data are acquired, wherein the first control data is voice interaction data and image interaction data collected by the smart panel to be controlled, and the second control data is environmental data and personnel behavior data of the entire hospital area; Performing noise reduction processing on the voice interaction data using a pre-trained first convolutional neural network model to obtain noise-reduced voice interaction data; Analyze the image interaction data and the noise reduction voice interaction data to obtain user instruction data; Obtaining a context label based on the image interaction data, the noise reduction voice interaction data, the second control data, and a pre-trained second convolutional neural network model; Adjusting the display content data and interface layout data of the smart panel to be controlled based on the user instruction data and the context label; Controlling the intelligent panel to be controlled to display an interactive interface based on the adjusted display content data and the adjusted interface layout data; The pre-trained first convolutional neural network model includes: a convolutional layer, an activation function and a decoder, and the pre-trained first convolutional neural network model is used to perform noise reduction processing on the voice interaction data to obtain noise-reduced voice interaction data, including: Performing short-time Fourier transform on the voice interaction data to obtain a voice interaction data feature matrix set; Using a convolution layer to perform a convolution operation on the voice interaction data feature matrix set to obtain a voice interaction data feature matrix graph set, and using an activation function to process the voice interaction data feature matrix graph set to obtain a nonlinear activation feature graph set; Processing the nonlinear activation feature map set using a decoder to obtain noise-reduced speech interaction data; The analyzing the image interaction data and the noise reduction voice interaction data to obtain the user instruction data includes: Determine user identity data and instruction content data based on the image interaction data and the noise reduction voice interaction data; Determine the user authority data based on the user identity data and the mapping relationship between the user identity data and the user authority data; Obtaining user instruction data based on the user identity data, the instruction content data and the user authority data; The obtaining of a context label based on the image interaction data, the noise reduction voice interaction data, the second control data and a pre-trained second convolutional neural network model includes: Extracting facial features, action features and object visual features from the image interaction data; Extract emotional and intonation features from noise-reduced speech interaction data; Extracting environmental data features and personnel behavior data features from the second control data; The facial features, action features, object visual features, emotional features, intonation features, environmental data features and personnel behavior data features are integrated according to the timestamp to obtain event stream data; Inputting the event stream data into a pre-trained second convolutional neural network model to obtain a context label; The adjusting the display content data and the interface layout data of the smart panel to be controlled based on the user instruction data and the context label includes: Determining whether the instruction content data and the user authority data meet preset conditions; If the instruction content data and the user authority data meet the preset conditions, the first display content data and the second display content data are adjusted based on the user identity data, the instruction content data, the user authority data and the context label, the display content corresponding to the first display content data is used to adjust the medical device parameters, and the display content corresponding to the second display content data is used to adjust the environmental parameters; Adjusting the interface layout data based on the adjusted first display content data and the adjusted second display content data; The method further comprises: The instruction content data is monitored for keywords, and in response to detecting that the instruction content data contains preset keywords, the smart panel to be controlled is controlled to communicate with the department smart panel of the department corresponding to the smart panel to be controlled in real time, and the image interaction data and the noise reduction voice interaction data are displayed on the smart panel to be controlled and the department smart panel.

2. The method according to claim 1, characterized in that The adjusting the display content data and the interface layout data of the smart panel to be controlled based on the user instruction data and the context label includes: If the instruction content data and the user authority data do not meet the preset conditions, adjusting the first display content data based on the user identity data and the user instruction data, and adjusting the second display content data based on the user instruction data and the context label; The interface layout data is adjusted based on the adjusted first display content data and the adjusted second display content data.

3. The method according to claim 1, characterized in that The user identity data includes user identity type data, and the user identity type includes medical type and patient type. If the instruction content data and the user authority data do not meet the preset conditions, the first display content data is adjusted based on the user identity data and the user instruction data, and the second display content data is adjusted based on the user authority data and the context label, including: If the instruction content data and the user authority data do not meet the preset conditions, determining the user identity type based on the user identity data; If the user identity type is medical care type, adjusting the first display content data to permission opening request operation interface data based on the user identity data and the user instruction data, and adjusting the second display content data to medical care type environment operation interface data based on the user permission data and the context label; If the user identity type is patient type, the first display content data is adjusted to unauthorized interface data based on the user identity data and the user instruction data, and the second display content data is adjusted to patient type environment operation interface data based on the user authority data and the context label.

4. The method according to claim 1, characterized in that The method further comprises: In response to detecting at least two users' wake-up actions on the same smart panel to be controlled, priority ranking data of the at least two users is determined, and a target user is determined based on the priority ranking data, where the target user is the user who wakes up the smart panel to be controlled.

Images