Radiology room intelligent voice reminding system based on internet of things

Abstract

The application relates to the field of intelligent reminding systems, and particularly discloses a radiotherapy room intelligent voice reminding system based on the Internet of Things, which comprises an image acquisition module, an image analysis module, a sound receiving module and a voice interaction module. The image acquisition module is used for acquiring on-site image information in a radiotherapy room; the image analysis module is used for image recognition on the on-site image information, recognizing moving objects in the on-site image information according to a frame difference method, recognizing personnel images according to preset contour features, then recognizing protective clothing images according to preset recognition features, and then generating voice prompt information according to the relative position relationship between the protective clothing images and the personnel images; the sound receiving module is used for receiving sound information in an environment, and performing voice recognition on the sound information to obtain voice interaction information; and the voice interaction module is used for receiving the voice prompt information and the voice interaction information. The technical scheme of the application can accurately recognize the wearing state of the protective clothing of a user and effectively guide the user.

Description

Technical Field

[0001] This invention relates to the field of intelligent reminder systems, and in particular to an intelligent voice reminder system for radiology clinics based on the Internet of Things. Background Technology

[0002] In traditional radiology clinics, patients undergoing radiation examinations are required to don protective equipment such as lead aprons under the guidance of radiation workers. However, this direct instruction from staff is limited by the communication and understanding between the two parties, making communication breakdowns highly likely. For example, the guidance from radiation workers may vary due to differences in individual attitudes and experience, resulting in inconsistent quality of instruction. Sometimes, the instructions are unclear, affecting the patient's ability to correctly wear protective equipment. Furthermore, given the importance of radiation protection, patients are highly concerned about the adequacy of protective measures, and inappropriate instruction can easily lead to patient questions and complaints.

[0003] Although various smart devices have emerged both domestically and internationally, intelligent voice prompt systems specifically designed for radiology clinics are still relatively rare. In 2021, Huawei proposed the "HUAWEI Smart Home" solution, showcasing the trend of intelligentization in the smart home field. However, the application of this technology in the field of radiology is still in its early stages.

[0004] OpenHarmony, formerly known as Huawei HarmonyOS, is a cross-platform, multi-language distributed operating system designed for all scenarios. It is a crucial technology branch for realizing the interconnectivity of everything in the Internet of Things (IoT). Since its release in 2019, after two years of development and open-source integration, it has become an important foundation for building an independent and controllable technology ecosystem. The addition of OpenHarmony provides a new option for smart terminal operating systems and gives "HUAWEI Smart Home" a concrete implementation solution.

[0005] However, currently, the OpenHarmony operating system and the "HUAWEI Smart Home" solution are more focused on the smart home field and have not been fully applied to the radiation protection reminder scenario. In the smart home scenario, the focus is more on accurately identifying user commands and realizing intelligent control of related devices; however, in the radiation protection reminder scenario, there is currently no voice reminder system that can accurately identify the user's protective clothing wearing status and provide effective guidance. Summary of the Invention

[0006] This invention provides an intelligent voice reminder system for radiology clinics based on the Internet of Things, which can accurately identify the user's protective clothing wearing status and provide effective guidance.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0008] The IoT-based intelligent voice reminder system for radiology clinics includes:

[0009] Image acquisition module: Used to acquire on-site image information within the radiology clinic;

[0010] The image analysis module is used to perform image recognition on the scene image information. It identifies moving objects in the scene image information according to the inter-frame difference method, and identifies personnel images according to preset contour features. Then, it identifies protective clothing images according to preset recognition features. Finally, it generates voice prompt information based on the relative positional relationship between the protective clothing image and the personnel image.

[0011] The sound receiving module is used to receive sound information from the environment and perform speech recognition on the sound information to obtain voice interaction information;

[0012] The voice interaction module is used to receive voice prompts and voice interaction information, then perform human-computer interaction according to the preset human-computer interaction model, and output voice prompts during the human-computer interaction process.

[0013] Furthermore, the image analysis module is also used to preprocess the on-site image information. The preprocessing includes noise reduction using Kalman filtering, smoothing using Gaussian fusion, enhancing contrast, and adjusting brightness.

[0014] Furthermore, when the image analysis module identifies moving objects in the on-site image information using the inter-frame difference method, it identifies the moving region by comparing the differences between two consecutive frames of images, and optimizes the recognition effect of the moving region by combining the results of Kalman filtering. The detected moving region is then subjected to morphological processing, which includes dilation and erosion, to remove noise and extract a more accurate contour of the moving object.

[0015] Furthermore, the image analysis module identifies personnel images based on preset contour features. Specifically, it uses a convolutional neural network in a deep learning model to identify personnel images. This convolutional neural network is trained with a large amount of labeled data and can identify different postures and features of personnel. During the training process, Gaussian fusion is introduced to enhance the model's ability to identify image details.

[0016] Furthermore, the image analysis module is also used to separate the protective clothing image from the on-site image information through image segmentation, and analyze the relative positional relationship between the protective clothing image and the personnel image. It uses a spatial relationship analysis algorithm to determine whether the protective clothing is correctly worn on the personnel. In this process, Gaussian fusion technology is combined to optimize the edge detection capability of the image.

[0017] Furthermore, the relative positional relationship between the protective suit image and the personnel image includes situations where the protective suit is not worn, the protective suit is not fully worn, the protective suit is worn incorrectly, and the protective suit is worn correctly. The image analysis module is also used to generate a voice prompt message: "Please wear the protective suit" when there is no protective suit image in the personnel image, or when the relative position of the protective suit image and the personnel image indicates that the protective suit is not worn correctly. The image analysis module is also used to generate a voice prompt message: "Please ensure that the protective suit completely covers your body" when the protective suit image is incomplete in the personnel image. The image analysis module is also used to generate a voice prompt message: "Please wear the protective suit on the correct body part" when the protective suit image appears in the wrong position in the personnel image.

[0018] Furthermore, the image analysis module is also used to determine the positional relationship between the protective clothing image and the personnel image after detecting movement in the personnel image.

[0019] Furthermore, the image analysis module, after multiple recognitions, detects that the patient is still not wearing protective clothing correctly and generates an abnormality reminder message; the voice interaction module is also used to record the content and time of each reminder, as well as the patient's response, and when receiving an abnormality reminder message, it sends assistance processing information to a preset communication address through a preset communication method.

[0020] Furthermore, the voice interaction module is also used to collect feedback from staff providing on-site guidance, analyze the reasons why repeated reminders are ineffective, and optimize the voice prompt content and interaction logic based on the feedback results; it also includes a video display module, which is used to generate educational materials for loop playback based on the voice prompt information.

[0021] Furthermore, it also includes a radiation dose detection module and a light and humidity detection module. The radiation dose detection module is installed behind the user's position to detect radiation dose information and send it to the voice interaction module. The light and humidity detection module is used to detect the light intensity and humidity information in the radiology clinic and send it to the voice interaction module. The voice interaction module is also used to broadcast radiation dose information, light intensity and humidity information when it receives a user inquiry.

[0022] The principle and beneficial effects of this solution are as follows: In this solution, the image acquisition module is responsible for capturing high-definition real-time images within the radiology clinic, providing raw data for subsequent image analysis. Furthermore, the real-time images are not used as monitoring data and are not directly displayed; they are only used for subsequent analysis to protect the privacy of relevant personnel.

[0023] The image analysis module employs Kalman filtering for image denoising, Gaussian fusion for image smoothing, and contrast enhancement and brightness adjustment to improve image quality. Moving objects are identified using inter-frame difference, and Kalman filtering is combined to optimize the recognition of moving regions. Deep learning models, especially convolutional neural networks, are used to identify personnel images, and image segmentation techniques are used to separate the protective clothing image, analyzing its relative position to the personnel image to determine the wearing status of the protective clothing.

[0024] The sound receiving module receives ambient sound information, performs speech recognition, and extracts user voice interaction information to enable real-time interaction with the user. The voice interaction module processes voice prompts and interaction information based on a pre-defined human-computer interaction model, enabling natural language communication with the user and providing personalized voice guidance based on the user's wear status.

[0025] The video demonstration module generates educational materials and plays common mistakes in wearing protective clothing in a loop to help users learn the correct way to wear protective clothing.

[0026] This solution utilizes image processing technology to accurately identify the wearing status of protective clothing, including not wearing, incompletely worn, incorrectly worn, and correctly worn. Based on the identification results, it then generates and outputs real-time voice prompts to guide users in correctly wearing the protective clothing, thereby improving the safety of radiation testing.

[0027] The voice interaction module enables natural language communication with users, providing more personalized guidance and feedback. It continuously optimizes the recognition algorithm and voice prompts based on user feedback and wearing habits, improving the effectiveness of guidance. In cases where users repeatedly fail to wear protective clothing correctly, it can promptly notify staff for on-site guidance, ensuring that every user receives appropriate assistance. The video demonstration module plays educational materials to help users understand and master the correct way to wear protective clothing, preventing errors. Automated monitoring and reminders reduce reliance on radiation workers and improve the efficiency of radiation testing. Standardized and multilingual voice prompts increase user satisfaction and reduce repetitive operations caused by improper wearing.

[0028] In summary, the intelligent voice reminder system for radiology clinics of the present invention significantly improves the safety and efficiency of radiological examinations by accurately identifying the user's protective clothing wearing status and providing effective voice guidance, while also optimizing the user experience. Attached Figure Description

[0029] Figure 1 This is a logic block diagram of an IoT-based intelligent voice reminder system for radiology clinics. Detailed Implementation

[0030] The following detailed description illustrates the specific implementation method:

[0031] IoT-based intelligent voice reminder system for radiology clinics (e.g.) Figure 1 (As shown), including:

[0032] Image acquisition module: Used to acquire on-site image information within the radiology clinic;

[0033] The image analysis module is used to perform image recognition on the scene image information. It identifies moving objects in the scene image information according to the inter-frame difference method, and identifies personnel images according to preset contour features. Then, it identifies protective clothing images according to preset recognition features. Finally, it generates voice prompt information based on the relative positional relationship between the protective clothing image and the personnel image.

[0034] The sound receiving module is used to receive sound information from the environment and perform speech recognition on the sound information to obtain voice interaction information;

[0035] The voice interaction module is used to receive voice prompts and voice interaction information, then perform human-computer interaction according to the preset human-computer interaction model, and output voice prompts during the human-computer interaction process.

[0036] The video display module is used to generate educational materials for loop playback based on voice prompts (which can be achieved by selecting and playing preset video clips);

[0037] It also includes a radiation dose detection module and a light and humidity detection module. The radiation dose detection module is installed behind the user and is used to detect radiation dose information and send it to the voice interaction module. The light and humidity detection module is used to detect the light intensity and humidity information in the radiology clinic and send it to the voice interaction module. The voice interaction module is also used to broadcast radiation dose information, light intensity and humidity information when it receives a user inquiry.

[0038] Specifically, the image analysis module is also used to preprocess the on-site image information. The preprocessing includes noise reduction using Kalman filtering, smoothing using Gaussian fusion, enhancing contrast, and adjusting brightness.

[0039] When the image analysis module identifies moving objects in the on-site image information using the inter-frame difference method, it identifies the moving region by comparing the differences between two consecutive frames of images. It also combines the results of Kalman filtering to optimize the recognition effect of the moving region and performs morphological processing on the detected moving region. The morphological processing includes dilation and erosion to remove noise and extract a more accurate contour of the moving object.

[0040] The image analysis module identifies personnel images based on preset contour features. Specifically, it uses a convolutional neural network in a deep learning model to identify personnel images. This convolutional neural network is trained with a large amount of labeled data and can identify different postures and features of personnel. During the training process, Gaussian fusion is introduced to enhance the model's ability to identify image details.

[0041] The image analysis module is also used to separate the protective clothing image from the on-site image information through image segmentation, and to analyze the relative positional relationship between the protective clothing image and the personnel image. It uses spatial relationship analysis algorithms to determine whether the protective clothing is correctly worn on the personnel. In this process, Gaussian fusion technology is combined to optimize the edge detection capability of the image.

[0042] The relative positional relationship between the protective suit image and the personnel image includes situations where the protective suit is not worn, the protective suit is not fully worn, the protective suit is worn incorrectly, and the protective suit is worn correctly. The image analysis module is also used to generate a voice prompt message: "Please wear protective clothing" when there is no protective suit image in the personnel image, or when the relative position of the protective suit image and the personnel image indicates that the protective suit is not worn correctly. The image analysis module is also used to generate a voice prompt message: "Please ensure that the protective suit completely covers your body" when the protective suit image is incomplete in the personnel image. The image analysis module is also used to generate a voice prompt message: "Please wear the protective suit on the correct body part" when the protective suit image appears in the wrong position in the personnel image.

[0043] The image analysis module is also used to determine the positional relationship between the protective clothing image and the personnel image after detecting movement in the personnel image.

[0044] After multiple recognitions, the image analysis module detects that the patient is still not wearing protective clothing correctly and generates an abnormality reminder message. The voice interaction module is also used to record the content and time of each reminder, as well as the patient's response. When an abnormality reminder message is received, it sends assistance information to a preset communication address through a preset communication method.

[0045] The voice interaction module is also used to collect feedback from staff on-site guidance, analyze the reasons why repeated reminders are ineffective, and optimize the voice prompts and interaction logic based on the feedback results.

[0046] In practical use: Multiple high-resolution cameras are installed in the radiology clinic to ensure omnidirectional coverage for the image acquisition module. The cameras capture images from inside the clinic in real time and transmit the data to the image analysis module. Upon receiving the images, the image analysis module first applies Kalman filtering to remove noise, then performs Gaussian fusion for smoothing, enhancing contrast, and adjusting brightness to improve the accuracy of subsequent recognition. Inter-frame difference is used to compare two consecutive frames to identify moving objects. Combining the Kalman filtering results, morphological processing is performed on the moving regions, including dilation and erosion operations, to extract accurate contours of the moving objects.

[0047] Using a convolutional neural network trained on a large amount of labeled data, the system identifies personnel images in images, recognizing their different poses and features. Image segmentation techniques are used to separate the protective suit image from the background, and the relative positional relationship between the protective suit and the personnel image is analyzed to determine the wearing status of the protective suit. Based on the relative positional relationship between the protective suit and the personnel image, the system generates corresponding voice prompts, such as "Please wear the protective suit," "Please ensure the protective suit completely covers your body," or "Please wear the protective suit on the correct body part." The sound receiving module captures ambient sounds and performs speech recognition, extracting user interaction information. The voice interaction module provides real-time voice feedback based on a human-computer interaction model.

[0048] If the system detects that the user is still not wearing protective clothing correctly after multiple identifications, it will generate an abnormality alert and notify staff to provide on-site guidance.

[0049] The video demonstration module generates and loops educational materials, showing the correct way to wear protective clothing, helping users learn and correct mistakes. The radiation dose detection module and the light and humidity detection module monitor the clinic environment in real time and send the data to the voice interaction module. When a user asks a question through the voice interaction module, the system announces the current radiation dose, light intensity, and humidity information.

[0050] Suppose a patient enters the radiology office to undergo an X-ray examination. The following are the specific steps involved in the system implementation:

[0051] Once the patient enters the examination room, the camera starts working, capturing the patient's movements and condition.

[0052] The image analysis module preprocesses the acquired images and uses a deep learning model to identify patients who are not wearing protective clothing.

[0053] The system generates a voice prompt: "Please wear protective clothing," which is then played through the speaker.

[0054] The patient began to put on protective clothing after hearing the instructions, but did not put it on completely.

[0055] The system re-analyzed the image and identified that the protective suit was not fully worn, generating a new voice prompt: "Please ensure that the protective suit completely covers your body."

[0056] After several attempts, the patient still did not wear the protective clothing correctly, and the system generated an abnormal alert message.

[0057] The voice interaction module notifies the staff, who then enter the consultation room and provide on-site guidance to the patient on how to properly wear protective clothing.

[0058] While waiting, the video display module plays educational videos demonstrating the correct way to wear the protective suit. When putting on the protective suit, the patient inquires about their current radiation dose, and the system broadcasts real-time radiation dose information via the voice interaction module.

[0059] In another embodiment, in a radiology clinic, a patient fails to wear protective clothing correctly. After the image acquisition module captures the patient's image, the image analysis module, through the aforementioned preprocessing and recognition steps, determines that the protective clothing is not worn correctly. The system then generates a voice prompt to guide the patient on how to wear it correctly. If the patient fails multiple times, the system will notify staff to intervene via the voice interaction module and use the video demonstration module to play the correct wearing method, while simultaneously broadcasting current environmental information to ensure the patient receives correct guidance and is in a safe environment.

[0060] The specific implementation logic is as follows: The camera captures images in the radiology clinic in real time and transmits the data to the image analysis module. Upon receiving the image data, the image analysis module first performs preprocessing: Kalman filtering is applied to remove random noise from the image. Gaussian fusion technology is used to smooth the image, reducing detail noise while preserving edge information. Image contrast is enhanced and brightness is adjusted to adapt to image recognition under different lighting conditions. Inter-frame difference is used to compare two consecutive frames to identify moving objects in the image. The recognition effect of moving objects is optimized by combining the Kalman filtering results.

[0061] Morphological operations are performed on the detected motion regions, applying dilation to fill gaps in the object contours. Erosion is applied to remove noise attached to the object contours. A convolutional neural network (CNN) in a deep learning model is used to identify people in the images. This CNN model, trained on a large amount of labeled data, is capable of recognizing people with different poses and features.

[0062] Image segmentation technology separates the protective suit image and analyzes the relative positional relationship between the protective suit and the personnel image. Spatial relationship analysis algorithms are used to determine whether the protective suit is worn correctly. Based on the relative positional relationship between the protective suit and the personnel image, the system generates corresponding voice prompts, such as "Please wear the protective suit" and "Please ensure the protective suit completely covers your body."

[0063] The sound receiving module captures and recognizes the user's voice feedback. The voice interaction module adjusts the voice prompts based on this feedback to enable natural language communication. If the user repeatedly fails to wear the protective clothing correctly, the image analysis module generates an anomaly alert. The voice interaction module records the alert content and the user's response and notifies staff via communication methods.

[0064] The video demonstration module generates educational materials, looping common mistakes in wearing protective clothing to help users learn the correct way to wear it. Radiation dose detection and light / humidity detection modules monitor the clinic environment in real time and send the data to the voice interaction module. When users inquire, the system announces the current radiation dose, light intensity, and humidity information. Based on on-site guidance and feedback from staff and the user's wearing habits, the system continuously optimizes its recognition algorithm and voice prompts to improve accuracy and effectiveness of user guidance.

[0065] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An intelligent voice reminder system for radiology clinics based on the Internet of Things, characterized in that: include: Image acquisition module: Used to acquire on-site image information within the radiology clinic; The image analysis module is used to perform image recognition on the scene image information. It identifies moving objects in the scene image information according to the inter-frame difference method, and identifies personnel images according to preset contour features. Then, it identifies protective clothing images according to preset recognition features. Finally, it generates voice prompt information based on the relative positional relationship between the protective clothing image and the personnel image. The sound receiving module is used to receive sound information from the environment and perform speech recognition on the sound information to obtain voice interaction information; The voice interaction module is used to receive voice prompts and voice interaction information, then perform human-computer interaction according to the preset human-computer interaction model, and output voice prompts during the human-computer interaction process.

2. The intelligent voice reminder system for radiology clinics based on the Internet of Things as described in claim 1, characterized in that: The image analysis module is also used to preprocess the on-site image information. The preprocessing includes noise reduction using Kalman filtering, smoothing using Gaussian fusion, enhancing contrast, and adjusting brightness.

3. The intelligent voice reminder system for radiology clinics based on the Internet of Things as described in claim 2, characterized in that: When the image analysis module identifies moving objects in the on-site image information using the inter-frame difference method, it identifies the moving region by comparing the differences between two consecutive frames of images, and optimizes the recognition effect of the moving region by combining the results of Kalman filtering. The detected moving region is then subjected to morphological processing, which includes dilation and erosion, to remove noise and extract a more accurate contour of the moving object.

4. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 3, characterized in that: The image analysis module identifies personnel images based on preset contour features. Specifically, it uses a convolutional neural network in a deep learning model to identify personnel images. This convolutional neural network is trained with a large amount of labeled data and can identify different postures and features of personnel. During the training process, Gaussian fusion is introduced to enhance the model's ability to identify image details.

5. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 4, characterized in that: The image analysis module is also used to separate the protective clothing image from the on-site image information through image segmentation, and to analyze the relative positional relationship between the protective clothing image and the personnel image. It uses spatial relationship analysis algorithms to determine whether the protective clothing is correctly worn on the personnel. In this process, Gaussian fusion technology is combined to optimize the edge detection capability of the image.

6. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 5, characterized in that: The relative positional relationship between the protective suit image and the personnel image includes: protective suit not being worn, protective suit not being worn completely, protective suit being worn incorrectly, and protective suit being worn correctly. The image analysis module is also used to generate a voice prompt message: "Please wear protective suit" when there is no protective suit image in the personnel image, or when the relative position of the protective suit image and the personnel image indicates that the protective suit is not worn correctly. The image analysis module is also used to generate a voice prompt message: "Please ensure that the protective suit completely covers your body" when the protective suit image is incomplete in the personnel image. The image analysis module is also used to generate a voice prompt message: "Please wear the protective suit on the correct body part" when the protective suit image appears in the wrong position in the personnel image.

7. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 6, characterized in that: The image analysis module is also used to determine the positional relationship between the protective clothing image and the personnel image after detecting movement in the personnel image.

8. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 7, characterized in that: After multiple recognitions, the image analysis module will generate an abnormality alert if it finds that the patient is still not wearing protective clothing correctly. The voice interaction module is also used to record the content and time of each reminder, as well as the patient's response. When receiving abnormal reminder information, it sends assistance information to a preset communication address through a preset communication method.

9. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 8, characterized in that: The voice interaction module is also used to collect feedback from staff on-site guidance, analyze the reasons why repeated reminders are ineffective, and optimize the voice prompt content and interaction logic based on the feedback results; it also includes a video display module, which is used to generate educational materials for loop playback based on the voice prompt information.

10. The intelligent voice reminder system for radiology clinics based on the Internet of Things according to claim 9, characterized in that: It also includes a radiation dose detection module and a light and humidity detection module. The radiation dose detection module is installed behind the user and is used to detect radiation dose information and send it to the voice interaction module. The light and humidity detection module is used to detect the light intensity and humidity information in the radiology clinic and send it to the voice interaction module. The voice interaction module is also used to broadcast radiation dose information, light intensity and humidity information when it receives a user inquiry.

Images