Health and Safety Monitoring Method and Device for Medical and Nursing Robots, and Electronic Equipment

Abstract

The present disclosure provides a health and safety monitoring method, device, and electronic device for a medical and elderly care robot, belonging to the technical field of health and safety monitoring. The method includes: monitoring a target person based on a first radar to obtain first point cloud data; determining the state of the target person based on the first point cloud data; and controlling the robot based on the state of the target person. Wherein, the first radar is used to monitor the state of the target person in the target area, and the first point cloud data is the state data of the target person. The health and safety monitoring method, device, and electronic device for a medical and elderly care robot provided by the present disclosure can improve the stability and reliability of safety monitoring.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of health and safety monitoring, and more specifically, relates to a health and safety monitoring method, device, and electronic device for medical and elderly care robots. Background Art

[0002] In current society, the trend of population aging is significant, and at the same time, people's attention to health and safety is increasing day by day. The health and safety monitoring of individuals, especially vulnerable groups such as the elderly and patients, has become a key issue that urgently needs to be solved.

[0003] Traditional monitoring methods have obvious shortcomings and limitations in the process of health and safety monitoring of monitored personnel by medical and elderly care robots. Medical and elderly care robots can obtain three-dimensional point cloud data through radar and then judge the state of the target person. However, once the target person exceeds the monitoring range or makes abnormal actions, false alarms are likely to occur, and the monitoring effect will be affected, resulting in low accuracy and reliability of the monitoring of the target person.

[0004] Therefore, an accurate and reliable health and safety monitoring method. Summary of the Invention

[0005] The purpose of the present disclosure is to provide a health and safety monitoring method, device, and electronic device for medical and elderly care robots to improve the accuracy and reliability of safety monitoring.

[0006] In the first aspect of the embodiments of the present disclosure, a health and safety monitoring method for a medical and elderly care robot is provided, including:

[0007] Monitoring a target person based on a first radar to obtain first point cloud data;

[0008] Determining the state of the target person based on the first point cloud data;

[0009] Controlling the robot based on the state of the target person;

[0010] Wherein, the first radar is used to monitor the state of the target person in the target area, and the first point cloud data is the state data of the target person.

[0011] In the second aspect of the embodiments of the present disclosure, a health and safety monitoring device for a medical and elderly care robot is provided, including:

[0012] A monitoring module for monitoring a target person based on a first radar to obtain first point cloud data;

[0013] A state determination module for determining the state of the target person based on the first point cloud data;

[0014] A control module for controlling the robot based on the state of the target person;

[0015] Among them, the first radar is used to monitor the status of target personnel in the target area, and the first point cloud data is the status data of the target personnel.

[0016] In a third aspect of the embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above-mentioned health and safety monitoring method for the medical care robot are implemented.

[0017] The beneficial effects of the health and safety monitoring method, device, and electronic device for the medical care robot provided by the embodiments of the present disclosure are as follows:

[0018] The present disclosure monitors the target personnel through the first radar, can obtain the point cloud data of the target personnel in real time, and can continuously and accurately collect data without interfering with the natural behavior of the target personnel. The present disclosure accurately identifies the status of the target personnel through the first point cloud data, and executes different actions according to the status of the target personnel, improving the accuracy and reliability of safety monitoring. The present disclosure processes the data monitored by the first radar through the random forest algorithm to obtain multiple recognition results, and determines whether to activate the second radar to monitor the target personnel according to the difference degree of the multiple recognition results, so that the present disclosure activates the second radar when the single radar cannot monitor the target personnel, improving the accuracy and reliability of health and safety monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic flow chart of the health and safety monitoring method for the medical care robot provided by an embodiment of the present disclosure;

[0021] Figure 2 It is a structural block diagram of the health and safety monitoring device for the medical care robot provided by an embodiment of the present disclosure;

[0022] Figure 3 It is a schematic block diagram of the electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art should clearly understand that the present disclosure can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present disclosure.

[0024] To make the objectives, technical solutions, and advantages of the present disclosure clearer, the following will be described through specific embodiments in conjunction with the accompanying drawings.

[0025] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a health and safety monitoring method for a medical care robot provided in an embodiment of the present disclosure. The method includes:

[0026] S101: Monitor a target person based on a first radar to obtain first point cloud data.

[0027] In this embodiment, the first radar can be a radar at a fixed position, which can monitor the status of a target person within a preset area (i.e., the target area). The status can be whether the target person has fallen, and the installation position can be a location such as a wall or ceiling of the target area that is conducive to data collection.

[0028] The target person refers to the person being monitored. For example, it can be an elderly person in a nursing home or a patient undergoing exercise rehabilitation in a hospital, etc., who needs to closely monitor their physical status and whether they have fallen or other actions. The target area can be a living room in a nursing home or a ward, etc.

[0029] The first point cloud data refers to the data obtained by monitoring the target person based on the first radar, which can include the position information and action information of the target person in space. This set of points is called the first point cloud data, and the first point cloud data can intuitively present the shape and position distribution of the target person in space.

[0030] It should be noted that the first point cloud data specifically refers to the point cloud data of the target person. In the actual application process, when using the first radar to monitor the target person, the obtained data includes the target person and the environment around the target person. Therefore, before obtaining the first point cloud data, the background point cloud data can be eliminated. For example: by collecting and modeling the background point cloud data when there is no human activity, and in the subsequent monitoring process, comparing the real-time obtained point cloud data with the background model, subtracting the background point cloud, and only retaining the point cloud data that may belong to moving targets (such as humans). The background model can be updated regularly to adapt to environmental changes, such as the movement of furniture, etc.

[0031] Alternatively, a target tracking algorithm can be adopted. For example, track the identified human target. In consecutive frames, determine the motion trajectory of the same human target by matching the feature and position information of the point cloud data. If it is found that the motion trajectory of a certain point cloud data does not conform to the law of human motion or loses association with the previously tracked human target, it can be determined as the motion data of other objects and excluded. The process of eliminating the environmental point cloud data to finally obtain the first point cloud data will not be elaborated in this article.

[0032] S102: Determine the state of the target person based on the first point cloud data.

[0033] In this embodiment, as can be seen from the above content, the first point cloud data can reflect the form and position distribution of the target person in space. The state can be whether the person has fallen or other preset abnormal behaviors. In this application, the case of falling is taken as an example for illustration, and it should also be understood that this application is not limited to the monitoring of falling behaviors.

[0034] In this embodiment, the first point cloud data can be input into the target neural network model, and the state of the target person, such as whether the person has fallen, can be obtained according to the target neural network model.

[0035] It should be noted that the target neural network model should be trained with a large amount of point cloud data and the corresponding personnel state information data.

[0036] S103: Control the robot based on the state of the target person; wherein, the first radar is used to monitor the state of the target person in the target area, and the first point cloud data is the state data of the target person.

[0037] In this embodiment, different control strategies or control methods can be set according to the different states of the target person. For example, when it is monitored that the state of the target person is an abnormal state, at this time, the robot can be controlled to give an audible and visual alarm and send the information of the person falling to the relevant devices. The relevant devices can be the mobile phones of the nursing staff in the nursing home or the mobile phones of the nurses in the hospital or the mobile phones of the family members. When it is monitored that the state of the target person is frequently in a pre-abnormal state, no alarm may be given, and only this situation is sent to the relevant devices. The robot in this application can also obtain the health information of the target person through the health monitoring device worn by the target person, such as monitoring the heart rate through the worn smart bracelet, and can also send the monitored health data to the relevant devices together.

[0038] The pre-abnormal state refers to the state that is about to be determined as an abnormal state. For example, it is the state where the target person is tripped but immediately stabilizes before falling or the abnormal state caused by the physical reason of the target person.

[0039] As can be seen from the above, the present disclosure monitors the target person through the first radar, can obtain the point cloud data of the target person in real time, and can continuously and accurately collect data without disturbing the natural behavior of the target person. The present disclosure accurately identifies the state of the target person through the first point cloud data, and performs different actions according to the state of the target person, improving the accuracy and reliability of safety monitoring.

[0040] In an embodiment of the present disclosure, the health and safety monitoring method for a medical and elderly care robot further includes:

[0041] Making a decision on the first point cloud data based on the first random forest model to obtain the difference degrees of multiple recognition results;

[0042] In response to the difference degrees of multiple recognition results being greater than or equal to the first difference degree, monitoring the target person through the second radar to obtain second point cloud data, where the second radar is a radar built in the robot;

[0043] Determining target point cloud data based on the first point cloud data and the second point cloud data;

[0044] Determining the state of the target person based on the target point cloud data.

[0045] In this embodiment, the first random forest model is a model based on the random forest algorithm. The random forest is an ensemble learning method composed of multiple decision trees, and makes a decision by comprehensively considering the results of multiple decision trees. The first random forest model here is used to analyze and process the first point cloud data to identify the state of the target person, and the decision trees in the random forest algorithm are relatively independent.

[0046] The first random forest model is trained with a large amount of point cloud data and its corresponding personnel status information data. It should be noted that the datasets used by the first random forest model and the above-mentioned first neural network model can be the same, that is, the data can be the same, but the training processes of the first neural network model and the first random forest model are different, and the outputs are also different. The first neural network model outputs a single recognition result, while this application considers that multiple recognition results can be output by each decision tree in the first random forest model.

[0047] The difference degree can be judged according to the similarity of multiple recognition results, or it can also be determined according to whether multiple recognition results are consistent. The similarity can be calculated by the cosine similarity formula or the Pearson correlation coefficient formula. The difference degree can be obtained by subtracting the similarity value from 1, or a mapping table between similarity and difference degree can be preset in advance.

[0048] Alternatively, and more simply, since the essence of the random forest algorithm model is that each decision tree votes, the degree of difference can be determined based on whether the voting results are consistent. For example, if the voting results of all decision trees are completely consistent, that is, they all believe that the target person is in a certain specific state (for example, they all believe that the target person is walking normally), then the degree of difference can be determined to be 0 because there is no different judgment at this time and the consistency of the results is the highest.

[0049] If there are some decision trees with voting results different from those of other decision trees, then the degree of difference is greater than 0. The specific value of the degree of difference can be determined according to the proportion of the number of different voting results to the total number of voting results. For example, assume that there are 100 decision trees in the first random forest model, among which 80 decision trees believe that the target person is in state A and 20 decision trees believe that the target person is in state B. Then the degree of difference can be determined to be 0.2 (that is, the proportion of the number of voting results other than the data with the most votes in the total number of voting results) to represent the degree of difference between these recognition results.

[0050] If the degree of difference is greater than or equal to the first degree of difference, it means that the first radar cannot effectively monitor the target person. The first degree of difference can be set according to experience or the requirements of the user.

[0051] Considering that the first radar is a fixed radar with a limited monitoring range, which is only limited to monitoring the target person in the target area. And due to wall obstruction or other reasons in actual application scenarios such as nursing homes, hospitals, etc., the first radar may not be able to effectively identify the state of the target person. Therefore, the second radar can be enabled when the first radar cannot effectively monitor the target person.

[0052] The second radar is an in-built radar of the robot, that is, the second radar can be mobile. Considering that the power of the radar is relatively high and the heat generation is relatively serious, the second radar does not need to be enabled when the first radar can effectively monitor the target person. The robot can follow or rotate around the target person within a certain area. However, it should be noted that the monitoring area of the second radar in-built in the robot should have a certain overlap with the monitoring area of the first radar (i.e., the target area) to better complete the monitoring of the target person.

[0053] The second point cloud data is essentially the same as the first point cloud data, and the difference lies in the different positions of the monitoring radars. At this time, since both the first point cloud data and the second point cloud data exist simultaneously, the two data should be weighted and calculated or combined in other ways to obtain a final point cloud data, that is, the target point cloud data. The state of the target person can be determined in the same way using the first neural network model because the target point cloud data and the first point cloud data are both essentially point cloud data, and the training data of the first neural network model is the point cloud data and the corresponding states.

[0054] As can be seen from the above, the present disclosure makes decisions on the first point cloud data through the first random forest model, which can make full use of the advantages of the ensemble learning of the random forest algorithm, combine the output results of multiple decision trees, and effectively reduce the recognition errors that may be brought by a single decision tree. When the difference degrees of multiple recognition results are relatively large, the second radar is enabled for supplementary monitoring, enabling this embodiment to automatically cope with special monitoring environments, such as wall blockages, etc., thereby improving the accuracy and robustness of safety monitoring.

[0055] In an embodiment of the present disclosure, determining the target point cloud data based on the first point cloud data and the second point cloud data includes:

[0056] Making decisions on the state feature points in the first point cloud data based on the second random forest model to obtain multiple state feature point sets, where each state feature point set includes multiple state feature points;

[0057] Determining the expanded basic points based on the multiple state feature point sets;

[0058] Determining the expanded basic data based on the expanded basic points;

[0059] Taking the expanded basic data or the data obtained by expanding the expanded basic data as the third point cloud data;

[0060] Performing weighted calculation on the third point cloud data and the second point cloud data to obtain the target point cloud data.

[0061] In this embodiment, the second random forest model is trained using a data set composed of a large amount of point cloud data and its corresponding state feature point data. State feature points refer to the key data points that can reflect the state of the target person. These points contain important information related to the state of the target person. For example, for the action of monitoring whether a person has fallen, the state feature points can be the human head, feet, human joint points, center of gravity points, etc. These points are also the points used for data expansion or the points to be expanded later.

[0062] Based on the above description, there are multiple state feature points, and the set composed of all state feature points is the state feature point set. Each decision tree in the random forest can output a state feature point set. It is possible to determine which state feature points are relatively accurate and which are relatively fuzzy by analyzing whether the state feature points in the state feature point sets output by each decision tree are consistent. For example, for a random forest composed of 100 decision trees, 50 of its output results are [A1, A2, A3, A4], 30 are [A1, A2, A3, A5], and 20 are [A1, A2, A6, A5], where A1, A2, A3, A4, A5, and A6 are the identified state points, and the preset threshold is 75. Then the data with more than 75 identical ones is the expansion base point. So in the above results, A1, A2, and A3 are the expansion base points. That is, when the number of state feature points in all state feature point sets is greater than the first preset number, this state feature point is defined as the expansion base point.

[0063] The expansion base data is the data determined based on the expansion base points. For example, if the expansion base point is the head of the target person, the expansion base data should be the point cloud data around the expansion base point. The expansion base data can be the point cloud data of the head and neck parts. This is because for the same person, once the position and coordinates of their head are determined, the position and coordinates of their neck part are also basically fixed. The expansion base data is the point cloud data around the expansion base point. It should be noted that the expansion base data corresponding to each expansion base point should be different, and the expansion rule can be pre-trained and set according to the various physical indicators of the target person. The above steps are all carried out for the first point cloud data obtained by the first radar monitoring.

[0064] Considering that when the number of expansion base points is small, the reliability of the expanded point cloud data will be correspondingly reduced. Therefore, in this application, the expansion base data or the data obtained by expanding the expansion base data can be used as the third point cloud data. And the third point cloud data and the second point cloud data are weighted and calculated.

[0065] Specifically, in an embodiment of the present disclosure, using the expansion base data or the data obtained by expanding the expansion base data as the third point cloud data includes:

[0066] In response to the number of expansion base points being less than the total number of state feature points of the preset ratio, using the expansion base data as the third point cloud data;

[0067] In response to the number of expansion base points being greater than or equal to the total number of state feature points of the preset ratio, expanding the data in the area to be expanded based on the second point cloud data and the expansion base points to obtain the expansion data;

[0068] Determine the third point cloud data based on the augmented basic data and the augmented data.

[0069] In this embodiment, when the number of augmented basic points is less than the total number of state feature points of the preset ratio, it indicates that even if the data is augmented, the reliability of the obtained point cloud data will be affected to a certain extent. Therefore, at this time, the data augmentation is no longer performed, and only the data around the augmented basic points (augmented basic data) is used as the third point cloud data, and then weighted calculation is performed with the second point cloud data. For example, if the augmented basic points are only the head of the target person, and the augmented basic data is the point cloud data of the head and neck of the target person, then the third point cloud data is only the point cloud data of the head and neck of the target person, and the point cloud data of other parts is neither augmented nor weighted calculated.

[0070] When the number of augmented basic points is greater than or equal to the total number of state feature points of the preset ratio, it indicates that it is still possible to consider augmenting the point cloud data of the area to be augmented based on the augmented basic points determined in the first point cloud data, and use the augmented data as the third point cloud data, and perform weighted calculation with the second point cloud data.

[0071] Similarly, there are also the same state feature points in the second point cloud data as in the first point cloud data. The relative positions of the state feature points in the second point cloud data can be used as the augmented basis and augmentation logic in the first point cloud data, that is, the relative position relationship of each state feature point is obtained through the second point cloud data, and then the points to be augmented in the area to be augmented are determined according to the relative position relationship and the augmented basic points in the first point cloud data. The points to be augmented refer to the state feature points except the augmented basic points. Then, the data around the points to be augmented is obtained according to the points to be augmented, and the method is the same as the method of determining the augmented basic data according to the augmented basic points above. For example, if the points to be augmented are the head, then the area to be augmented is the head and neck.

[0072] Through the above process, the third point cloud data can be obtained. That is, there are two cases for the third point cloud data. If the number of augmented basic points is less than the total number of state feature points of the preset ratio, the third point cloud data is the augmented basic data. If the number of augmented basic points is greater than or equal to the total number of state feature points of the preset ratio, the third point cloud data is the complete point cloud data after augmentation.

[0073] It can be concluded from the above that the present disclosure determines the augmented basic points by analyzing the state feature point sets output by multiple decision trees, which represent reliable and representative state features. Further, based on the augmented basic data determined by the augmented basic points, the third point cloud data is determined, and weighted calculation is performed with the second point cloud data, so that the target point cloud data can comprehensively reflect the monitoring results of the two radars, improving the accuracy and reliability of safety monitoring.

[0074] In one embodiment of the present disclosure, calculating a weighted sum of the third point cloud data and the second point cloud data to obtain target point cloud data includes:

[0075] In response to the third point cloud data being augmented basic data, determining a first weight adjustment step based on the importance level of the augmented basic data;

[0076] Alternatively, in response to the third point cloud data being data obtained by augmenting the augmented basic data, determining a first weight adjustment step based on the number of augmented basic points;

[0077] Adjusting a first weight reference value based on the first weight adjustment step to obtain a first weight; adjusting a second weight reference value based on the first weight adjustment step to obtain a second weight;

[0078] Performing weighted calculation based on the first weight, the second weight, the third point cloud data, and the second point cloud data to obtain target point cloud data;

[0079] The first weight is the weight corresponding to the third point cloud data, the second weight is the weight corresponding to the second point cloud data, and the adjustment directions of the first weight reference value and the second weight reference value are different.

[0080] In this embodiment, considering that there are two cases for the third point cloud data, different control strategies are adopted for the above two cases in this embodiment.

[0081] When the third point cloud data is augmented basic data, that is, incomplete data, at this time, the first weight adjustment step can be determined according to the importance level of the augmented basic data. The importance level of the augmented basic data refers to the importance level during fall monitoring (abnormality monitoring). The importance level of the augmented basic data can be equal to the importance level of the augmented basic points. The importance level of the augmented basic points can be preset. For example, during the process of fall monitoring, the importance level of the head is relatively high, which can be 4, then its corresponding importance level should be the highest. The importance level of the feet is relatively low, which can be 2, then its corresponding importance level can be appropriately reduced.

[0082] The first weight adjustment step can be determined according to the mapping table of the importance level of the augmented basic data and its corresponding first weight adjustment step, as shown in Table 1:

[0083] Table 1 Mapping table of the importance level of the augmented basic data and its corresponding first weight adjustment step

[0084]

[0085] It should be noted that the data in the above table has positive and negative signs, and the above positive and negative signs should be carried out during subsequent calculations. A positive number indicates that the subsequent adjustment of the first weight reference value is an increase, and a negative sign indicates that the subsequent adjustment of the first weight reference value is a decrease. This is because when the expanded basic data is accurate data determined by the second random forest model, and the greater the importance level of the expanded basic data, the weight should be appropriately increased during the weighted calculation. On the contrary, as the importance level decreases, the corresponding weight during the weighted calculation should also be appropriately decreased. The first weight reference value is the weight corresponding to the third point cloud data, and the second weight reference value is the weight corresponding to the second point cloud data. The sum of the finally adjusted first weight and second weight should be 1, so the adjustment directions of the first weight reference value and the second weight reference value are different.

[0086] When the third point cloud data is the data expanded from the expanded basic data, the first weight adjustment step size should be determined based on the number of expanded basic points. This is because the more the number of expanded basic points, the higher the reliability of the third point cloud data expanded from it, and the first weight adjustment step size can be determined based on the first formula. First formula: , where is the first weight adjustment step size, is the basic step size, is the proportionality coefficient, is the number of expanded basic points. The basic step size can be determined based on experience or usage preferences. For example, it can be 0.05, and the proportionality coefficient can be adjusted according to the actual situation. For example, it can be 0.01.

[0087] As the number of expanded basic points increases, the first weight adjustment step size also increases. The adjustment direction of the first weight reference value should be an increase, that is, as the number of expanded basic points increases, the reliability of the third point cloud data increases, and its proportion weight in the weighted calculation should be greater. Correspondingly, the reference value of the second weight should be decreased by the same step size. However, it should be noted that since the major premise of the above steps is that the first radar monitoring is inaccurate and the second radar is turned on, even the expanded third point cloud data has a reliability less than that of the second point cloud data. Therefore, when setting the above weight step size and reference value, it should be ensured that the first weight is less than the second weight. The first weight reference value and the second weight reference value can be determined based on experience.

[0088] As can be seen from the above, according to different situations of the third point cloud data, the present disclosure adopts different weight adjustment strategies to ensure the rationality and accuracy of weights in the weighted calculation process. When the third point cloud data is the expanded basic data, the weight adjustment step size is determined by the importance level of the expanded basic data. The method takes into account the importance differences of different parts in the monitoring, making the weighted result more in line with the actual needs. When the third point cloud data is the data after expansion, the weight adjustment step size is based on the number of expanded basic points, reflecting the positive impact of data expansion on improving data reliability, thereby ensuring the reasonable distribution of weights in the weighted calculation.

[0089] In an embodiment of the present disclosure, determining the state of the target person based on the target point cloud data includes:

[0090] In response to the occurrence of abnormal features and / or pre-abnormal features in the first point cloud data, increasing the monitoring frequency of the first radar based on the first frequency step size;

[0091] In response to the occurrence of abnormal features and / or pre-abnormal features in the target point cloud data, increasing the monitoring frequency of the first radar and the monitoring frequency of the second radar based on the first frequency step size;

[0092] Determining the state of the target person based on adjacent frame data in the target point cloud data.

[0093] In this embodiment, the abnormal feature refers to an abnormal behavior of the target person, such as the feature reflected in the point cloud data when falling, for example, it can be that the height difference between the head and the feet reflected in the point cloud data is less than a certain threshold, or the feature of the action of the arm propping down, etc., which can be set based on a large amount of falling behavior data. The pre-abnormal feature refers to the feature that is about to be determined as an abnormal feature, corresponding to the pre-abnormal state in S103. The pre-abnormal feature is the data feature for determining the pre-abnormal state. For example, it is the data feature in the point cloud data of the action that the target person is tripped but immediately stabilizes before falling. Whether there are abnormal features or pre-abnormal features in the target point cloud data can be identified by the second neural network model, and the second neural network model is a model trained with a large amount of normal point cloud data, point cloud data with abnormal features, and point cloud data with pre-abnormal features.

[0094] If abnormal features and / or pre-abnormal features appear in the target point cloud data, the monitoring frequency of the currently activated radar should be increased by the same step size, and the target person should be monitored more closely based on the adjusted monitoring frequency. It should be noted that the initially set monitoring frequencies of the first radar and the second radar should be the same, and the first frequency step size should also be the same, because only in this way can the data of the two radars be effectively expanded and weighted calculated. The first frequency step size can be determined according to the actual scenario or experiment.

[0095] In this embodiment, the status of the target person can be determined based on the data of adjacent frames in the target point cloud data. This is because the target person's action features such as squatting, tying shoelaces, and going to the toilet are easily judged as abnormal features or pre-abnormal features, so whether an abnormality or pre-abnormality has actually occurred can be judged according to the following method.

[0096] In one embodiment of the present disclosure, determining the state of a target person based on adjacent frame data in the target point cloud data includes:

[0097] In response to the similarities of a first number of consecutive adjacent frame data in the target point cloud data being less than a first similarity, and an abnormal feature appearing in the target point cloud data, the state of the target person is an abnormal state;

[0098] In response to the similarities of a first number of consecutive adjacent frame data in the target point cloud data being less than a first similarity, and pre-abnormal features appearing in the target point cloud data, and no abnormal features appearing, the state of the target person is a pre-abnormal state.

[0099] In this embodiment, considering that squatting or shoe tying is a relatively slow process, the posture changes in the multi-frame data are relatively gentle and have a certain degree of continuity; while falling often happens suddenly, and the posture changes in the continuous multi-frames are more drastic and sudden. By analyzing the similarity of the point cloud data in the multi-frame data, the two situations can be distinguished more accurately.

[0100] The similarity of two point cloud data can be calculated based on the Hausdorff distance method. The principle is to calculate the maximum value of the two-way closest distance between all points in the two frames of point cloud, which measures the maximum distance from a point in one point cloud set to another point cloud set, and can reflect the overall difference between the two frames of point cloud. It will not be repeated in this application. The first quantity and the first similarity can be determined based on experience and actual usage scenarios. When both abnormal features and pre-abnormal features appear in the target point cloud data, the abnormal features shall prevail, because the pre-abnormal features generally appear before the abnormal state features.

[0101] In one embodiment of the present disclosure, controlling a robot based on the state of a target person includes:

[0102] In response to the target person being in an abnormal state, controlling the robot to issue a first-level alarm;

[0103] In response to the frequency of the target person appearing in the pre-abnormal state exceeding the first frequency, the robot is controlled to issue a secondary alarm.

[0104] In this embodiment, similar to the explanation of S103, different control methods can be set according to the different states of the target person. When an abnormal state occurs, the robot is controlled to give an audible and visual alarm and send the information of the person falling to relevant devices, which is the first-level alarm. When it is detected that the state of the target person is frequently in a pre-abnormal state (that is, the frequency of the target person appearing in the pre-abnormal state exceeds the first frequency), no alarm may be given, and only this situation is sent to relevant devices, which is the second-level alarm. The first frequency is a preset threshold. Relevant personnel can take different handling methods according to the received alarm information. For example, in the case of a second-level alarm, timely review and observation are carried out to check whether there are foreign objects that are likely to trip people at the location where the pre-abnormal state appears.

[0105] It can be concluded from the above that when abnormal features or pre-abnormal features appear in the first point cloud data, the present disclosure can automatically increase the monitoring frequency of the first radar based on the first frequency step, which can ensure that when potential risks appear, the state changes of the target person can be captured more quickly. If abnormal features or pre-abnormal features also appear in the target point cloud data, this embodiment further increases the monitoring frequencies of the first radar and the second radar, so as to realize all-round and closer monitoring of the target person, and improve the accuracy and reliability of safety monitoring. By analyzing the similarity of multiple consecutive frames in the target point cloud data, the present disclosure can more accurately judge the state of the target person, which helps to distinguish normal behaviors such as squatting and tying shoelaces from abnormal behaviors such as falling, and further improves the judgment accuracy of safety monitoring.

[0106] In an embodiment of the present disclosure, the health and safety monitoring method for a medical and elderly care robot further includes:

[0107] In response to the frequency of the target person appearing in the pre-abnormal state exceeding the first frequency, the initial monitoring frequency of the first radar is increased at the first compensation frequency as the monitoring frequency of the first radar, and the initial monitoring frequency of the second radar is increased at the first compensation frequency as the monitoring frequency of the second radar.

[0108] In this embodiment, considering that when a person frequently makes an action of about to fall, the preset initial monitoring frequencies of the first radar and the second radar should be appropriately increased to more frequently monitor the state information of the target person. The first compensation frequency is a preset frequency, which can be determined according to experience.

[0109] Secondly, it is also considered that due to the increase in the initial frequency of the radar, when the second neural network model detects the appearance of abnormal features and / or pre-abnormal features, the monitoring frequency of the first radar will still be increased based on the first frequency step. However, at this time, due to the increase in frequency, the number of frames will become more dense, and when judging the similarity between adjacent frames, it is more likely to be judged as a normal behavior, which may lead to misjudgment. Therefore, the similarity threshold and the first quantity should be set more strictly.

[0110] For example, in one embodiment of the present disclosure, the first similarity adjustment step size is determined based on the second formula, and the first similarity is reduced based on the first similarity adjustment step size;

[0111] The first quantity adjustment step size is determined based on the third formula, and the first quantity is increased based on the first quantity adjustment step size.

[0112] In this embodiment, the second formula may be , where is the first similarity adjustment step size, is the first adjustment coefficient, which is used to control the sensitivity of the first similarity adjustment and needs to be determined according to actual experiments and scenarios. is the first compensation frequency, is the maximum monitoring frequency preset by the system. As a reference value, it normalizes the first compensation frequency to the range of 0 - 1 to ensure the rationality and consistency of the adjustment step size.

[0113] The third formula may be , where is the first quantity adjustment step size, is the second adjustment coefficient, which is used to adjust the amplitude of the first quantity adjustment, and its value also needs to be set according to the actual situation.

[0114] From the above, it can be obtained that when the frequency of the target person's pre - abnormal state exceeds the preset first frequency in the present disclosure, the initial monitoring frequencies of the first radar and the second radar can be automatically increased at the first compensation frequency, thereby improving the timeliness and accuracy of safety monitoring. This embodiment also takes into account that after the initial frequency of the radar is increased, the similarity judgment between adjacent frames may be affected, and a more stringent similarity threshold and the first quantity setting are adopted to ensure that even when the monitoring frequency is increased, an accurate judgment of the target person's behavior can still be maintained, avoiding misjudgment problems caused by dense frames, and improving the accuracy and reliability of safety monitoring.

[0115] Corresponding to the health and safety monitoring method for the medical and elderly care robot in the above - mentioned embodiment, Figure 2 is the structural block diagram of the health and safety monitoring device for the medical and elderly care robot provided by an embodiment of the present disclosure. For the sake of convenience of description, only the parts related to the embodiment of the present disclosure are shown. Refer to Figure 2 The health and safety monitoring device 20 for the medical and elderly care robot includes: a monitoring module 21, a state determination module 22, and a control module 23.

[0116] Among them, the monitoring module 21 is used to monitor the target person based on the first radar to obtain the first point cloud data;

[0117] A status determination module 22, configured to determine the status of a target person based on first point cloud data;

[0118] A control module 23, configured to control a robot based on the status of the target person;

[0119] Wherein, a first radar is used to monitor the status of a target person in a target area, and the first point cloud data is the status data of the target person.

[0120] In an embodiment of the present disclosure, a health and safety monitoring device 20 for a medical and elderly care robot further includes: a second radar monitoring module;

[0121] The second radar monitoring module is configured to make a decision on the first point cloud data based on a first random forest model to obtain a difference degree of multiple recognition results;

[0122] In response to the difference degree of multiple recognition results being greater than or equal to a first difference degree, the second radar is used to monitor the target person to obtain second point cloud data, and the second radar is a radar built in the robot;

[0123] Determine target point cloud data based on the first point cloud data and the second point cloud data;

[0124] Determine the status of the target person based on the target point cloud data.

[0125] In an embodiment of the present disclosure, the second radar monitoring module is specifically configured to make a decision on status feature points in the first point cloud data based on a second random forest model to obtain multiple status feature point sets, where each status feature point set includes multiple status feature points;

[0126] Determine an extended base point based on multiple status feature point sets;

[0127] Determine extended base data based on the extended base point;

[0128] Use the extended base data or data obtained by expanding the extended base data as the third point cloud data;

[0129] Perform weighted calculation on the third point cloud data and the second point cloud data to obtain target point cloud data.

[0130] In an embodiment of the present disclosure, the second radar monitoring module is further specifically configured to:

[0131] In response to the number of extended base points being less than a preset ratio of the total number of status feature points, use the extended base data as the third point cloud data;

[0132] In response to the number of extended base points being greater than or equal to a preset ratio of the total number of status feature points, expand the data in the area to be expanded based on the second point cloud data and the extended base points to obtain expanded data;

[0133] Determine the third point cloud data based on the augmented basic data and the augmented data.

[0134] In one embodiment of the present disclosure, the second radar monitoring module is further specifically configured to:

[0135] In response to the third point cloud data being the augmented basic data, determine the first weight adjustment step based on the importance level of the augmented basic data;

[0136] Alternatively, in response to the third point cloud data being the data obtained by augmenting the augmented basic data, determine the first weight adjustment step based on the number of augmented basic points;

[0137] Adjust the first weight reference value based on the first weight adjustment step to obtain the first weight; adjust the second weight reference value based on the first weight adjustment step to obtain the second weight;

[0138] Perform weighted calculation based on the first weight, the second weight, the third point cloud data, and the second point cloud data to obtain the target point cloud data;

[0139] The first weight is the weight corresponding to the third point cloud data, the second weight is the weight corresponding to the second point cloud data, and the adjustment directions of the first weight reference value and the second weight reference value are different.

[0140] In one embodiment of the present disclosure, the second radar monitoring module is further specifically configured to:

[0141] In response to the appearance of abnormal features and / or pre-abnormal features in the first point cloud data, increase the monitoring frequency of the first radar based on the first frequency step;

[0142] In response to the appearance of abnormal features and / or pre-abnormal features in the target point cloud data, increase the monitoring frequency of the first radar and the monitoring frequency of the second radar based on the first frequency step;

[0143] Determine the state of the target person based on the adjacent frame data in the target point cloud data.

[0144] In one embodiment of the present disclosure, the second radar monitoring module is further specifically configured to:

[0145] In response to the similarity of consecutive first numbers of adjacent frame data in the target point cloud data being less than the first similarity, and abnormal features appearing in the target point cloud data, the state of the target person is an abnormal state;

[0146] In response to the similarity of consecutive first numbers of adjacent frame data in the target point cloud data being less than the first similarity, and pre-abnormal features appearing in the target point cloud data and no abnormal features appearing, the state of the target person is a pre-abnormal state.

[0147] In one embodiment of the present disclosure, the control module is specifically configured to:

[0148] In response to the status of the target person being an abnormal status, control the robot to perform a first-level alarm;

[0149] In response to the frequency of the target person appearing in a pre-abnormal status exceeding a first frequency, control the robot to perform a second-level alarm.

[0150] See Figure 3 , Figure 3 is a schematic block diagram of an electronic device provided in an embodiment of the present disclosure. As Figure 3 shown, the electronic device 300 in this embodiment may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The above-mentioned processors 301, input devices 302, output devices 303, and memories 304 communicate with each other through a communication bus 305. The memory 304 is used to store computer programs, and the computer programs include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. Among them, the processor 301 is configured to call the program instructions to execute the functions of each module / unit in the above-mentioned device embodiments, for example Figure 2 the functions of the modules 21 to 23 shown.

[0151] It should be understood that in the embodiments of the present disclosure, the so-called processor 301 may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0152] The input device 302 may include a touchpad, a fingerprint acquisition sensor (for acquiring the fingerprint information and the direction information of the fingerprint of the user), a microphone, etc., and the output device 303 may include a display (such as an LCD), a speaker, etc.

[0153] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A part of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0154] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure may implement the implementation manners described in the first and second embodiments of the health and safety monitoring method for the medical and elderly care robot provided by the embodiments of the present disclosure, and may also implement the implementation manner of the electronic device described in the embodiments of the present disclosure, which will not be elaborated herein.

[0155] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0156] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described electronic devices and units can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated herein.

[0157] In several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces or units, and can also be in electrical, mechanical, or other forms of connection.

[0158] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present disclosure.

[0159] In addition, the functional units in the various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0160] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A health and safety monitoring method for a medical robot, characterized in that: include: The target person is monitored based on the first radar to obtain first point cloud data; Determine the status of the target person based on the first point cloud data; controlling the robot based on the status of the target person; Wherein, the first radar is used to monitor the status of the target person in the target area, and the first point cloud data is the status data of the target person; The health and safety monitoring method for the medical robot also includes: Making a decision on the first point cloud data based on a first random forest model to obtain differences of multiple recognition results; In response to the difference between the multiple recognition results being greater than or equal to the first difference, monitoring the target person based on a second radar to obtain second point cloud data, where the second radar is a built-in radar of the robot; Determine target point cloud data based on the first point cloud data and the second point cloud data; Determine the status of the target person based on the target point cloud data; The determining the state of the target person based on the target point cloud data includes: In response to the presence of abnormal features and / or pre-abnormal features in the first point cloud data, increasing the monitoring frequency of the first radar based on a first frequency step size; In response to the presence of abnormal features and / or pre-abnormal features in the target point cloud data, increasing the monitoring frequency of the first radar and the monitoring frequency of the second radar based on a first frequency step size; The state of the target person is determined based on adjacent frame data in the target point cloud data.

2. The health and safety monitoring method for a medical robot according to claim 1, characterized in that: The determining target point cloud data based on the first point cloud data and the second point cloud data comprises: Making decisions on the state feature points in the first point cloud data based on the second random forest model to obtain a plurality of state feature point sets, wherein each state feature point set includes a plurality of state feature points; Determining an expanded basic point based on the multiple state feature point sets; Determining extended basic data based on the extended basic points; Using the expanded basic data or data obtained by expanding the expanded basic data as third point cloud data; The target point cloud data is obtained by performing weighted calculation on the third point cloud data and the second point cloud data.

3. The health and safety monitoring method for a medical robot according to claim 2, characterized in that: The step of using the expanded basic data or the data obtained by expanding the expanded basic data as the third point cloud data includes: In response to the number of the expanded basic points being less than the total number of state feature points of a preset ratio, using the expanded basic data as third point cloud data; In response to the number of the expanded basic points being greater than or equal to the total number of state feature points of a preset ratio, expanding the data of the area to be expanded based on the second point cloud data and the expanded basic points to obtain expanded data; Third point cloud data is determined based on the extended basic data and the extended data.

4. The health and safety monitoring method for a medical robot according to claim 3, characterized in that: The step of performing weighted calculation on the third point cloud data and the second point cloud data to obtain the target point cloud data comprises: In response to the third point cloud data being the extended basic data, determining a first weight adjustment step length based on an importance level of the extended basic data; Alternatively, in response to the third point cloud data being data obtained by expanding the expanded basic data, a first weight adjustment step is determined based on the number of the expanded basic points; Adjusting a first weight reference value based on the first weight adjustment step length to obtain a first weight; adjusting a second weight reference value based on the first weight adjustment step length to obtain a second weight; Perform weighted calculation based on the first weight, the second weight, the third point cloud data and the second point cloud data to obtain the target point cloud data; The first weight is a weight corresponding to the third point cloud data, the second weight is a weight corresponding to the second point cloud data, and the first weight reference value and the second weight reference value are adjusted in different directions.

5. The health and safety monitoring method for a medical robot according to claim 1, characterized in that: The determining the state of the target person based on adjacent frame data in the target point cloud data includes: In response to the similarities of a first number of consecutive adjacent frame data in the target point cloud data being less than a first similarity, and an abnormal feature appearing in the target point cloud data, the state of the target person is an abnormal state; In response to the similarities of a first number of consecutive adjacent frame data in the target point cloud data being less than a first similarity, and pre-abnormal features appearing in the target point cloud data, and no abnormal features appearing, the state of the target person is a pre-abnormal state.

6. The health and safety monitoring method for a medical robot according to claim 5, characterized in that: The controlling the robot based on the state of the target person comprises: In response to the target person being in an abnormal state, controlling the robot to issue a first-level alarm; In response to the frequency of the target person appearing in the pre-abnormal state exceeding the first frequency, the robot is controlled to issue a secondary alarm.

7. A health and safety monitoring device for a medical robot, characterized in that: include: A monitoring module, used for monitoring the target person based on the first radar to obtain first point cloud data; A status determination module, used to determine the status of the target person based on the first point cloud data; A control module, used for controlling the robot based on the state of the target person; Wherein, the first radar is used to monitor the status of the target person in the target area, and the first point cloud data is the status data of the target person; The health and safety monitoring device for the medical robot also includes: A second radar monitoring module, used for making a decision on the first point cloud data based on a first random forest model to obtain a difference between multiple recognition results; In response to the difference between the multiple recognition results being greater than or equal to the first difference, monitoring the target person based on a second radar to obtain second point cloud data, where the second radar is a built-in radar of the robot; Determine target point cloud data based on the first point cloud data and the second point cloud data; Determine the status of the target person based on the target point cloud data; a second radar monitoring module, further configured to, in response to the presence of abnormal features and / or pre-abnormal features in the first point cloud data, increase the monitoring frequency of the first radar based on a first frequency step; In response to the presence of abnormal features and / or pre-abnormal features in the target point cloud data, increasing the monitoring frequency of the first radar and the monitoring frequency of the second radar based on a first frequency step size; The state of the target person is determined based on adjacent frame data in the target point cloud data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images