A multimodal sensor fusion-based robot dog device safety monitoring method

Abstract

This application relates to the field of electronic digital data processing technology, specifically to a multimodal sensor fusion-based method for safety monitoring of a robotic dog device. The method includes: real-time acquisition of internal and external temperature data of the robotic dog under monitoring, as well as current data of the servo motor and pressure data of each leg of the robotic dog under monitoring, within a preset time period; dividing the time period into sub-time periods and obtaining the long-term temperature drift of each sub-time period; obtaining the current instability of each sub-time period based on the degree of change in current data within each sub-time period; obtaining the thermoelectric misalignment feedback degree of the robotic dog under monitoring; obtaining the linkage coefficient of the robotic dog under monitoring; evaluating the operating status of the robotic dog under monitoring; and performing safety monitoring of the robotic dog. This application aims to improve the accuracy of judging the operating status of the robotic dog by extracting features during its operation, thereby improving the efficiency of safety monitoring of the robotic dog device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital data processing, in particular to a multi-modal sensor fusion robot dog device safety monitoring method. BACKGROUND

[0002] The robot dog is a kind of bionic quadruped robot, which imitates the shape and action of a biological dog, can perform complex movements in complex places, and meets the shortcomings that the biological dog cannot move for a long time. It can not only carry various emergency equipment and emergency materials to supplement the emergency resources in disaster areas, but also carry various sensors to reach places where rescue personnel cannot reach, such as high temperature and small space. At the same time, the search and rescue robot dog usually embeds various search and rescue sensors to assist the rescue personnel to efficiently search and rescue the disaster area. Since the running environment of the robot dog is complex, it is very important to monitor the safety of the robot dog during operation.

[0003] Since the robot dog is in a complex environment during the search and rescue process, the collected data usually has a large amount of noise. The traditional technology pre-processes the data by filtering algorithm to remove noise and improve data quality, and then uses neural network technology to monitor the robot dog device. However, the traditional method can only achieve local smoothing, and cannot distinguish between environmental disturbances and device faults, resulting in problems such as lag and misjudgment of real-time monitoring results when the early fault characteristics such as blocked heat dissipation and stuck bearing are much smaller than the environmental noise. SUMMARY

[0004] In view of the above, it is necessary to provide a multi-modal sensor fusion robot dog device safety monitoring method. Compared with the traditional robot dog device safety monitoring method, the method extracts the features of the robot dog during operation, improves the accuracy of the robot dog operation state judgment, and improves the efficiency of the robot dog device safety monitoring.

[0005] The multi-modal sensor fusion robot dog device safety monitoring method of the present application adopts the following technical scheme:

[0006] An embodiment of the present application provides a multi-modal sensor fusion robot dog device safety monitoring method, which comprises the following steps:

[0007] Real-time collection of internal temperature data, external temperature data of the robot dog to be monitored, and current data of the servo motor of the robot dog to be monitored, and pressure data of each foot of the robot dog to be monitored within a preset period of time;

[0008] The time period is divided into sub-time periods. The long-term temperature drift of each sub-time period is obtained by analyzing the difference distribution range and trend of internal and external temperature data within each sub-time period. The current instability of each sub-time period is obtained by analyzing the degree of change of current data within each sub-time period. The thermoelectric misalignment feedback degree of the robot dog under monitoring is obtained by analyzing the synchronicity of changes between the long-term temperature drift and current instability of all sub-time periods. The asynchronous operation degree of the robot dog under monitoring is obtained by comparing the pressure data distribution of the legs during synchronous movement of the robot dog under monitoring. Combined with the thermoelectric misalignment feedback degree, the linkage coefficient of the robot dog under monitoring is obtained.

[0009] The operating status of the robot dog under monitoring is evaluated by the current data distribution, internal temperature data distribution, thermoelectric misalignment feedback degree, and linkage coefficient during the time period, so as to carry out safety monitoring of the robot dog under monitoring.

[0010] In one embodiment, the process of obtaining the long-term temperature drift is as follows:

[0011] Calculate the difference between the internal temperature data and the external temperature data at each time point, and record the ratio of the range of all the difference values ​​in each sub-period to the mean as the first ratio.

[0012] The number of positive and negative values ​​in the first-order difference results of all the difference values ​​in each sub-period is counted. The maximum value among the number of positive and negative values ​​is counted. The ratio of the maximum value to the total number of difference values ​​in each sub-period is recorded as the second ratio.

[0013] The long-term temperature drift is obtained by combining the first ratio with the second ratio.

[0014] In one embodiment, the long-term temperature drift is the product of the first ratio and the second ratio.

[0015] In one embodiment, the process of obtaining the current instability is as follows:

[0016] Obtain the absolute value of the slope of the fitted straight line of all current data in each sub-period on the time series;

[0017] The ratio of the range to the mean of all current data in each sub-period is denoted as the current ratio.

[0018] The current instability is obtained by combining the normalized result of the absolute value with the current ratio.

[0019] In one embodiment, the current instability is the product of the normalized result of the absolute value and the current ratio.

[0020] In one embodiment, the process of obtaining the thermoelectric misalignment feedback degree is as follows:

[0021] The long-term temperature drift and current instability of all sub-periods are processed by first-order difference in time series. Positive values ​​in the first-order difference results are marked as 1, negative values ​​as -1, and 0 as 0.

[0022] Calculate the temporal difference between the labeled results corresponding to the long-term temperature drift and current instability during the aforementioned time period;

[0023] Calculate the mean value of the labeling results corresponding to the long-term temperature drift and the mean value of the labeling results corresponding to the current instability within the time period, and calculate the difference between the mean values ​​of the corresponding labeling results for the long-term temperature drift and the current instability.

[0024] The ratio of the difference to the total number of sub-periods is denoted as the quantification ratio.

[0025] The thermoelectric misalignment feedback degree is positively correlated with the quantization ratio and the difference amount, respectively.

[0026] In one embodiment, the thermoelectric misalignment feedback degree is the product of the positive number obtained by mapping the difference and the quantization ratio.

[0027] In one embodiment, the process of obtaining the asynchronous execution degree is as follows:

[0028] The time intervals are numbered sequentially, and the pressure data of each foot and leg within the time intervals are arranged sequentially. Adjacent consecutive non-zero pressure data are grouped into a first pressure segment, and adjacent consecutive zero pressure data are grouped into a second pressure segment.

[0029] Calculate the average of the time intervals between the first pressure data point and the first subsequent zero pressure data point in each first pressure segment of each foot and leg within the time period; calculate the arithmetic mean of the time intervals between the first pressure data point and the first subsequent non-zero pressure data point in each second pressure segment of each foot and leg within the time period.

[0030] The average values ​​and arithmetic mean of each leg within the time period are arranged in chronological order to form a time sequence of pressure changes for each leg. The maximum value among the distances between any two synchronously moving legs of the robot dog to be monitored is taken as the asynchronous running degree of the robot dog to be monitored.

[0031] In one embodiment, the process of obtaining the linkage coefficient is as follows:

[0032] Calculate the normalized value of the ratio of the asynchronous running degree to the total number of pressure data for a single leg within the time period;

[0033] The linkage coefficient is the product of the normalized value and the thermoelectric misalignment feedback degree.

[0034] In one embodiment, the process of assessing the operating status of the robot dog to be monitored is as follows:

[0035] The maximum value of the current data within the time period, the current data at the last moment, the internal temperature data at the last moment, the thermoelectric misalignment feedback degree, and the linkage coefficient are combined to form the device state vector of the robot dog to be monitored; the device state vectors of the robot dog in various operating states are obtained for training the classification model, wherein the operating states include normal states and various fault states.

[0036] Based on the device state vector of the robot dog to be monitored, the operating status of the robot dog to be monitored is evaluated using a trained classification model.

[0037] This application has at least the following beneficial effects:

[0038] This application divides a preset time period into multiple sub-time periods, enabling more detailed analysis of temperature and current changes. By calculating the long-term temperature drift, it can effectively identify the temperature change trend inside the robot dog, helping to distinguish temperature changes under normal operation and fault conditions. By calculating the current instability, it can reflect the current change trend and amplitude, effectively monitoring abnormal current changes and timely detecting current faults. Furthermore, by combining the long-term temperature drift and current instability for analysis, it obtains the thermoelectric misalignment feedback degree, assesses the synchronicity of current and temperature changes, distinguishes current faults from other faults, and can more accurately identify fault types, improving the accuracy of subsequent fault diagnosis.

[0039] Furthermore, by monitoring and comparing the running postures of the robot dog's synchronously moving legs during operation, the synchronicity of the robot dog's synchronously moving legs can be measured, which helps to identify leg faults in the robot dog. In addition, by combining the thermoelectric misalignment feedback degree, the linkage coefficient can be obtained, which can comprehensively consider the changes of multiple parameters during the operation of the robot dog and improve the accuracy of leg fault identification.

[0040] Furthermore, by comprehensively considering current, temperature, thermoelectric misalignment feedback degree, and linkage coefficient, the characteristics of the robot dog during operation can be extracted. This can reduce the interference of the complex environment in which the robot dog operates on the data, improve the accuracy of judging the robot dog's operating status, provide early warnings, and thus improve the efficiency of safety monitoring of the robot dog equipment. Attached Figure Description

[0041] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating the steps of a multimodal sensor fusion-based safety monitoring method for robot dog devices provided in this application;

[0043] Figure 2 This is a schematic diagram illustrating the process of obtaining long-term temperature drift.

[0044] Figure 3 This is a schematic diagram illustrating the process of obtaining the linkage coefficient. Detailed Implementation

[0045] In the description of the embodiments in this application, the words "exemplary," "or," and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary," "or," and "for example" is intended to present the relevant concepts in a specific manner.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It should be understood that, unless otherwise stated, " / " in this application means "or".

[0047] It should also be noted that the terms "first" and "second" in this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0048] The following description, in conjunction with the accompanying drawings, details a specific scheme for a multimodal sensor fusion-based safety monitoring method for robot dog devices provided in this application.

[0049] This application provides an embodiment of a multimodal sensor fusion-based safety monitoring method for robot dog devices. Specifically, it provides the following multimodal sensor fusion-based safety monitoring method for robot dog devices. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps:

[0050] Step 1: Collect the internal temperature data, external temperature data, current data of the servo motors of the robot dog to be monitored, and pressure data of each leg of the robot dog to be monitored in real time within a preset time period.

[0051] Temperature sensors installed inside and outside the robot dog under monitoring are used to collect internal and external temperature data in real time within a preset time period. Current sensors and thin-film pressure sensors installed on the robot dog are used to collect current data of the servo motor and pressure data of each leg in real time within a preset time period. The pressure data of each leg represents the pressure of each leg in contact with the ground when the robot dog is running.

[0052] In this embodiment, the acquisition frequency of internal temperature data, external temperature data, current data and pressure data is 5kHz, and the preset time period is 1s. The acquisition frequency and the preset time period are both preset by humans, and the implementer can set them according to the actual situation. This application does not impose any special restrictions.

[0053] Furthermore, taking internal temperature data as an example, if there are missing values ​​in the collected internal temperature data, a data imputation method is used to fill in the missing values. The imputed internal temperature data is then denoised to remove noise. The same imputation and denoising methods are used for external temperature data, current data, and pressure data as for internal temperature data.

[0054] In this embodiment, linear interpolation is used to fill in missing values, and moving average is used to denoise the filled internal temperature data. Both linear interpolation and moving average are well-known technologies and will not be described in detail here. As other implementation methods, based on the ability to fill in missing values, implementers may use other existing feasible data filling methods, and based on the ability to denoise the filled internal temperature data, implementers may use other existing feasible denoising methods. This application does not impose any special restrictions.

[0055] Step 2: Divide the time period into sub-time periods, and obtain the long-term temperature drift of each sub-time period by the difference distribution range and change trend between the internal temperature data and the external temperature data within each sub-time period.

[0056] During the search and rescue operation of the monitored robotic dog, its movement generates heat, causing the internal temperature of the robot to rise. However, the presence of a heat sink inside the robot ensures that the heat dissipated by the heat sink is equal to the heat generated during operation, resulting in a dynamic temperature equilibrium within the robot, meaning the temperature fluctuates within a very small range. Simultaneously, due to the diffusivity of heat, the ambient temperature will have some influence on the internal temperature of the robot, but the internal and external temperatures exhibit the same variation. Therefore, the difference between the internal and external temperature data at each moment is calculated. The temporal variation of this difference across all moments within a preset time period characterizes the temperature difference between the robot's internal and external dimensions.

[0057] In this embodiment, the difference between the internal temperature data and the external temperature data is the absolute value of the difference.

[0058] Furthermore, when the monitored robot dog malfunctions and stops moving, its operating current changes. The longer the current change lasts and the greater the change, the more severe the malfunction. Simultaneously, the current change causes a change in the power of the robot dog's servo motor, resulting in changes in heat dissipation. To differentiate the changes in the robot dog's internal state at different times, a preset time period is divided into multiple sub-time periods. By analyzing the distribution range and trend of the differences between internal and external temperature data within each sub-time period, the long-term temperature drift of each sub-time period is obtained, expressed as:

[0059] In the formula, This represents the long-term temperature drift in the i-th sub-period. , Let represent the range and mean of all the difference values ​​within the i-th sub-period, respectively; and count the number of positive and negative values ​​in the first-order difference results of all the difference values ​​within the i-th sub-period in the time series. The value represents the maximum of the number of positive values ​​and the number of negative values ​​within the i-th sub-period, used to characterize the degree of change in temperature data trends; n represents the total number of the difference values ​​within a single sub-period.

[0060] In this embodiment, the number of sub-time periods is 60. The number of sub-time periods is preset by the user and can be set by the implementer according to the actual situation. This application does not impose any special restrictions.

[0061] It should be noted that after the monitored robot dog malfunctions, the operating current of the robot dog will change, leading to changes in the internal temperature of the robot dog. An increase in current leads to increased heat loss and a rise in temperature; a decrease in current leads to decreased heat loss and a decrease in temperature. Therefore, the more severe the malfunction of the monitored robot dog, the greater the temperature change, resulting in a greater variation in the ratio of the range to the mean of the observed differences. As the current increases, and considering that the current change typically exhibits a single trend over a short period after a malfunction in the monitored robot dog, the ratio of the maximum value among the positive and negative values ​​to the total number of differences is considered important. The increase in temperature drift results in an increase in the long-term temperature drift of the i-th sub-period; a larger long-term temperature drift indicates a greater likelihood of changes in operating status caused by a malfunction in the monitored robot dog. A schematic diagram of the process for obtaining the long-term temperature drift is shown below. Figure 2 As shown.

[0062] Step 3: Obtain the current instability of each sub-period by measuring the degree of change in current data within each sub-period.

[0063] When a monitored robot dog malfunctions, it enters abnormal operation, causing a sudden change in its current. The greater the change in current, the greater the impact of the malfunction on the robot dog's operation. Timely repair is necessary to prevent prolonged malfunction from worsening the equipment, ensuring the robot dog's normal operation, and minimizing the impact on search and rescue operations. Therefore, a sudden change in the current of the monitored robot dog indicates a malfunction; the greater the increase or decrease in current, the more severe the malfunction.

[0064] Furthermore, since the operating environment of the monitored robot dog is typically not a flat area but rather an uneven one, the operating current fluctuates. When the monitored robot dog malfunctions, the current also changes, but the overall trend of the current change remains constant. Therefore, the greater the overall trend of the current, the more likely the current has become unstable, indicating a more likely serious malfunction in the monitored robot dog. Therefore, the slope of the fitted straight line of all current data in each sub-period is obtained to characterize the overall trend of current change in each sub-period. The calculation of the slope is a well-known technique and will not be elaborated upon in this application.

[0065] In this embodiment, the least squares method is used to obtain the fitted straight line of the current data on the time series. The least squares method is a well-known technique. As other implementation methods, based on the ability to obtain the fitted straight line of the current data on the time series, the implementer may use other existing techniques, such as linear regression analysis, weighted least squares method, etc. This application does not impose any special restrictions.

[0066] Furthermore, the current instability of each sub-period is obtained by measuring the degree of change in current data within each sub-period, expressed as:

[0067] In the formula, This represents the current instability in the i-th sub-time period; Represents an exponential function with the natural constant as the base; This represents the slope of the fitted straight line on the time series for all current data within the i-th sub-period; This indicates the absolute value operation; , Let represent the range and mean of all current data within the i-th sub-period, respectively. Used for Normalization is performed.

[0068] It should be noted that when the monitored robot dog malfunctions, its movement rhythm is disrupted, causing joint movement to become jerky, reducing the smoothness of operation of various structures, and causing the operating current of the monitored robot dog to increase or decrease. This results in an overall increasing trend in current change, meaning the absolute value of the slope of the time-series fitted line of the current data in the i-th sub-period increases, and the difference between the maximum and minimum values ​​of the current data in the i-th sub-period increases, causing the ratio of the range to the mean of the current data in the i-th sub-period to increase. The increase in current instability leads to an increase in the current instability of the i-th sub-time period. The larger the calculated current instability, the more likely it is that a malfunction in the monitored robot dog will cause a change in its operating current.

[0069] Step 4: Obtain the thermoelectric misalignment feedback degree of the robot dog under monitoring by synchronizing the changes between the long-term temperature drift and current instability in all sub-periods.

[0070] When a current fault occurs in the monitored robot dog, the increase or decrease in current will cause changes in the heat loss inside the robot dog, resulting in changes in its internal temperature. Therefore, when the changes in the operating current and temperature of the monitored robot dog are synchronized, it indicates that the temperature change is caused by the current fault. When the synchronization between current and temperature is low, it indicates that other faults are causing changes in the internal temperature of the robot dog, such as obstructed heat dissipation leading to a temperature increase, while the current remains unchanged.

[0071] Based on the above analysis, since the numerical ranges of long-term temperature drift and current instability within a preset time period may differ significantly, first-order difference processing is performed on the long-term temperature drift and current instability for all sub-time periods. Positive values ​​in the first-order difference results are marked as 1, negative values ​​as -1, and 0 as 0. The changes in the marking results over time characterize the trend changes of current and temperature during the operation of the monitored robot dog. The first-order difference processing is a known technique and will not be elaborated upon in this application.

[0072] Furthermore, by observing the synchronicity between the long-term temperature drift and current instability across all sub-periods, the thermoelectric misalignment feedback degree of the monitored robot dog is obtained, expressed as:

[0073] In the formula, This indicates the thermoelectric misalignment feedback degree of the robot dog to be monitored; This represents the temporal difference between the marking results corresponding to the long-term temperature drift and current instability within a preset time period; N represents the total number of sub-time periods within the preset time period. , These represent the average values ​​of the labeled results corresponding to the long-term temperature drift and current instability within the preset time period, respectively. This indicates the absolute value operation; 'a' represents a default constant greater than 0, used to avoid... In the case of , the calculation result of thermoelectric misalignment feedback degree is forced to be 0, and the value of 'a' is preset by human. When the value of 'a' is within the range [0.001,1], the implementer can set the value of 'a' according to the actual situation. In this embodiment, the value of 'a' is 1.

[0074] In this embodiment, The calculation process is as follows: the marking results corresponding to the long-term temperature drift and the marking results corresponding to the current instability within the preset time period are arranged in time sequence to form a first marking sequence and a second marking sequence. The sum of the absolute values ​​of the differences between all elements at the same position between the first marking sequence and the second marking sequence is calculated.

[0075] It should be noted that when a malfunction occurs in the monitored robot dog, causing a change in its operating current, this change in current will lead to a change in the internal heat loss of the monitored robot dog. This results in the monitored robot dog's long-term temperature drift and current instability exhibiting the same trend, causing a time-series difference in the corresponding labeled results between the long-term temperature drift and current instability. The smaller the value, the smaller the difference between the long-term temperature drift and the current instability, resulting in a smaller thermoelectric misalignment feedback degree for the monitored robot dog, indicating a greater likelihood of current failure. Conversely, when the cooling system fails, the current change is smaller while the temperature change is larger, thus increasing the thermoelectric misalignment feedback degree of the monitored robot dog. By calculating the thermoelectric misalignment feedback degree, the robot dog's status can be effectively identified.

[0076] Step 5: By comparing the pressure data distribution of the legs of the robot dog under monitoring with the synchronous movement of the robot dog under monitoring, the asynchronous operation degree of the robot dog under monitoring is obtained, and combined with the thermoelectric misalignment feedback degree, the linkage coefficient of the robot dog under monitoring is obtained.

[0077] In order to maintain balance, the robot dog under monitoring needs to keep at least two legs in contact with the ground. The robot dog under monitoring is a biomimetic movement posture of a real dog. Therefore, during the movement, the front leg and the non-same-side hind leg of the robot dog under monitoring move synchronously. Therefore, the left front leg and the right hind leg are considered as synchronously moving legs, and the right front leg and the left hind leg are considered as synchronously moving legs.

[0078] When the robot dog's legs are damaged, the pressure of the legs contacting the ground will no longer be synchronized. Since the environment in which search and rescue robot dogs are located is usually not flat, the legs that move synchronously may not necessarily contact the ground at the same time, thus causing the pressure of the legs that move synchronously to be different. However, since there is a certain time after the legs contact the ground, the intermediate moment when the legs that move synchronously contact the ground should be the same.

[0079] Since the pressure data is not zero upon contact with the ground, to identify the moments when the foot and leg contact or leave the ground, the moment when the pressure data is non-zero is taken as the moment when the foot and leg contact the ground, and the moment when the pressure data is zero is taken as the moment when the foot and leg leave the ground. The temporal distribution of the pressure data represents the temporal distribution of foot and leg contact and departure from the ground. Furthermore, the moments within a preset time period are numbered sequentially, and the pressure data of each foot and leg within the preset time period are arranged sequentially. Adjacent consecutive non-zero pressure data are grouped into a first type of pressure segment, and adjacent consecutive zero pressure data are grouped into a second type of pressure segment. The average of the time numbers between the first pressure data and the first zero pressure data in each first type of pressure segment for each foot and leg within the preset time period is calculated to represent the midpoint of each foot and leg contacting the ground. The arithmetic mean of the time numbers between the first pressure data and the first non-zero pressure data in each second type of pressure segment for each foot and leg within the preset time period is calculated to represent the midpoint of each foot and leg leaving the ground.

[0080] Furthermore, by comparing the moments when the legs of the monitored robot dog contact the ground and leave the ground during synchronous movement, the asynchronous running degree of the monitored robot dog is obtained, expressed as:

[0081] In the formula, This indicates the asynchronous operation level of the robot dog to be monitored; This represents the function that takes the maximum value. This indicates a distance calculation operation; the average value and arithmetic mean of all values ​​corresponding to each foot and leg within a preset time period are arranged chronologically to form a time sequence of pressure changes for each foot and leg. , , , These represent the time sequences of pressure changes in the left front leg, right front leg, left hind leg, and right hind leg of the robot dog to be monitored.

[0082] In this embodiment, the DTW (Dynamic Time Warping) distance between pressure change time series is calculated. The calculation of DTW distance is a well-known technique and will not be described in detail here. As other implementation methods, implementers may use other existing techniques, such as Euclidean distance, to measure the degree of difference between elements in the pressure change time series. This application does not impose any special restrictions.

[0083] It should be noted that after the legs of the robot dog under monitoring malfunction, the movement postures of the legs that move synchronously are no longer synchronized, which increases the difference in the intermediate time between when the legs touch the ground and when they do not. Therefore, the asynchronous operation of the robot dog under monitoring is greater.

[0084] Furthermore, by measuring the asynchronous operation degree of the monitored robot dog and combining it with the thermoelectric misalignment feedback degree of the monitored robot dog, the linkage coefficient of the monitored robot dog is obtained, expressed as:

[0085] In the formula, This represents the linkage coefficient of the robot dog to be monitored; This indicates the thermoelectric misalignment feedback degree of the robot dog to be monitored; Represents the sigmoid function; This indicates the asynchronous operation level of the robot dog to be monitored; This indicates the total number of pressure data points for a single foot or leg within a preset time period.

[0086] It should be noted that after a malfunction occurs in the legs of the monitored robot dog, the synchronous movement of the legs will no longer be synchronized. The greater the degree of asynchronous operation of the monitored robot dog, and the greater the change in the operating current caused by the leg abnormality, the greater the thermoelectric misalignment feedback degree of the monitored robot dog. The higher the calculated linkage coefficient, the greater the probability of leg malfunction and the easier it is to identify the leg malfunction. A schematic diagram of the linkage coefficient acquisition process is shown below. Figure 3 As shown.

[0087] Step 6: Evaluate the operating status of the robot dog under monitoring by using the current data distribution, internal temperature data distribution, thermoelectric misalignment feedback degree, and linkage coefficient during the time period, so as to perform safety monitoring on the robot dog under monitoring.

[0088] Furthermore, the maximum value of the current data of the robot dog under monitoring within a preset time period, the current data at the last moment, the internal temperature data at the last moment, the thermoelectric misalignment feedback degree and the linkage coefficient of the robot dog under monitoring are combined to form the device state vector of the robot dog under monitoring. Among them, the maximum value of the current data is used to reflect the rising state of the fault current, the current data at the last moment is used to reflect the state of the current drop caused by the fault, the internal temperature data at the last moment is used to characterize the internal temperature state of the robot dog under monitoring, the thermoelectric misalignment feedback degree of the robot dog under monitoring is used to characterize the synchronicity of the current change and temperature change of the robot dog under monitoring, and the linkage coefficient of the robot dog under monitoring is used to characterize the degree of correlation between different parameters during the operation of the robot dog under monitoring.

[0089] Furthermore, following the method for obtaining the device state vector of the robot dog to be monitored, the device state vectors of the robot dog in various operating states are obtained. The operating states include normal states and various fault states. In this embodiment, the fault states include motor faults, heat dissipation faults, foot bearing jamming faults, and permanent magnet partial demagnetization faults.

[0090] In this embodiment, the number of device state vectors for the robot dog in various operating states is 100. 100 is only one embodiment of this application. Implementers can set it according to the actual situation. This application does not impose any special restrictions.

[0091] Furthermore, the operating state corresponding to each device state vector is used as the state label of each device state vector. All device state vectors of all robot dogs with all kinds of operating states, as well as the state label of each device state vector, are used as input to the decision tree. The optimal features of the decision tree are calculated through information gain, and the trained decision tree is output.

[0092] Furthermore, the device state vector of the robot dog to be monitored is used as input to the trained decision tree, and the output is a state label for the device state vector of the robot dog to be monitored. The operating status of the robot dog to be monitored is monitored through the state label. If the robot dog to be monitored malfunctions, the malfunction status is sent as an alarm message to the on-site maintenance personnel for timely repair. The decision tree is merely one embodiment of this application; implementers may choose other existing feasible classification models, and this application does not impose any special restrictions.

[0093] In summary, this application, by dividing a preset time period into multiple sub-time periods, enables more detailed analysis of temperature and current changes; by calculating the long-term temperature drift, it can effectively identify the temperature change trend inside the robot dog, helping to distinguish temperature changes under normal operation and fault conditions; by calculating the current instability, it can reflect the current change trend and amplitude, effectively monitoring abnormal current changes and timely detecting current faults; furthermore, by combining the long-term temperature drift and current instability for analysis, it obtains the thermoelectric misalignment feedback degree, assesses the synchronicity of current and temperature changes, distinguishes current faults from other faults, and can more accurately identify fault types, improving the accuracy of subsequent fault diagnosis.

[0094] Furthermore, by monitoring and comparing the running postures of the robot dog's synchronously moving legs during operation, the synchronicity of the robot dog's synchronously moving legs can be measured, which helps to identify leg faults in the robot dog. In addition, by combining the thermoelectric misalignment feedback degree, the linkage coefficient can be obtained, which can comprehensively consider the changes of multiple parameters during the operation of the robot dog and improve the accuracy of leg fault identification.

[0095] Furthermore, by comprehensively considering current, temperature, thermoelectric misalignment feedback degree, and linkage coefficient, the characteristics of the robot dog during operation can be extracted. This can reduce the interference of the complex environment in which the robot dog operates on the data, improve the accuracy of judging the robot dog's operating status, provide early warnings, and thus improve the efficiency of safety monitoring of the robot dog equipment.

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0097] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from its essential characteristics. Therefore, the embodiments described above should be considered exemplary and non-limiting in all respects.

Claims

1. A method for safety monitoring of a robot dog device using multimodal sensor fusion, characterized in that, The method includes the following steps: The system collects the internal and external temperature data of the robot dog under monitoring, as well as the current data of the servo motor and the pressure data of each leg of the robot dog under monitoring in real time within a preset time period. The time period is divided into sub-time periods. The long-term temperature drift of each sub-time period is obtained by analyzing the difference distribution range and trend of internal and external temperature data within each sub-time period. The current instability of each sub-time period is obtained by analyzing the degree of change of current data within each sub-time period. The thermoelectric misalignment feedback degree of the robot dog under monitoring is obtained by analyzing the synchronicity of changes between the long-term temperature drift and current instability of all sub-time periods. The asynchronous operation degree of the robot dog under monitoring is obtained by comparing the pressure data distribution of the legs during synchronous movement of the robot dog under monitoring. Combined with the thermoelectric misalignment feedback degree, the linkage coefficient of the robot dog under monitoring is obtained. The operating status of the robot dog under monitoring is evaluated by the current data distribution, internal temperature data distribution, thermoelectric misalignment feedback degree, and linkage coefficient during the time period, so as to carry out safety monitoring of the robot dog under monitoring. The process of assessing the operating status of the robot dog to be monitored is as follows: The maximum value of the current data within the time period, the current data at the last moment, the internal temperature data at the last moment, the thermoelectric misalignment feedback degree, and the linkage coefficient are combined to form the device state vector of the robot dog to be monitored; the device state vectors of the robot dog in various operating states are obtained for training the classification model, wherein the operating states include normal states and various fault states. Based on the device state vector of the robot dog to be monitored, the operating status of the robot dog to be monitored is evaluated using a trained classification model.

2. The method for safety monitoring of a robot dog device based on multimodal sensor fusion as described in claim 1, characterized in that, The process for obtaining the long-term temperature drift is as follows: Calculate the difference between the internal temperature data and the external temperature data at each time point, and record the ratio of the range of all the difference values ​​in each sub-period to the mean as the first ratio. The number of positive and negative values ​​in the first-order difference results of all the difference values ​​in each sub-period is counted. The maximum value among the number of positive and negative values ​​is counted. The ratio of the maximum value to the total number of difference values ​​in each sub-period is recorded as the second ratio. The long-term temperature drift is obtained by combining the first ratio with the second ratio.

3. The method for safety monitoring of a robot dog device based on multimodal sensor fusion as described in claim 2, characterized in that, The long-term temperature drift is the product of the first ratio and the second ratio.

4. The method for safety monitoring of a robot dog device using multimodal sensor fusion as described in claim 1, characterized in that, The process for obtaining the current instability is as follows: Obtain the absolute value of the slope of the fitted straight line of all current data in each sub-period on the time series; The ratio of the range to the mean of all current data in each sub-period is denoted as the current ratio. The current instability is obtained by combining the normalized result of the absolute value with the current ratio.

5. The method for safety monitoring of a robot dog device using multimodal sensor fusion as described in claim 4, characterized in that, The current instability is the product of the normalized result of the absolute value and the current ratio.

6. The method for safety monitoring of a robot dog device using multimodal sensor fusion as described in claim 1, characterized in that, The process of obtaining the thermoelectric misalignment feedback degree is as follows: The long-term temperature drift and current instability of all sub-periods are processed by first-order difference in time series. Positive values ​​in the first-order difference results are marked as 1, negative values ​​as -1, and 0 as 0. Calculate the temporal difference between the labeled results corresponding to the long-term temperature drift and current instability during the aforementioned time period; Calculate the mean value of the labeling results corresponding to the long-term temperature drift and the mean value of the labeling results corresponding to the current instability within the time period, and calculate the difference between the mean values ​​of the corresponding labeling results for the long-term temperature drift and the current instability. The ratio of the difference to the total number of sub-periods is denoted as the quantification ratio. The thermoelectric misalignment feedback degree is positively correlated with the quantization ratio and the difference amount, respectively.

7. The method for safety monitoring of a robot dog device based on multimodal sensor fusion as described in claim 6, characterized in that, The thermoelectric misalignment feedback degree is the product of the positive number obtained by mapping the difference quantity and the quantization ratio.

8. The method for safety monitoring of a robot dog device using multimodal sensor fusion as described in claim 1, characterized in that, The process of obtaining the asynchronous running degree is as follows: The time intervals are numbered sequentially, and the pressure data of each foot and leg within the time intervals are arranged sequentially. Adjacent consecutive non-zero pressure data are grouped into a first pressure segment, and adjacent consecutive zero pressure data are grouped into a second pressure segment. Calculate the average of the time intervals between the first pressure data point and the first subsequent zero pressure data point in each first pressure segment of each foot and leg within the time period; calculate the arithmetic mean of the time intervals between the first pressure data point and the first subsequent non-zero pressure data point in each second pressure segment of each foot and leg within the time period. The average values ​​and arithmetic mean of each leg within the time period are arranged in chronological order to form a time sequence of pressure changes for each leg. The maximum value among the distances between any two synchronously moving legs of the robot dog to be monitored is taken as the asynchronous running degree of the robot dog to be monitored.

9. The method for safety monitoring of a robot dog device using multimodal sensor fusion as described in claim 1, characterized in that, The process of obtaining the linkage coefficient is as follows: Calculate the normalized value of the ratio of the asynchronous running degree to the total number of pressure data for a single leg within the time period; The linkage coefficient is the product of the normalized value and the thermoelectric misalignment feedback degree.

Images