Robot, monitoring method and system thereof and storage medium

Abstract

The invention relates to the technical field of robots, and discloses a robot, a monitoring method and system thereof and a storage medium, and the method comprises the steps: collecting the operation time sequence data of each sensor carried by a target robot in real time; performing frequency domain conversion on the operation time sequence data, and extracting an energy distribution characteristic in a preset frequency interval as a target frequency domain characteristic component; performing fusion processing on the target frequency domain characteristic components of the sensors to obtain fused spectrum data, and comparing the fused spectrum data with a pre-constructed reference spectrum model; and if the comparison result exceeds a judgment threshold value, determining that the target robot runs abnormally. According to the method, feature extraction is performed on the time sequence data of the sensor by introducing frequency domain transformation, and the operation state of the robot is accurately monitored by combining fusion analysis of the multi-source frequency domain features with comparison of the reference model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and in particular to a robot, a monitoring method and system thereof, and a storage medium. BACKGROUND

[0002] With the development of smart home technology, sweeping robots as typical service robots have been widely applied in home and office environments. By integrating laser radar, ultrasonic sensor, infrared sensor, collision sensor, odometer and other sensors, it can collect environmental information and its own running state data in real time, providing important support for realizing autonomous navigation, obstacle avoidance and path planning functions. However, the existing sensor monitoring mechanism still has obvious limitations. On the one hand, abnormal detection relies on manually preset fixed thresholds, which is difficult to adapt to dynamic changes such as ground material, lighting conditions or equipment aging, resulting in high false alarm rate and poor adaptability in complex environments; on the other hand, each sensor data is usually processed in isolation, lacking effective fusion analysis means, and the overall running state of the robot cannot be comprehensively evaluated from the system level, which is prone to missed detection or misjudgment. In addition, when an abnormality occurs, the system usually only triggers a simple alarm, lacks intelligent identification and automatic response capability for fault types, and requires user intervention to troubleshoot, which is low in maintenance efficiency and affects the user experience. SUMMARY

[0003] Therefore, the embodiments of the present application provide a robot monitoring method, system, robot and computer readable storage medium, which extracts features from sensor time series data by introducing frequency domain transformation, and combines multi-source frequency domain feature fusion analysis and reference model comparison to accurately detect the running state of the robot.

[0004] In a first aspect, the embodiments of the present application provide a robot monitoring method, comprising: collecting running time series data of each sensor carried by a target robot in real time; performing frequency domain conversion on the running time series data, and extracting energy distribution features in a preset frequency interval as target frequency domain feature components; fusing the target frequency domain feature components of each sensor to obtain fused frequency spectrum data, and comparing the fused frequency spectrum data with a pre-constructed reference frequency spectrum model; if the comparison result exceeds a judgment threshold, determining that the target robot is running abnormally.

[0005] In an optional implementation, the construction of the reference frequency spectrum model comprises: collecting sample time series data output by each sensor carried by a sample robot of the same type as the target robot, and performing frequency domain conversion on the sample time series data to obtain sample frequency spectrum data; extracting energy distribution features of each of the sensors in a corresponding preset frequency interval from the sample spectrum data as sample frequency domain feature components; based on the sample frequency domain feature components, determining the reference spectrum model representing normal operation state of the target robot by using a statistical analysis method.

[0006] In an optional implementation, if the comparison result exceeds the determination threshold, it is determined that the target robot is operating abnormally, including: If the state deviation between the fusion spectrum data and the reference spectrum model is greater than the current determination threshold and the duration exceeds the set delay time, it is confirmed that the target robot is in an abnormal operation state.

[0007] In an optional implementation, the determination threshold is determined in real time based on the mean, standard deviation of the target frequency domain feature components of the target robot in the sliding window, and a confidence coefficient.

[0008] In an optional implementation, the confidence coefficient is determined based on the accuracy level, application scenario risk level, and historical failure rate of each of the sensors.

[0009] In an optional implementation, the fusion processing of the target frequency domain feature components of each of the sensors to obtain fusion spectrum data includes: performing weighted fusion on the target frequency domain feature components of each of the sensors to obtain the fusion spectrum data representing the current operation state of the target robot; wherein the weights of the weighted fusion are determined according to the measurement noise variances of each of the sensors.

[0010] In an optional implementation, after determining that the target robot is operating abnormally, it further includes: extracting the target frequency domain feature components of each of the sensors during the abnormal operation to form a frequency domain feature vector group as a frequency domain feature mode of the current abnormal event; matching the frequency domain feature mode with a pre-stored fault knowledge base to determine a fault type; generating a corresponding control instruction according to the fault type and sending it to a main control system of the target robot through a communication interface to trigger execution of a corresponding response strategy.

[0011] In a second aspect, the embodiments of the present application provide a robot monitoring system, including: a collection module configured to collect runtime sequence data of each sensor carried by a target robot in real time; a conversion module configured to perform frequency domain conversion on the runtime sequence data and extract energy distribution features in a preset frequency interval as target frequency domain feature components; a comparison module, configured to fuse the target frequency domain feature components of the sensors, obtain fused frequency spectrum data, and compare the fused frequency spectrum data with a pre-constructed reference spectrum model; a judgment module, configured to determine that the target robot is running abnormally if the comparison result exceeds a determination threshold.

[0012] In a third aspect, an embodiment of the present application provides a robot, which comprises a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to implement the robot monitoring method described above.

[0013] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed on a processor, implements the robot monitoring method described above.

[0014] The embodiments of the present application have the following beneficial effects: The embodiments of the present application collect the running time sequence data of multiple sensors of a robot in real time, perform frequency domain conversion on the data to extract the energy distribution features in the key frequency interval, and then realize the state representation conversion from the time domain to the frequency domain, so that the periodic changes and potential abnormal patterns in the sensor data can be effectively captured. Further, the target frequency domain feature components of the sensors are fused to obtain the fused frequency spectrum data representing the overall running state of the robot, and the fused frequency spectrum data is compared with the pre-constructed reference spectrum model, so that the integrity and accuracy of the state evaluation are improved. The method can more sensitively identify the deviation of the equipment running in a complex dynamic environment, and improve the stability and adaptability of the abnormal detection. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0016] Figure 1 Fig. 1 shows a structural schematic diagram of robot monitoring according to an embodiment of the present application; Figure 2 Fig. 2 shows a first flowchart of a robot monitoring method according to an embodiment of the present application; Figure 3 Fig. 3 shows a second flowchart of a robot monitoring method according to an embodiment of the present application; Figure 4 Fig. 4 shows a third flowchart of a robot monitoring method according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely in the embodiments of the present application combined with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0018] The components of the embodiments of the present application generally described and illustrated in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] Hereinafter, the terms "include", "have", and their conjugates used in various embodiments of the present application are only intended to denote that specific features, numbers, steps, operations, elements, components, or combinations thereof are present, and should not be construed as excluding the presence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof. In addition, the terms "first", "second", "third", and the like are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0020] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present application belong. The terms (such as those defined in commonly used dictionaries) will be interpreted as having a meaning that is the same as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized or overly formal meaning unless clearly defined in various embodiments of the present application.

[0021] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0022] First, the architecture of the robot monitoring system is described. The robot monitoring system is built based on the robot operating system, adopts a "node-topic-service" distributed communication model, and realizes efficient data interaction and flexible expansion between modules. The architecture of the system mainly includes a data acquisition layer, a data processing layer, a monitoring decision layer, and a human-computer interaction layer, which communicate through ROS topics and ROS services.

[0023] In the embodiment, the robot is specifically applied to a household sweeping robot, but is not limited to this type, and can also be applied to other service robots, such as a delivery robot or a security patrol robot.

[0024] Specifically, as shown in the system architecture includes a collection module 100, a conversion module 200, a comparison module 300, and a judgment module 400. Figure 1

[0025] The collection module 100 is in the data collection layer, and is configured to collect runtime sequence data of each sensor carried by the target robot in real time. The sensors include, but are not limited to, a laser radar, an inertial measurement unit, an ultrasonic sensor, an odometer, and a collision sensor. Various sensors publish raw data streams through ROS topics, for example, a laser radar publishes a lidar_data topic to transmit point cloud data; an IMU publishes an imu topic to transmit angular velocity and acceleration information; and an odometer publishes an odom topic to transmit pose estimation results. The collection module 100 is an independent subscription node, listens to the above topics, and performs timestamp synchronization and format standardization processing on the obtained data, to ensure that the data under different sampling frequencies and coordinate systems are consistent and comparable.

[0026] The conversion module 200 is in the data processing layer, and is configured to perform frequency domain conversion on the received runtime sequence data, and extract energy distribution features in a preset frequency interval as target frequency domain feature components. In the embodiment, the frequency domain conversion can use discrete Fourier transform or fast Fourier transform. For the time sequence signals output by each type of sensor, the system maps the time sequence signals from the time domain to the frequency domain to obtain a frequency spectrum representation. Subsequently, according to a pre-set frequency interval of interest, the corresponding energy amplitudes are extracted as key features. For example, for a distance change sequence of a laser radar, the main frequency component of the horizontal direction 40 cm detection distance near 2 Hz is focused on; for the distance reading of an ultrasonic sensor, the energy components in the 1-3 Hz range of the near distance 20 cm region are extracted; and for the angular velocity signal output by the IMU, the 5-10 Hz frequency band reflecting the body jitter characteristics is extracted. The target frequency domain feature components are the frequency domain features filtered above, and are used for subsequent fusion and state evaluation.

[0027] The comparison module 300 is in the data processing layer, and is configured to perform fusion processing on the target frequency domain feature components of each sensor, generate fusion spectrum data, and compare the fusion spectrum data with a pre-constructed reference spectrum model. Specifically, the module first performs a weighted fusion operation, and the fusion weights are determined according to the measurement noise variances of the sensors. The smaller the noise variance, the greater the allocated weight. The fusion spectrum data generated after fusion represents the overall running state of the target robot. Subsequently, the module calculates the state deviation degree between the fusion spectrum data and the reference spectrum model as a basic input for anomaly detection.​

[0028] The judgment module 400 is located in the monitoring decision layer, and is used for judging whether an abnormal running state exists according to the comparison result. When the state deviation degree is greater than the dynamically updated judgment threshold and the duration exceeds the set delay time, it is confirmed that the target robot is in an abnormal running state. The judgment threshold is not a fixed value, but is adjusted in real time based on historical normal data in a sliding time window. Specifically, the mean value of the target frequency domain feature component in the current window is calculated and the standard deviation , and a confidence coefficient is combined to form an interval as a dynamic judgment threshold. The confidence coefficient is determined according to the sensor accuracy level, the application scene risk level and the historical failure rate, and the scoring weights are 0.4, 0.3 and 0.3 respectively.

[0029] Further, the robot monitoring system further includes a fault diagnosis module, a response control module and a man-machine interaction module.

[0030] The fault diagnosis module is located in the monitoring decision layer. After confirming the abnormality, the target frequency domain feature component of each sensor during the abnormal running is extracted to form a frequency domain feature vector group as the frequency domain feature mode of the current abnormal event. The frequency domain feature mode is matched with the pre-stored fault knowledge base to identify the corresponding fault type (such as radar shielding, wheel skidding, IMU failure, etc.).

[0031] The response control module is located in the monitoring decision layer, and is used for generating a corresponding control instruction according to the diagnosed fault type, such as sending a command to pause work, starting a self-checking program or executing an obstacle avoidance, and sending the command to the main control system of the target robot through a ROS service calling mechanism to trigger an automatic response strategy. The man-machine interaction module is located in the man-machine interaction layer, and is used for providing a graphical user interface to display sensor data curves, fused frequency spectrum data, device running states and alarm information in real time. The man-machine interaction module supports manual setting of monitoring parameters (such as delay time, weight configuration) by a user and receiving remote control instructions.

[0032] On the basis of the above system, the robot monitoring method of the embodiment will be described below.

[0033] Figure 2 A flowchart of a robot monitoring method according to an embodiment of the application is shown. The robot monitoring method includes steps S100-S400. In step S100, the running time series data of each sensor carried by the target robot is collected in real time.

[0034] In this step, the raw data stream continuously output by the multiple heterogeneous sensors carried by the target robot during operation, i.e., runtime sequential data, is acquired. The runtime sequential data refers to a sequence of sensor measurement values that changes continuously over time, and each data point contains a measurement value and a corresponding timestamp.

[0035] Specifically, the target robot is a service robot with autonomous navigation and environmental perception capabilities, and in this embodiment, a household sweeping robot is taken as an example. The sensors carried by the robot include, but are not limited to, a laser radar, an inertial measurement unit, an ultrasonic sensor, an odometer, a collision sensor, an infrared sensor, etc.; among them, the laser radar is used to scan the surrounding environment to generate two-dimensional or three-dimensional point cloud data, which assists in positioning and obstacle avoidance; the inertial measurement unit integrates an accelerometer and a gyroscope to output motion state information such as angular velocity and linear acceleration; the ultrasonic sensor is used to measure the distance between the sensor and an obstacle by emitting ultrasonic waves and receiving echoes; the odometer is used to calculate the pose change of the robot based on wheel encoder data; the collision sensor is used to detect whether the robot has physical contact with an object; and the infrared sensor is used to identify cliff, step, and other falling risk areas.

[0036] At the system architecture level, each sensor acts as an independent data source and publishes its raw data through the topic mechanism of the Robot Operating System (ROS).

[0037] The acquisition module, as a subscription node, listens to the above topics and receives data packets from each sensor at a preset frequency. To ensure the accuracy of subsequent processing, the collected data needs to be processed during the acquisition process, which includes, but is not limited to, time synchronization, format standardization processing, and data caching processing, etc.

[0038] For time synchronization processing, since the sampling frequencies of different sensors are different (e.g., 10 Hz for laser radar and 50 Hz for IMU), the timestamps of the received data may be misaligned. Therefore, the system adopts a time synchronization strategy based on ROS to align the data from multiple sensors according to the timestamps, thereby constructing a multi-source data snapshot at the same time. For format standardization processing, i.e., converting all sensor data into a common data structure required for internal processing. For example, all distance data is converted to meters (m) as the unit; angular velocity data is normalized to radians per second (rad / s); and cloud data is converted to distance-angle representation in the polar coordinate system for subsequent frequency domain analysis. For data caching and buffer management, the collected data is temporarily stored in a ring buffer, and each sensor corresponds to an independent data queue, thereby ensuring that high-frequency data will not be lost due to processing delay.

[0039] By the above manner, the system can stably and continuously acquire multi-dimensional running time sequence data of the target robot in the cleaning operation process, thereby providing a data basis for subsequent frequency domain conversion and state evaluation.

[0040] In step S200, frequency domain conversion is performed on the running time sequence data, and energy distribution features in a preset frequency interval are extracted as target frequency domain feature components.

[0041] In this step, the frequency domain conversion operation is performed on the running time sequence data of each type of sensor. The frequency domain conversion can adopt discrete Fourier transform or fast Fourier transform. After the frequency domain conversion, the running time sequence data is a complex sequence, representing amplitude and phase information of the original signal at different frequencies. The system further extracts the amplitude spectrum, i.e., the energy intensity of each frequency component, and performs normalization processing, so as to facilitate cross-sensor comparison. Subsequently, the system extracts the energy distribution features in the preset frequency interval from the amplitude spectrum as the target frequency domain feature components. The preset frequency interval is set in advance according to the functional characteristics and historical running data analysis of each type of sensor, and is used to focus on the key frequency band reflecting the device running state, while filtering irrelevant noise or interference components.

[0042] In this embodiment, the target frequency domain feature components of different sensors are different. The target frequency domain feature component of the laser radar focuses on the periodic signal formed by the change of the scanning angle with time. Under normal circumstances, the laser radar rotates at a constant angular velocity, and a significant peak value appears at the main driving frequency (for example, 2 Hz) in the frequency spectrum. The system extracts the energy amplitude in the main frequency and the adjacent ±0.5 Hz range as the target frequency domain feature component, which is used to monitor whether there is a jam, jitter or out-of-step phenomenon. The target frequency domain feature component of the IMU angular velocity signal is used to detect the body vibration, tilt or slip. Under normal operation, the angular velocity fluctuation is concentrated in the low frequency band (for example, 0-2 Hz). If the wheel is idling or colliding, obvious harmonics will be generated in the 5-10 Hz range. Therefore, the system extracts the energy component in this frequency band as the target frequency domain feature component. The target frequency domain feature component of the ultrasonic sensor is used to detect the front obstacle. When the sensor works normally and the environment is stable, the frequency spectrum of the distance sequence is mainly concentrated near the direct current component; if there is continuous fluctuation or jump (such as being blocked by dust), the high frequency component will be significantly increased. ​) will be significantly enhanced. Therefore, the system sets the energy of the high frequency interval as one of the target frequency domain feature components. The target frequency domain feature component (linear velocity sequence) of the odometer is used to reflect the overall movement rhythm of the robot. As a stable low-frequency signal under normal walking, abnormal sliding will cause high-frequency jitter, so a specific frequency band (such as 3-7Hz) is also extracted as a target frequency domain feature component. The above target frequency domain feature components can be stored in vector form, and each dimension corresponds to the energy value of a frequency point. This feature component not only retains the core dynamic behavior of the original signal, but also suppresses the influence of transient interference and background noise.

[0043] Step S300, the target frequency domain feature components of each sensor are fused to obtain fused frequency spectrum data, and the fused frequency spectrum data are compared with a pre-constructed reference frequency spectrum model.

[0044] This step mainly weights and fuses the target frequency domain feature components of each sensor to generate fused frequency spectrum data representing the current running state of the target robot, and compares the fused frequency spectrum data with a pre-constructed reference frequency spectrum model to provide a basis for subsequent abnormality judgment.

[0045] Specifically, due to the differences in precision, stability and environmental adaptability of different sensors, the contribution of the target frequency domain feature components output by the sensors to the overall state evaluation should also be different. Therefore, the system uses a weighted fusion method to synthesize the target frequency domain feature components from multiple sensors.

[0046] It can be understood that if the system contains n types of sensors, the target frequency domain feature component corresponding to the i-th sensor is a vector , which represents the energy distribution in the preset frequency interval. The system assigns a weight to the feature component, and the weight satisfies the sum of the weights of all sensors equal to 1, and then performs a fusion operation to generate fused frequency spectrum data X, which can be represented as ; wherein F is the generated fused frequency spectrum data, which is a vector with the same dimension as each target frequency domain feature component, representing the overall running state spectrum characteristics of the target robot. The weight is determined according to the measurement noise variance of each sensor. The smaller the noise variance, the more stable and reliable the sensor data, and the greater the weight assigned. The measurement noise variance can be obtained through factory calibration or historical data analysis. For example, the standard deviation of the distance reading of a laser radar in a stable environment is about 0.02m, and the corresponding variance is ; the standard deviation of an ultrasonic sensor is greatly affected by temperature and can reach 0.08m, and the corresponding variance is Therefore, the weight of the laser radar is obviously higher than that of the ultrasonic sensor, and the laser radar will dominate in the fusion process.

[0047] It should be noted that all target frequency domain feature components participating in fusion need to have the same data structure (such as the same frequency resolution and frequency band division). If the original dimensions are inconsistent, interpolation or dimension reduction processing is performed to align them.

[0048] After obtaining the fused frequency spectrum data, the system compares it with a pre-constructed reference frequency spectrum model to determine whether the current operating state deviates from the normal mode. The reference frequency spectrum model is a typical frequency spectrum template representing the target robot operating in a healthy state, stored in the local or cloud database of the system. This model is not static and unchangeable, but can be dynamically updated and optimized according to long-term operating data during the life cycle of the device.

[0049] The comparison process can use Euclidean distance, cosine similarity, dynamic time warping distance, etc. to calculate the state deviation between the two, to quantify the difference between the current state and the normal state. The larger the state deviation value, the more the current operating state deviates from the normal mode, and the more likely it is to have an anomaly.

[0050] In some embodiments, as shown in Figure 3 The construction of the reference frequency spectrum model includes steps S310-S330: Step S310, collect sample time series data output by each sensor carried by a sample robot of the same type as the target robot, and perform frequency domain transformation on the sample time series data to obtain sample frequency spectrum data.

[0051] Exemplarily, in the batch testing phase, a plurality of sample robots of the same type as the target robot are selected, and the sample robots are operated in a standard cleaning task in a normal working state (no fault, flat ground, moderate illumination). The sample time series data output by each sensor carried by the sample robots is continuously collected. The collected data includes but is not limited to the distance scanning sequence of the laser radar, the angular velocity and acceleration signals of the IMU, the distance readings of the ultrasonic sensor, the speed change sequence of the odometer, etc. Subsequently, the same frequency domain conversion operation as step S200 is performed on each sample time series data to obtain the corresponding sample frequency spectrum data, to reflect the typical frequency response of each type of sensor in the normal working condition.

[0052] Step S320, extract the energy distribution characteristics of each sensor in the corresponding preset frequency interval from the sample frequency spectrum data as sample frequency domain feature components.

[0053] Exemplarily, after obtaining the sample spectrum data, the energy distribution characteristics of each sensor in the corresponding preset frequency interval are extracted in the same manner as step S200, and then a set of standardized sample frequency domain feature components are formed. For example, for a laser radar, the energy amplitude near the main driving frequency (about 2 Hz) is extracted; for an IMU, the vibration energy in the low frequency band is extracted;

[0054] Step S330, based on the sample frequency domain feature components, a statistical analysis method is used to determine the reference spectrum model representing the normal operating state of the target robot.

[0055] In this embodiment, the mean method, clustering method, confidence interval modeling, etc. can be used to determine the reference spectrum model representing the normal operating state of the target robot. If the mean method is used, the arithmetic mean of the sample frequency domain feature components of all samples at each frequency point is calculated to generate a typical spectrum curve as the reference spectrum model; if the clustering method is used, the K-means algorithm can be used to cluster the sample frequency domain feature vectors, and the center vector of the largest cluster is selected as the reference spectrum model to enhance robustness; if the confidence interval modeling method is used, not only the mean value is recorded, but also the standard deviation information is saved, so that the subsequent anomaly detection can be combined with the fluctuation range to determine.

[0056] In this embodiment, the reference spectrum model can be stored in a structured file form and distributed to the terminal device with the firmware, or it can be updated regularly by the server under network conditions.

[0057] Step S400, if the comparison result exceeds the determination threshold, it is determined that the target robot is operating abnormally.

[0058] Specifically, if the state deviation between the fusion spectrum data and the reference spectrum model is greater than the dynamically updated determination threshold and the duration exceeds the set delay time, it is confirmed that there is an abnormal operating state.

[0059] It can be understood that this step determines whether the target robot is in an abnormal operating state according to the comparison result obtained in the above steps, i.e. the state deviation between the fusion spectrum data and the reference spectrum model. In order to improve the detection accuracy and avoid false positives caused by transient interference, this embodiment adopts a double-threshold joint determination method, i.e. only when both the state deviation exceeds the dynamically updated determination threshold and the duration exceeds the set delay time, it is confirmed that there is an abnormal operating state.

[0060] ​The determination threshold dynamically updated is determined based on the mean value, the standard deviation and the confidence coefficient of the target frequency domain feature component corresponding to the target robot in the current sliding window. It can be understood that the determination threshold is not a fixed value, but a dynamic parameter that is adjusted in real time according to the current running environment and the device state, so as to adapt to long-term trends such as ground material changes, battery aging, sensor performance attenuation and the like.

[0061] In the embodiment, the determination manner of the dynamically updated determination threshold is as follows: the system first sets a sliding time window, for example, the last 10 continuous monitoring periods (for example, 3 seconds per period, a total of 30 seconds), and only uses the historical data identified as the normal running state in the window to calculate the statistical features. For the target frequency domain feature component sequence of each type of sensor, the system calculates the mean value and the standard deviation of the target frequency domain feature component in the current sliding window in real time, and then combines a preset confidence coefficient to construct the determination threshold interval at the current moment; wherein the state deviation exceeds the upper limit or the lower limit of the interval, that is, exceeds the determination threshold. It should be noted that only in the case that the current period is not marked as abnormal, the data is used for the sliding window to participate in the update of the mean value and the standard deviation, so as to prevent the abnormal data from polluting the statistical model and guarantee the stability and reliability of the threshold.

[0062] The confidence coefficient is a key parameter for adjusting the sensitivity of the anomaly detection, and the greater the value is, the more fluctuations are allowed and the detection is more relaxed; the smaller the value is, the more sensitive it is and the alarm is easily triggered. In the embodiment, the confidence coefficient is not artificially and empirically set, but is determined by weighting based on the accuracy level of the sensor, the risk level of the application scenario and the historical failure rate. The accuracy level of the sensor can reflect the fault tolerance ability of the measurement result of the sensor, and the higher the accuracy is, the higher the score is. The risk level of the application scenario is used to reflect the complexity and potential harm of the working environment of the robot, and the higher the risk is, the higher the score is. The historical failure rate is used to obtain based on the factory test and user feedback statistics, and the higher the failure rate is, the higher the score is. The scores of the above three items are normalized to the interval [0, 1], and are combined according to the following weights , and then the confidence coefficient is obtained. The confidence coefficient value can be written once in the device configuration stage, or can support remote update or online learning optimization.

[0063] The set delay time refers to the length of time that the abnormal state must be continuously maintained, and can be used to filter short-term fluctuations. In this embodiment, the delay time can be pre-set by experimental calibration method. For example, a large number of tests are carried out in simulated living rooms, bedrooms, kitchens and other various home scenes, the maximum transient deviation duration under normal operation and the response time of real faults are recorded, and the minimum stable value that can effectively distinguish the two is selected as the final delay time. The value range can be 1.0-2.5 seconds, and the specific configuration can be based on the sensor type and application requirements: for laser radar anomaly detection, the delay time can be set to 1.5 seconds; for collision sensor continuous trigger detection, the delay time can be set to 1.0 second. The system starts the timer when it detects that the state deviation degree exceeds the judgment threshold for the first time, and resets the timer if the state falls back to the normal range during this period; only when the over-limit state is continuously maintained for more than the delay time, it is finally confirmed as an anomaly.

[0064] In some embodiments, as shown in Figure 4 After determining that the target robot is running abnormally, the method further includes steps S500-S700: Step S500, extracting the target frequency domain feature components of each sensor during the running abnormality to form a frequency domain feature vector group as the frequency domain feature mode of the current abnormal event.

[0065] Exemplarily, in order to accurately identify the fault type, the system first extracts representative feature information from the data in the abnormal period. Specifically, while determining the existence of abnormal running state, the system automatically backtracks and locks the key time period before and after the abnormal trigger (for example, 2 seconds before the abnormal start to 3 seconds after the abnormal end, about 5 seconds long), and extracts the original data from the target frequency domain feature components of each sensor in this time period. Subsequently, these target frequency domain feature components are organized into an ordered set according to the sensor category, i.e. a frequency domain feature vector group. Each item of the vector group corresponds to the energy distribution characteristics of a certain sensor in a specific frequency interval, forming a point in a multi-dimensional feature space. Then the frequency domain feature vector group is defined as the frequency domain feature mode of the current abnormal event.

[0066] Step S600, matching the frequency domain feature mode with a pre-stored fault knowledge base to determine the fault type.

[0067] Exemplarily, after obtaining the frequency domain feature mode of the current abnormal event, the system performs similarity matching between the frequency domain feature mode and the known fault templates in the stored fault knowledge base to identify the most likely fault type.

[0068] The fault knowledge base is a structured database, which is established by a large number of experimental tests, factory calibration and historical operation and maintenance data accumulation in advance. Each record corresponds to a common fault type, which includes but is not limited to fault type (such as laser radar shielding, wheel skidding, IMU failure, etc.), feature spectrum template (corresponding to the typical frequency domain feature mode under fault, which is expressed as the energy distribution vector of each sensor at the key frequency point), fault reason (such as dust covering, wet ground, hardware damage, etc.), occurrence scene, priority level, recommended response strategy (corresponding control instruction suggestion, such as suspending the work, starting self-checking, obstacle avoidance, etc.) and the like. The knowledge base can be pre-installed when the device is shipped, or it can be updated regularly by the server in a networked state.

[0069] When matching, the similarity (such as Euclidean distance, cosine similarity, etc.) between the current frequency domain feature mode and each template in the knowledge base can be calculated to determine, that is, the matching process traverses all entries in the fault knowledge base, calculates the similarity score, and selects the entry with the highest matching degree and exceeding the preset threshold as the diagnostic result. If multiple fault templates are highly matched, the system can output a composite fault prompt, and select the fault type with the highest priority as the main fault.

[0070] Step S700, according to the fault type, the corresponding control instruction is generated and sent to the main control system of the target robot through the communication interface, triggering the execution of the corresponding response strategy.

[0071] If the fault type is laser radar data anomaly, the response strategy can be to suspend the cleaning task to prevent misnavigation; if the fault type is wheel continuous idling, the response strategy can be to start the self-checking program to check the motor state; if the fault type is collision sensor frequent triggering, the response strategy can be to perform the back + steering obstacle avoidance action, etc. All control instructions can be sent to the control management module of the main control system through the ROS service mechanism to ensure real-time and reliability. In addition, the system can also record the related information (timestamp, frequency domain feature mode, diagnostic result, response action) of this abnormal event to the local log file for subsequent analysis and use.

[0072] The embodiment realizes intelligent, refined and closed-loop monitoring of the running state of the robot by constructing a ROS-based multi-sensor frequency domain transformation preprocessing adaptive fusion monitoring system and method. Specifically, the embodiment extracts the energy distribution characteristics in the key frequency interval as the target frequency domain feature component by performing discrete Fourier transform on the time series data of various sensors, and then can capture the periodic changes and potential degradation trends in the device running from the frequency domain perspective, and then can improve the sensitivity and recognition ability to early abnormalities. On this basis, a weighted fusion algorithm is used to cooperatively analyze the multi-source heterogeneous frequency domain features, and a dynamically updated judgment threshold and time persistence judgment mechanism are combined to realize state evaluation in a complex and variable environment, and to reduce the false positive rate and the missed detection rate. Further, by constructing a reference spectrum model and a fault knowledge base, the system can automatically complete pattern matching and diagnosis of the fault type after detecting an abnormality, and trigger the corresponding control instructions to form a closed-loop response, thereby reducing the need for manual intervention and improving the autonomous operation and maintenance capability of the device.

[0073] The application also provides a robot, which exemplarily comprises a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program, so that the terminal device executes the functions of each module of the robot monitoring method or the robot monitoring system.

[0074] The processor can be an integrated circuit chip with a signal processing capability. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or at least one of them. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the application.

[0075] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0076] This application also provides a computer-readable storage medium for storing the computer program used in the robot described above. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0078] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0079] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0080] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A robot monitoring method, characterized in that, include: Real-time acquisition of runtime sequence data from various sensors mounted on the target robot; The runtime sequence data is frequency domain transformed, and the energy distribution characteristics within a preset frequency range are extracted as target frequency domain feature components. The target frequency domain feature components of each sensor are fused to obtain fused spectrum data, and the fused spectrum data is compared with a pre-constructed reference spectrum model. If the comparison result exceeds the judgment threshold, the target robot is determined to be malfunctioning.

2. The robot monitoring method according to claim 1, characterized in that, The construction of the reference spectrum model includes: Collect sample time-series data output from various sensors on a sample robot of the same type as the target robot, and perform frequency domain transformation on the sample time-series data to obtain sample spectrum data; The energy distribution characteristics of each sensor in the corresponding preset frequency range are extracted from the sample spectrum data as sample frequency domain feature components. Based on the frequency domain feature components of the samples, a statistical analysis method is used to determine the reference spectrum model that characterizes the normal operating state of the target robot.

3. The robot monitoring method according to claim 1, characterized in that, If the comparison result exceeds the judgment threshold, the target robot is determined to be malfunctioning, including: If the deviation between the fused spectrum data and the reference spectrum model is greater than the current judgment threshold and the duration exceeds the set delay time, the target robot is confirmed to be in an abnormal operating state.

4. The robot monitoring method according to claim 3, characterized in that, The determination threshold is determined in real time based on the mean and standard deviation of the target frequency domain feature components corresponding to the target robot within the sliding window, combined with the confidence coefficient.

5. The robot monitoring method according to claim 4, characterized in that, The confidence coefficient is determined based on the accuracy level of each sensor, the risk level of the application scenario, and the historical failure rate.

6. The robot monitoring method according to claim 1, characterized in that, The step of fusing the target frequency domain feature components of each of the sensors to obtain fused spectrum data includes: The target frequency domain feature components of each of the sensors are weighted and fused to obtain the fused spectrum data characterizing the current operating state of the target robot; wherein the weights of the weighted fusion are determined based on the measurement noise variance of each of the sensors.

7. The robot monitoring method according to claim 1, characterized in that, After determining that the target robot is malfunctioning, the process further includes: Extract the target frequency domain feature components of each sensor during the abnormal operation period, form a frequency domain feature vector group, and use it as the frequency domain feature pattern of this abnormal event; The frequency domain feature pattern is matched with a pre-stored fault knowledge base to determine the fault type; Based on the fault type, a corresponding control command is generated and sent to the main control system of the target robot through the communication interface to trigger the execution of the corresponding response strategy.

8. A robot monitoring system, characterized in that, include: The data acquisition module is used to collect runtime sequence data from the various sensors mounted on the target robot in real time. The conversion module is used to perform frequency domain conversion on the runtime sequence data and extract the energy distribution characteristics within a preset frequency range as target frequency domain feature components. The comparison module is used to fuse the target frequency domain feature components of each of the sensors to obtain fused spectrum data, and compare the fused spectrum data with a pre-built reference spectrum model. The judgment module is used to determine that the target robot is malfunctioning if the comparison result exceeds the judgment threshold.

9. A robot, characterized in that, The robot includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the robot monitoring method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed on a processor, implements the robot monitoring method according to any one of claims 1-7.

Images