Fault state monitoring method, device and equipment for mobile robot

Abstract

The present invention discloses a method, device and equipment for monitoring the fault status of a mobile robot. The method comprises: obtaining at least two status data of each device to be monitored in the mobile robot; inputting the at least two status data into a pre-trained associated fault monitoring model of each device to be monitored to obtain monitoring results of each device to be monitored; determining the faulty device in the mobile robot according to the monitoring results to locate the cause of the fault of the mobile robot. The technical solution of the present application realizes accurate positioning of the cause of the fault of the mobile robot by performing targeted monitoring on all sensors and / or hardware devices in the mobile robot; and the present solution is based on the characteristic association relationship between at least two status data of each sensor or hardware device, thereby improving the accuracy of monitoring each device and avoiding the influence caused by inaccurate single status data.

Description

Technical Field

[0001] The present invention relates to the technical field of robot fault monitoring, and in particular to a method, device and equipment for monitoring the fault state of a mobile robot. Background Art

[0002] With the development of computer theory, electronic information technology, automatic control theory, mechanical automation and other disciplines and the application of new materials, mobile robot technology has made great progress. As an important component of mobile robots, the sensors and hardware installed on mobile robots have important data accuracy and integrity for the decision-making and control of mobile robots. Once the mobile robot sensor is abnormal, it will have a great impact on the driving safety of the mobile robot.

[0003] In the prior art, the mobile robot is monitored as a whole by acquiring the operating status data of the mobile robot, which can only determine that a fault occurs in the mobile robot, but cannot accurately locate the specific position of the mobile robot. Technical personnel are still needed to conduct troubleshooting, which greatly increases the workload of the technical personnel and also affects the normal operation of the mobile robot. Therefore, it is very important to accurately locate the fault position of the mobile robot. Summary of the invention

[0004] The present invention provides a method, device and equipment for monitoring the fault status of a mobile robot to solve the problem of inaccurate monitoring of different types of monitoring devices, achieve effective and accurate monitoring and prediction of the monitoring devices, and further accurately locate the fault position of the mobile robot.

[0005] According to one aspect of the present invention, a method for monitoring a fault state of a mobile robot is provided, the method comprising:

[0006] Acquire at least two status data of each device to be monitored in the mobile robot; wherein the device to be monitored includes all sensor devices and / or hardware devices installed in the mobile robot;

[0007] Inputting the at least two state data into a pre-trained associated fault monitoring model of each device to be monitored to obtain a monitoring result of each device to be monitored; wherein the fault monitoring model is used to learn the characteristic association relationship between the at least two state data;

[0008] A faulty device in the mobile robot is determined according to the monitoring result to locate the cause of the fault of the mobile robot.

[0009] According to another aspect of the present invention, there is provided a fault status monitoring device for a mobile robot, the device comprising:

[0010] A data acquisition module, used to acquire at least two status data of each device to be monitored in the mobile robot; wherein the device to be monitored includes all sensor devices and / or hardware devices installed in the mobile robot;

[0011] A detection module, used for inputting the at least two state data into a pre-trained associated fault monitoring model of each device to be monitored, to obtain a monitoring result of each device to be monitored; wherein the fault monitoring model is used for learning a characteristic association relationship between the at least two state data;

[0012] A positioning module is used to determine a faulty device in the mobile robot according to the monitoring result so as to locate the cause of the fault of the mobile robot.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the fault status monitoring method of the mobile robot described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for monitoring the fault status of a mobile robot described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention realizes accurate positioning of the cause of failure of the mobile robot by performing targeted monitoring of all sensors and / or hardware devices in the mobile robot. In addition, this solution is based on the characteristic correlation relationship between at least two status data of each sensor or hardware device, thereby improving the accuracy of monitoring of each device and avoiding the impact caused by inaccurate single status data.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a method for monitoring a fault state of a mobile robot provided according to Embodiment 1 of the present invention;

[0022] Figure 2 is a flow chart of a method for monitoring a fault state of a mobile robot provided in accordance with Embodiment 2 of the present invention;

[0023] Figure 3 is a schematic structural diagram of a mobile robot fault status monitoring device provided in accordance with Embodiment 3 of the present invention;

[0024] Figure 4 It is a structural schematic diagram of an electronic device for implementing the method for monitoring the fault state of a mobile robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "sample", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1This is a flow chart of a mobile robot fault status monitoring method provided according to the first embodiment of the present invention. This embodiment can be applied to the situation of monitoring the running status of a mobile robot such as an AGV. The method can be executed by a mobile robot fault status monitoring device. The mobile robot fault status monitoring device can be implemented in the form of hardware and / or software. The mobile robot fault status monitoring device can be configured in an electronic device having a mobile robot fault status monitoring method. Figure 1 As shown, the method includes:

[0029] S110, obtaining at least two status data of each device to be monitored in the mobile robot; wherein the device to be monitored includes all sensor devices and / or hardware devices installed in the mobile robot.

[0030] The mobile robot senses the environment around the vehicle through various monitoring devices, and controls the steering and speed of the mobile robot based on the road, mobile robot position and obstacle information obtained through perception, so that the mobile robot can travel on the road safely and reliably. Unmanned vehicles, also known as autonomous driving vehicles, are a type of mobile robot.

[0031] The monitoring device includes all sensor devices and / or hardware devices installed in the mobile robot, for example, the sensor devices include but are not limited to lidar sensors, cameras (depth cameras, surround view cameras), ultrasonic sensors, anti-fall sensors, in-place detection sensors, and pallet identification sensors, and the hardware devices include but are not limited to motors and navigation module power supplies.

[0032] The state data is used to describe the data describing the operation status of the mobile robot obtained by each device to be monitored during operation, such as the current, voltage, speed or temperature of each sensor during operation, to determine whether there is an abnormality in the operation of each device to be monitored. In addition, at least two of the acquired state data have a certain correlation, so as to facilitate learning the correlation between the data through machine learning methods, and then determine whether there is an abnormality in the device to be monitored.

[0033] S120, inputting the at least two state data into a pre-trained associated fault monitoring model of each device to be monitored to obtain a monitoring result of each device to be monitored; wherein the fault monitoring model is used to learn the characteristic association relationship between the at least two state data.

[0034] Among them, the fault monitoring model is pre-trained on the cloud platform and deployed on the mobile robot for monitoring after training.

[0035] Among them, the characteristic correlation relationship between at least two types of state data refers to the mutual influence relationship between the state data output by each monitored device, for example, there is a linear relationship between the voltage value and the current value, or there is a mutual influence relationship between the voltage value and the temperature value. If any state data changes, and the corresponding change in the state data with the correlation relationship does not conform to the pre-learned correlation relationship, it means that there is an abnormal phenomenon in the corresponding device.

[0036] The fault monitoring model learns the characteristic correlation between at least two state data, which can more accurately determine the state changes of each monitored device, avoiding the misjudgment of the monitored device due to the judgment of a single state data. For example, if the fault result of the device is directly determined based on the comparison result of a single state data and the corresponding threshold, the occasional abnormality of the single state data will cause an error in the judgment of the device fault result.

[0037] The monitoring result may refer to whether the operating status of each device to be monitored is abnormal. The monitoring result may be used to guide the technicians to perform timely maintenance on the mobile robot.

[0038] Specifically, during the operation of the mobile robot, the status data of each device to be monitored will be recorded and uploaded to the cloud platform. Since there are many mobile robots and the amount of data is huge, running the fault monitoring model on the cloud platform is conducive to improving the efficiency of fault monitoring model training. The fault monitoring model is trained with large-scale massive data, and after continuous iteration and convergence of the model, the trained associated fault monitoring model of each device to be monitored is obtained, and then the trained associated fault monitoring model is deployed to the mobile robot side to monitor the mobile robot in real time. The associated fault monitoring model runs on the mobile robot side, which can monitor the data of each device to be monitored in real time, obtain the operating status and life cycle of each device to be monitored, etc., which is conducive to guiding technical personnel to perform operation and maintenance work. At the same time, it avoids the delay problem caused by deployment on the cloud platform and ensures the accuracy of the data.

[0039] Optionally, since different devices to be monitored have different functions and different real-time requirements for the tasks to be performed, in order to ensure full utilization of the resources on the robot side, different monitoring frequencies are set for each device to be monitored, that is, the status data of each device to be monitored is obtained at different frequencies. For example, for sensors with motors such as laser radar sensors, high frequencies can be used to monitor data, and for cameras, some circuit sensors or pure electronic components, low frequencies can be used to monitor data. Optionally, the average value or mean square error of the voltage and current of each device to be monitored within a monitoring cycle can be monitored. In addition, when acquiring status data, outlier points whose number is less than a preset number are filtered. Among them, outlier points are abnormal numerical points, for example, data exceeding a preset threshold. When the outlier points are within a preset number, it can be considered that the abnormal data belongs to accidental data, and the outlier points can be filtered out. However, when the outlier points exceed the preset number, it will affect the monitoring results and will not be filtered. This reduces the amount of data transmission and ensures the accuracy of the monitoring results.

[0040] In a feasible embodiment, the training process of the associated fault monitoring model of the device to be monitored includes the following steps A1-A3:

[0041] Step A1: Classify the devices to be monitored based on the device type.

[0042] Step A2: Obtain at least two types of sample status data and corresponding status results for each type of device to be monitored.

[0043] Step A3: training the at least two sample status data and the corresponding status results based on a deep learning network to obtain an associated fault monitoring model for each type of device to be monitored.

[0044] The sample status data may be all data related to the device to be monitored, for example, the physical status values ​​of the hardware monitored by devices such as pulse code modulation (PCM) or vehicle communication units (VCU), such as voltage and current values; and data sensed by sensing sensors, such as temperature values ​​sensed by temperature sensors. The status result may refer to the status of the device to be monitored under each sample status data, that is, whether the device to be monitored is in a normal state or an abnormal state under different sample status data.

[0045] Specifically, the correlation between the status data output by different devices is different, and the devices with the same or similar correlation are located as the same type of devices, so that the devices to be monitored can be classified. For example, if the monitoring status data of several devices to be monitored in the mobile robot are the same, and the characteristic correlation between the status data is the same, that is, the correlation between the status data and the monitoring results is also the same, then these devices to be monitored can be determined as devices of the same type, and the same associated fault monitoring model can be used to monitor the devices to be monitored of the same type. After the devices to be monitored are classified according to the relationship between the status data of the devices to be monitored, the model of each type of device to be monitored is trained separately to facilitate the monitoring of the devices to be monitored. Before training, at least two sample status data and corresponding status results of each type of device to be monitored are obtained in advance, so that the accuracy of the model can be higher. Finally, the at least two sample status data and corresponding status results of each type of device to be monitored are obtained, and the deep learning network is used to learn to obtain an accurate associated fault monitoring model for each type of device to be monitored.

[0046] In a feasible embodiment, training the at least two sample status data and the corresponding status results based on a deep learning network to obtain an associated fault monitoring model for each type of device to be monitored may include the following steps B1-B3:

[0047] Step B1: learning the feature association relationship between the at least two sample state data, and the direct association relationship between the feature association relationship and the corresponding state result based on a deep learning network.

[0048] Step B2: Based on the deep learning network, learn the feature change relationship between at least two types of sample state data under different data acquisition time information, and the trend change relationship between the feature change relationship and the corresponding state result change information.

[0049] Step B3: determining an associated fault monitoring model for each type of device to be monitored according to the direct association relationship and the trend change relationship.

[0050] The sample status data includes data acquisition time information, which refers to the time when the sample status data is acquired.

[0051] Specifically, at least two sample state data of the device to be monitored are obtained, and the characteristic association relationship between at least two sample state data and the direct association relationship between the characteristic association relationship and the corresponding state result are learned based on the deep learning network. For example, the current value and voltage value of the device to be monitored are obtained, and the characteristic association relationship between the current value and the voltage value is learned based on the deep learning network. At the same time, the relationship between the characteristic association relationship between the current value and the voltage value and the state result of the device to be monitored is learned based on the deep learning network to obtain the direct association relationship between the characteristic association relationship and the corresponding state result. In addition, it is also necessary to learn the characteristic change relationship between at least two sample state data under different data acquisition time information and the trend change relationship between the characteristic change relationship and the corresponding state result change information based on the deep learning network, so as to understand the corresponding change of the state data during the time period when the device to be monitored is abnormal. Finally, the associated fault monitoring model of each type of device to be monitored is obtained by training based on the direct association relationship and the trend change relationship, so that the obtained associated fault monitoring model is more accurate, and accurate real-time monitoring and prediction of the state of the device to be monitored are achieved.

[0052] The technical solution learns the direct correlation relationship between the feature correlation relationship of at least two types of sample state data and the corresponding state results based on a deep learning network, and learns the trend change relationship between the feature change relationship of at least two types of sample state data and the corresponding state result change information under different data acquisition time information based on a deep learning network. This ensures that the data granularity is sufficient during the training process, and further ensures the accuracy of the direct correlation relationship and the trend change relationship, which is more conducive to obtaining an accurate correlation fault monitoring model for each type of device to be monitored through the direct correlation relationship and the trend change relationship, and realizes more accurate real-time monitoring and prediction of the devices to be monitored of the mobile robot.

[0053] In a feasible embodiment, the at least two sample state data and the corresponding state results are trained based on a deep learning network to obtain an associated fault monitoring model for each type of device to be monitored, including:

[0054] Based on the deep learning network, the at least two sample state data and corresponding state results under the different operating parameters are trained to obtain an associated fault monitoring model for each type of device to be monitored under different operating conditions.

[0055] The sample state data includes sample state data under different operating parameters.

[0056] The operating condition parameters may be parameter information of the device to be monitored under different operating conditions, for example, parameters of the device to be monitored under abnormal conditions such as rain, snow or high temperature.

[0057] Specifically, the monitoring device of the mobile robot will generate abnormal status data under different working conditions, but this does not mean that the monitored device has an abnormality. Therefore, the associated fault monitoring model trained based on the status data generated by the monitoring device under normal conditions cannot be used. It is necessary to use at least two types of sample status data and corresponding status results under different working conditions parameters to train the fault monitoring model to obtain an accurate associated fault monitoring model, thereby avoiding misjudging the monitoring device as being in an abnormal state under different working conditions.

[0058] The technical solution, by training at least two kinds of sample state data and corresponding state results under different working parameters based on a deep learning network, makes it possible to obtain an associated fault monitoring model for each type of monitored device under different working conditions, thereby avoiding the use of the same associated fault monitoring model under different working conditions to monitor the monitored device, resulting in inaccurate monitoring results and affecting the normal operation of the mobile robot.

[0059] S130. Determine a faulty device in the mobile robot according to the monitoring result to locate the cause of the fault of the mobile robot.

[0060] Specifically, the monitoring results of each device to be monitored are obtained through the pre-trained associated fault monitoring model of each device to be monitored. According to the monitoring results, the faulty device in the mobile robot can be accurately judged, and then the cause of the failure of the mobile robot can be accurately understood, thereby achieving accurate positioning of the fault position of the mobile robot, which is convenient for technicians to repair the robot.

[0061] In a feasible embodiment, after determining the faulty device in the mobile robot according to the monitoring result, the method further includes steps C1-C2:

[0062] Step C1: determining an auxiliary sensor of the faulty sensor based on a predetermined mapping relationship.

[0063] Step C2: operating the auxiliary sensor based on a preset auxiliary mechanism to supplement the function of the faulty sensor.

[0064] The auxiliary sensor can be used to ensure that when a problem occurs in a sensor of the mobile robot, the function of the auxiliary sensor can be used in time to supplement the robot's faulty function to ensure the normal operation of the mobile robot. A corresponding auxiliary sensor is determined for each sensor in the mobile robot in advance according to the function of each sensor.

[0065] Specifically, auxiliary sensors will be matched to some sensors that are prone to problems in the mobile robot to ensure that when a sensor fails, the auxiliary sensor is promptly activated based on the pre-set auxiliary mechanism to ensure the normal operation of the mobile robot. For example, when the mobile robot is operating in abnormal weather (such as rain, snow, fog, etc.), when the point cloud data collected by the lidar sensor is abnormal, the mobile robot can be reminded through monitoring and early warning to turn on auxiliary sensors, such as auxiliary positioning sensors, to supplement the function of the failed lidar sensor.

[0066] This technical solution uses a pre-set auxiliary mechanism to activate an auxiliary sensor that has a mapping relationship with the faulty sensor to supplement the function of the faulty sensor, thereby ensuring the normal operation of the mobile robot.

[0067] The technical solution of the embodiment of the present invention obtains accurate monitoring results of each device to be monitored by inputting at least two status data of each device to be monitored in the mobile robot into a pre-trained associated fault monitoring model of each device to be monitored, wherein the associated fault monitoring model is obtained by training at least two status data of different types of monitoring devices, thereby improving the accuracy of monitoring each device and avoiding the influence caused by inaccurate single status data; because this solution performs targeted monitoring of all sensors and / or hardware devices in the mobile robot, the faulty device in the mobile robot can be accurately located according to the monitoring results, and then the cause of the failure of the mobile robot can be determined, thereby improving the accuracy of monitoring the mobile robot and realizing accurate and effective real-time monitoring and prediction of the mobile robot.

[0068] Embodiment 2

[0069] Figure 2 This is a flow chart of a method for monitoring the fault status of a mobile robot provided according to the second embodiment of the present invention. This embodiment describes in detail the monitoring results of each device to be monitored in the above embodiment. Figure 2 As shown, the method includes:

[0070] S210, obtaining at least two status data of each device to be monitored in the mobile robot.

[0071] S220: Determine a target fault monitoring model according to the type of the device to be monitored.

[0072] Specifically, the type of the device to be monitored can be determined according to the correlation between the status data of the device to be monitored, and the target fault monitoring model is trained according to the correlation between the status data of the device to be monitored, so the target fault monitoring model can be determined according to the type of the monitoring device.

[0073] S230: Input the at least two status data into the target fault monitoring model.

[0074] Specifically, during operation, the mobile robot inputs the acquired at least two kinds of status data into a target fault monitoring model determined according to the type of the device to be monitored, so as to obtain an accurate monitoring result of the device to be monitored.

[0075] In a feasible embodiment, inputting the at least two state data into the target fault monitoring model may include the following steps D1-D2:

[0076] Step D1: Based on the direct association relationship and the characteristic association relationship in the target fault monitoring model, determine the current state monitoring result according to the current state data.

[0077] Step D2: Based on the trend change relationship and the feature change relationship in the target fault monitoring model, determine the future state prediction result according to the current state data and the historical state data.

[0078] Optionally, the monitoring results include current state monitoring results and future state prediction results; the state data include current state data and historical state data.

[0079] The current state data may be data generated during the operation of the device to be monitored recorded by the mobile robot at the current moment, and the historical state data may be data generated during the historical operation of the device to be monitored recorded by the mobile robot before the current moment.

[0080] Specifically, the current state monitoring result refers to the information obtained by inputting the current state data into the target fault monitoring model based on the direct correlation relationship and feature correlation relationship in the target fault monitoring model. It is used to judge the current state of the mobile robot, and then guide the technicians to repair the designated fault location of the mobile robot, thereby achieving accurate judgment of the monitored device.

[0081] The future state prediction result refers to the possible future state change result of the mobile robot determined based on the trend change relationship and feature change relationship in the target fault monitoring model, according to the current state data and the historical state data, and is used to guide the technicians to maintain the mobile robot that is about to have problems in advance. Optionally, the future state prediction result refers to the health of the robot, that is, the health of the robot is determined based on the historical state data and the current state data of the mobile robot. When the health is lower than the preset threshold, it means that there are hidden dangers in the health of the robot and a fault problem may occur at any time. For example, in scenarios such as night operation or unmanned factories, technicians need to monitor the operating status of all the monitored devices of the mobile robot. When the health of the mobile robot drops below the preset value, it is necessary to send after-sales or operation and maintenance personnel to intervene for maintenance and inspection, so as to keep the mobile robot in normal health. For example, the preset value is 95. When the health of the mobile robot is lower than 95, the monitoring result of the mobile robot will be directly sent to the technician for processing. For example, if the mobile robot's motor, wearing parts or consumable parts are abnormal, or the mobile robot's life is about to expire, and the motor's wheelbase and axle clearance error increases, the technicians can directly replace the tires or the transmission device, which is convenient for technicians to perform maintenance, improve the level of automation, and reduce the technical requirements for operation and maintenance personnel. In addition, by predicting the future operating status of the monitoring device in advance, the technicians can be warned as early as possible to monitor or maintain the mobile robot that may have problems in the future to avoid accidents.

[0082] This technical solution obtains accurate current state monitoring results and future state prediction results by utilizing the target fault monitoring model to monitor the device to be monitored, realizes the accurate positioning of the mobile robot's fault location, and accurately predicts the future operating state of the mobile robot, and can timely and accurately maintain robots that have failed or are about to fail.

[0083] In a feasible embodiment, after obtaining the monitoring results of each device to be monitored, the method further includes the following steps E1-E2:

[0084] Step E1, determining mutually interfering devices to be monitored according to the monitoring results.

[0085] Step E2: providing independent power supply modules for the mutually interfering monitored devices.

[0086] Specifically, mutually interfering monitored devices refer to devices that fail, causing the status data of another non-faulty device to be affected and abnormal status data to appear. If the abnormality of the monitored device is judged based solely on the status data, it will cause a misjudgment of the non-faulty device. The mutual influence of the monitored devices is generally due to the same power supply. Therefore, the mutually interfering monitored devices are determined based on the monitoring results. If a device fails, the abnormal status data of another device is judged as a failure by the model, but after investigation, it is determined that the device is not faulty, then it is determined that the two devices interfere with each other. In order to avoid mutual interference among the monitored devices, independent power supply modules can be set, or the physical status can be judged independently to eliminate the influence between the mutually interfering monitored devices.

[0087] For example, if two monitored devices interfere with each other, if one of the monitored devices fails, it will affect the status data output by the other monitored device that is not faulty. It is easy to judge that both monitored devices are faulty during monitoring, which increases the workload of technicians and reduces operation and maintenance efficiency.

[0088] This technical solution, by providing independent power supply modules for mutually interfering monitored devices, avoids inaccurate status data obtained due to interference between the monitored devices, thereby affecting the monitoring results.

[0089] S240: Determine a faulty device in the mobile robot according to the monitoring result to locate the cause of the fault of the mobile robot.

[0090] The technical solution of the embodiment of the present invention obtains at least two state data of each device to be monitored in the mobile robot, and inputs the at least two state data into a pre-trained associated fault monitoring model under the current working condition of each device to be monitored according to the current working condition, so as to obtain the previous state monitoring result and the future state prediction result of each device to be monitored, thereby ensuring the accuracy of the monitoring result, and then determining the faulty device in the mobile robot according to the current monitoring result to accurately locate the cause of the failure of the mobile robot. At the same time, determining whether the device in the mobile robot is about to fail or the possible risks according to the future state prediction result, so that the technicians can perform maintenance on the mobile robot in advance to avoid accidents. The technical solution realizes the accurate positioning and prediction of the fault position of the mobile robot.

[0091] Embodiment 3

[0092] Figure 3 This is a schematic diagram of the structure of a mobile robot fault status monitoring device provided according to Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0093] The data acquisition module 310 is used to acquire at least two status data of each device to be monitored in the mobile robot; wherein the device to be monitored includes all sensor devices and / or hardware devices installed in the mobile robot.

[0094] The detection module 320 is used to input the at least two state data into a pre-trained associated fault monitoring model of each device to be monitored to obtain a monitoring result of each device to be monitored; wherein the fault monitoring model is used to learn the characteristic association relationship between the at least two state data.

[0095] The positioning module 330 is used to determine the faulty device in the mobile robot according to the monitoring result to locate the cause of the fault of the mobile robot.

[0096] Among them, the monitoring results include current state monitoring results and future state prediction results; the state data includes current state data and historical state data.

[0097] Optionally, the detection module includes a model training unit, which is specifically used to:

[0098] Classify the devices to be monitored based on device type;

[0099] Respectively obtain at least two types of sample status data and corresponding status results for each type of device to be monitored;

[0100] Training the at least two sample status data and corresponding status results based on a deep learning network to obtain an associated fault monitoring model for each type of device to be monitored;

[0101] Optional, detection module, specifically used for:

[0102] Determining a target fault monitoring model according to the type of the device to be monitored;

[0103] The at least two status data are input into the target fault monitoring model.

[0104] Optionally, the model training unit includes a first type of model training unit, which is specifically used for:

[0105] Learning the feature association relationship between the at least two sample state data and the direct association relationship between the feature association relationship and the corresponding state result based on the deep learning network;

[0106] Learning the characteristic change relationship between at least two types of sample state data under different data acquisition time information based on the deep learning network, and the trend change relationship between the characteristic change relationship and the corresponding state result change information;

[0107] The associated fault monitoring model of each type of device to be monitored is determined according to the direct association relationship and the trend change relationship.

[0108] The sample status data includes data acquisition time information.

[0109] Optionally, the detection module includes a result determination unit, which is specifically used to:

[0110] Based on the direct association relationship and the characteristic association relationship in the target fault monitoring model, determining a current state monitoring result according to the current state data;

[0111] Based on the trend change relationship and the feature change relationship in the target fault monitoring model, a future state prediction result is determined according to the current state data and the historical state data.

[0112] Optionally, the model training unit includes a second type of model training unit, which is specifically used for:

[0113] Based on the deep learning network, the at least two sample state data and corresponding state results under the different operating parameters are trained to obtain an associated fault monitoring model for each type of device to be monitored under different operating conditions.

[0114] The sample state data includes sample state data under different operating parameters.

[0115] Optionally, the detection module further includes a do not disturb unit, which is specifically used to:

[0116] Determining mutually interfering devices to be monitored according to the monitoring results;

[0117] Independent power supply modules are provided for the mutually interfering monitored devices.

[0118] Optionally, the positioning module further includes a replacement unit, which is specifically used for:

[0119] determining an auxiliary sensor of the faulty sensor based on a predetermined mapping relationship;

[0120] The auxiliary sensor is operated based on a preset auxiliary mechanism to supplement the function of the faulty sensor.

[0121] Among them, the fault monitoring model is pre-trained on the cloud platform and deployed on the mobile robot for monitoring after training.

[0122] The mobile robot fault state monitoring device provided in the embodiment of the present invention can execute the mobile robot fault state monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0123] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0124] Embodiment 4

[0125] Figure 4 The structural schematic diagram of the electronic device that can be used to implement the fault state monitoring method of the mobile robot of the embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0126] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0127] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the method for monitoring the fault state of a mobile robot.

[0129] In some embodiments, the method for monitoring the fault state of the mobile robot can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for monitoring the fault state of the mobile robot described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the method for monitoring the fault state of the mobile robot in any other appropriate manner (for example, by means of firmware).

[0130] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0132] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0134] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0135] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0136] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0137] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring the fault status of a mobile robot, characterized in that: include: Acquire at least two status data of each device to be monitored in the mobile robot; wherein the device to be monitored includes all sensor devices installed in the mobile robot and / or other hardware devices except the sensor devices; Inputting the at least two state data into a pre-trained associated fault monitoring model of each device to be monitored to obtain a monitoring result of each device to be monitored; wherein the fault monitoring model is used to learn the characteristic association relationship between the at least two state data; Determining a faulty device in the mobile robot according to the monitoring result to locate the cause of the fault of the mobile robot; The training process of the associated fault monitoring model of the device to be monitored is as follows: Classify the devices to be monitored based on device type; Respectively obtain at least two types of sample status data and corresponding status results for each type of device to be monitored; Training the at least two sample status data and corresponding status results based on a deep learning network to obtain an associated fault monitoring model for each type of device to be monitored; Accordingly, the at least two state data are input into a pre-trained associated fault monitoring model of each device to be monitored, including: Determining a target fault monitoring model according to the type of the device to be monitored; The at least two status data are input into the target fault monitoring model.

2. The method according to claim 1, characterized in that in, The sample status data includes data acquisition time information; Accordingly, the at least two sample status data and the corresponding status results are trained based on the deep learning network to obtain an associated fault monitoring model for each type of device to be monitored, including: Learning the feature association relationship between the at least two sample state data and the direct association relationship between the feature association relationship and the corresponding state result based on the deep learning network; Learning the characteristic change relationship between at least two types of sample state data under different data acquisition time information based on the deep learning network, and the trend change relationship between the characteristic change relationship and the corresponding state result change information; The associated fault monitoring model of each type of device to be monitored is determined according to the direct association relationship and the trend change relationship.

3. The method according to claim 2, characterized in that in, The monitoring results include current state monitoring results and future state prediction results; the state data include current state data and historical state data; Accordingly, inputting the at least two state data into the target fault monitoring model includes: Based on the direct association relationship and the characteristic association relationship in the target fault monitoring model, determining a current state monitoring result according to the current state data; Based on the trend change relationship and the feature change relationship in the target fault monitoring model, a future state prediction result is determined according to the current state data and the historical state data.

4. The method according to claim 1, characterized in that: The sample state data includes sample state data under different operating parameters; Accordingly, the at least two sample status data and the corresponding status results are trained based on the deep learning network to obtain an associated fault monitoring model for each type of device to be monitored, including: Based on the deep learning network, the at least two sample state data and corresponding state results under the different operating parameters are trained to obtain an associated fault monitoring model for each type of device to be monitored under different operating conditions.

5. The method according to claim 1, characterized in that: After obtaining the monitoring results of each device to be monitored, the method further includes: Determining mutually interfering devices to be monitored according to the monitoring results; Independent power supply modules are provided for the mutually interfering monitored devices.

6. The method according to claim 1, characterized in that After determining the faulty device in the mobile robot according to the monitoring result, the method further includes: determining an auxiliary sensor of the faulty sensor based on a predetermined mapping relationship; The auxiliary sensor is operated based on a preset auxiliary mechanism to supplement the function of the faulty sensor.

7. The method according to claim 1, characterized in that The fault monitoring model is pre-trained on the cloud platform and deployed on the mobile robot for monitoring after training.

8. A mobile robot fault status monitoring device, characterized in that: include: A data acquisition module, used to acquire at least two status data of each device to be monitored in the mobile robot; wherein the device to be monitored includes all sensor devices installed in the mobile robot and / or other hardware devices except the sensor devices; A detection module, used for inputting the at least two state data into a pre-trained associated fault monitoring model of each device to be monitored, to obtain a monitoring result of each device to be monitored; wherein the fault monitoring model is used for learning a characteristic association relationship between the at least two state data; A positioning module, used to determine a faulty device in the mobile robot according to the monitoring result, so as to locate the cause of the fault of the mobile robot; The detection module includes a model training unit, which is specifically used for: Classify the devices to be monitored based on device type; Respectively obtain at least two types of sample status data and corresponding status results for each type of device to be monitored; Training the at least two sample status data and corresponding status results based on a deep learning network to obtain an associated fault monitoring model for each type of device to be monitored; Accordingly, the detection module is specifically used for: Determining a target fault monitoring model according to the type of the device to be monitored; The at least two status data are input into the target fault monitoring model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the fault status monitoring method for a mobile robot according to any one of claims 1-7.

Images