Intelligent fault diagnosis robot based on deep learning

Abstract

The invention discloses an intelligent fault diagnosis robot based on deep learning, and the robot specifically comprises a robot body, the robot body is internally provided with a control system, and the control system comprises a master control module, a sensor module, a motion control module, a visual recognition module, and a power management module; the master control module is connected with the sensor module, the motion control module, the visual identification module and the power management module. The sensor module comprises a camera sensor, an ultrasonic sensor and an infrared sensor; the motion control module is used for setting the walking range of the robot; the visual identification module comprises an identification unit, a display and a wireless communication unit; the power management module is provided with a monitor, a power supply unit and a control management unit. The robot provided by the invention can accurately arrive at a target location in an industrial scene so as to identify and diagnose fault equipment, and meanwhile, a power supply is managed, so that the working continuity of the robot is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis, and particularly to an intelligent fault diagnosis robot based on deep learning. Background Art

[0002] In the field of industrial production, various industrial equipment are widely used and need to be put into production. However, industrial equipment often fails. Traditional fault detection methods rely on manual detection, which not only requires a large amount of labor costs, but also it is difficult to detect faulty equipment in real time due to the intermittency of detection, which will lead to the stagnation of the production line, greatly reducing production efficiency and causing economic losses to enterprises.

[0003] To solve the above problems, robots for detecting faulty equipment have emerged as the times require. Such robots can work continuously, effectively improving production efficiency and making up for the deficiencies of manual detection. However, the existing robots for detecting faulty equipment have obvious defects. For example, in terms of identifying faulty equipment, insufficient computing power makes it difficult for them to quickly and accurately analyze a large amount of equipment data, and the accuracy of extracting and judging the fault characteristics of the equipment is relatively low, and misjudgment or missed judgment is likely to occur. In addition, the travel route planning of the robot in the industrial scenario is not intelligent enough, and situations such as being blocked or detoured often occur, affecting the detection efficiency and timeliness. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an intelligent fault diagnosis robot based on deep learning, which can accurately reach the target location in the industrial scenario, and then identify and diagnose faulty equipment, while managing the power supply to ensure the continuity of the robot's work.

[0005] The embodiment of the present invention provides an intelligent fault diagnosis robot based on deep learning, including:

[0006] A robot body, in which a control system is provided. The control system includes a total control module, a sensor module, a motion control module, a visual recognition module, and a power management module; the total control module is respectively connected to the sensor module, the motion control module, the visual recognition module, and the power management module;

[0007] The total control module is provided with a first ESP8266 WIFI module, a second ESP8266 WIFI module, an STM32F401RCT6 single-chip microcomputer, and an L298N driver chip. The first ESP8266 WIFI module is respectively connected to the ultrasonic sensor and the STM32F401RCT6 single-chip microcomputer. The STM32F401RCT6 single-chip microcomputer is connected to the infrared sensor. The second ESP8266 WIFI module is connected to the L298N driver chip. The L298N driver chip is connected to the motion control module;

[0008] The sensor module includes a camera sensor, an ultrasonic sensor, and an infrared sensor. The camera sensor is used to capture the actual image of the surrounding environment, including the actual image of the device. The ultrasonic sensor and the infrared sensor are used to sense the position of an object within a set distance, including the position of the device within a set distance;

[0009] The motion control module is provided with a drive motor. The motion control module is used to set the walking range of the robot and control the robot to reach the target position within the walking range through a path planning method based on the A* algorithm;

[0010] The visual recognition module includes an identification unit, a display, and a wireless communication unit. The identification unit is used to identify and diagnose the real image of the device through an improved YOLO model to obtain the fault type of the device. The display is used to display the fault type of the device and the position of the device. The wireless communication unit is used to send the fault type of the device and the position of the device to the control terminal;

[0011] The power management module is provided with a monitor, a power supply unit, and a control management unit. The monitor is used to monitor the power and working status of the robot in real time. The power supply unit is provided with a solar cell and a wireless charging battery for supplying power to the robot. The control management unit is used to perform power supply management on the power supply unit.

[0012] Preferably, the path planning method based on the A* algorithm includes:

[0013] Establish an environmental map with grids and initialize the open list and the closed list;

[0014] Start searching for a path through a five-way search strategy based on coordinate differences. Select the node with the smallest total cost from the open list as the current node and add the current node to the closed list;

[0015] Determine whether the current node is the target node. If so, the path search is successful, and the path is traced back through the parent node. If not, add the current node to the closed list and traverse all adjacent nodes of the current node.

[0016] If the adjacent node is not in the open list and the closed list, add the adjacent node to the open list and record the parent node of the adjacent node as the current node.

[0017] If the adjacent node is in the open list and the new actual cost is smaller, update the total cost and the parent node of the adjacent node.

[0018] If the current node is the target node or the open list is empty, end the path search.

[0019] Preferably, the open list uses a priority queue and is sorted according to the total cost of the node. The node with the smallest total cost is preferentially selected to store the nodes to be expanded. The closed list directly stores the expanded nodes using a set. When initializing the open list and the closed list, the starting node is added to the open list, and at the same time, the closed list is made empty.

[0020] Preferably, the total cost is represented by the following formula:

[0021] f(n) = g(n) + h(n)g(n)

[0022] In the formula, g(n) is the actual cost from the starting node to the current node, and h(n) is a heuristic function used to estimate the cost from the current node to the target node.

[0023] h(n) is represented by the following formula:

[0024] h(n) = (D × straight_step + D2 × diagonal_step) × p

[0025] In the formula, D is the cost of moving in the horizontal or vertical direction, with a value of 1, and D2 is the cost of moving in the diagonal direction, with a value of straight_step is the number of steps in the horizontal or vertical direction in the shortest path from the current node to the target node, and diagonal_step is the number of steps in the diagonal direction in the shortest path from the current node to the target node; p is a weight parameter used to dynamically adjust the weight of the heuristic function, and its value range is 0.8 to 1.5.

[0026] Preferably, the real image of the device is recognized and diagnosed through an improved YOLO model, including:

[0027] Preprocess the sample image and divide the sample image into a first data set and a second data set;

[0028] Adjust the weight parameters of the convolutional kernel of the YOLOV8 model, combine the attention optimization mechanism, and adopt a knowledge distillation method based on multi-teacher model collaboration to optimize the YOLOV8n model through the first data set and the YOLOV8 model;

[0029] Train the YOLOV8n model through the second data set, and test and verify the YOLOV8n model after training.

[0030] Use the tested and verified YOLOV8n model to recognize and diagnose the real image of the device.

[0031] Preferably, the preprocessing of the sample image includes: processing the gray value of the sample image; normalizing the sample image using the normalization parameter.

[0032] Preferably, combining the attention optimization mechanism, adopting a knowledge distillation method based on multi-teacher model collaboration, and optimizing the YOLOV8n model through the first data set and the YOLOV8 model includes:

[0033] Set multiple YOLOV8 models as teacher models and the YOLOV8n model as a student model, and divide the first data set into a first training set, a first test set, and a first validation set.

[0034] Use the teacher model to extract features from the first training set and input the extracted features as soft labels into the student model.

[0035] Combine the cross-entropy loss function, the attention distillation loss function, and the feature distillation loss function to form a total loss function, and train the student model.

[0036] Test and verify the student model through the first test set and the first validation set to optimize the performance of the student model.

[0037] Preferably, the total loss function L total has the following expression:

[0038] L total = LCE + λ1·LAD + λ2·LFD

[0039] Wherein, LCE is the cross-entropy loss function, LAD is the attention distillation loss function, and LFD is the feature distillation loss function; λ1 and λ2 are the weights of the attention distillation loss function and the feature distillation loss function respectively, which are used to balance the contributions of the cross-entropy loss function and the distillation loss function.

[0040] Preferably, the monitor includes a power monitor and a working state monitor. The power monitor is used to monitor the power of the robot, and the working state monitor is used to monitor the working state of the robot.

[0041] The solar cell is provided with a solar panel, which directly powers the robot; the wireless charging battery can be inductively coupled with a charging device located in the wireless charging area to power the robot.

[0042] Preferably, the control and management unit performs power supply management on the power supply unit, including: the control and management unit adjusts the charging speed and / or charging power of the power supply unit to the robot according to the power and working state of the robot.

[0043] The embodiments of the present invention bring the following beneficial effects:

[0044] The robot body provided in the present application is provided with a control system, and the control system is provided with a sensor module, a motion control module, a visual recognition module, and a power management module.

[0045] Among them, through the information collection of the surrounding environment by the sensor module, the motion control module enables the robot to achieve autonomous navigation and obstacle avoidance, and reach the target location more accurately. The visual recognition module reduces the capacity of the recognition model, and then realizes the lightweight of the carrier on which the recognition model depends. It can not only accurately identify and diagnose faulty equipment, but also make the robot's actions more flexible. In addition, the power management module can enable the robot to dynamically adjust its own power consumption according to the task requirements, achieving efficient utilization of energy.

[0046] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, the claims, and the drawings.

[0047] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following preferred embodiments are specifically described below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 Schematic diagram of the control system structure of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0050] Figure 2 Front view of the robot body of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0051] Figure 3 Side view of the robot body of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0052] Figure 4 Schematic diagram of the effect of identifying PC problems of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0053] Figure 5 Schematic diagram of the effect of identifying VFD problems of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention.

[0054] Figure 6 Schematic diagram of the confusion matrix after training of the improved YOLO image recognition model of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0055] Figure 7 Schematic diagram of the confusion matrix after normalization of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0056] Figure 8 Schematic diagram of the F1-confidence curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0057] Figure 9 Schematic diagram of the precision-confidence curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0058] Figure 10 Schematic diagram of the precision-recall curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by the embodiment of the present invention;

[0059] Figure 11 Schematic diagram of the recall-confidence curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by an embodiment of the present invention;

[0060] Figure 12 Schematic diagram of the training loss function curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by an embodiment of the present invention;

[0061] Figure 13 Schematic diagram of the validation loss function curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by an embodiment of the present invention;

[0062] Figure 14 Schematic diagram of the performance index curve of the YOLOV8n model of the intelligent fault diagnosis robot based on deep learning provided by an embodiment of the present invention. Detailed implementation manners

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0064] For ease of understanding of this embodiment, in combination with Figure 1 a detailed introduction is given to the intelligent fault diagnosis robot based on deep learning disclosed in the embodiments of the present invention.

[0065] Embodiment 1:

[0066] The intelligent fault diagnosis robot based on deep learning disclosed in the embodiments of the present invention includes:

[0067] A robot body, in which a control system is provided. The control system includes a total control module, a sensor module, a motion control module, a visual recognition module, and a power management module; the total control module is respectively connected to the sensor module, the motion control module, the visual recognition module, and the power management module.

[0068] In this embodiment, the robot adopted is a wheeled robot, and the structural schematic diagram of the robot is as shown in Figure 1 、 Figure 2 . Among them, the operating system of the robot is the Robot Operating System (ROS); the size of the robot is 15 cm * 15 cm * 20 cm, and the weight is 4 kg.

[0069] The total control module is provided with a first ESP8266 WIFI module, a second ESP8266 WIFI module, an STM32F401RCT6 single-chip microcomputer, and an L298N drive chip. The first ESP8266 WIFI module is respectively connected to an ultrasonic sensor and the STM32F401RCT6 single-chip microcomputer. The STM32F401RCT6 single-chip microcomputer is connected to an infrared sensor. The second ESP8266 WIFI module is connected to the L298N drive chip. The L298N drive chip is connected to the motion control module.

[0070] Among them, the L298N drive chip drives the robot to move through the motion control module.

[0071] The sensor module includes a camera sensor, an ultrasonic sensor, and an infrared sensor. The camera sensor is used to capture the actual images of the surrounding environment, including the actual images of the device. The ultrasonic sensor and the infrared sensor are used to sense the positions of obstacles within a set distance, including the positions of devices within a set distance.

[0072] In this embodiment, the set distance is 40 cm. When the ultrasonic sensor and / or the infrared sensor senses a device within 20 cm, the motion control module is triggered to stop working. The infrared sensor senses the temperature distribution of the surrounding environment.

[0073] The motion control module is provided with a drive motor. The motion control module is used to set the walking range of the robot and control the robot to reach the target position within the walking range through a path planning method based on the A* algorithm.

[0074] Preferably, the path planning method based on the A* algorithm includes:

[0075] Step 101: Establish an environmental map with grids, and initialize the open list and the closed list.

[0076] Step 102: Start searching for a path through a five-way search strategy based on coordinate differences. Select the node with the smallest total cost from the open list as the current node, and add the current node to the closed list.

[0077] In this embodiment, the current node searches in the environmental map using a five-way search strategy based on coordinate differences, including searching for a path centered on the current node, and using the horizontal direction, vertical direction, and diagonal direction of the current node as the search directions.

[0078] Exemplarily, if the coordinates of the current node are (x, y), then the coordinates of the nodes searched in the horizontal direction are (x - 1, y) and (x + 1, y), the coordinates of the nodes searched in the vertical direction are (x, y - 1) and (x, y + 1), and the coordinates of the nodes in the diagonal direction are (x + 1, y + 1) or (x + 1, y - 1) or (x - 1, y - 1) or (x - 1, y + 1).

[0079] Among them, compared with the traditional eight-neighborhood search strategy, the five-way search strategy based on coordinate differences reduces the number of search directions by filtering out diagonal searches; since the path touches fewer obstacles, it not only improves the safety of the path, but also significantly reduces the number of nodes to be considered during the search, thus improving the overall efficiency of the algorithm.

[0080] Step 103: Determine whether the current node is the target node. If so, the path search is successful, and the path is traced back through the parent node; if not, add the current node to the closed list and traverse all adjacent nodes of the current node.

[0081] Step 104: If the adjacent node is not in the open list and the closed list, add the adjacent node to the open list and record the parent node of the adjacent node as the current node;

[0082] If the adjacent node is in the open list and the new actual cost is smaller, update the total cost and the parent node of the adjacent node.

[0083] Step 105: End the path search when the current node is the target node or the open list is empty.

[0084] Furthermore, the open list uses a priority queue and is sorted according to the total cost of the nodes. Nodes with the smallest total cost are preferentially selected to store the nodes to be expanded, and the closed list directly stores the expanded nodes using a set; when initializing the open list and the closed list, the starting node is added to the open list, and at the same time, the closed list is made empty.

[0085] Among them, when storing a node in the priority queue, it only needs to insert the node into the corresponding position according to the total cost, and the time complexity is O(logn); when deleting a node from the priority queue, the node at the head of the queue can be directly deleted, and the time complexity is O(logn); when searching for a node in the priority queue, the entire queue needs to be traversed, and the time complexity is O(n);

[0086] When storing a node in the set, the node can be directly added to the set, and the time complexity is O(1); when deleting a node from the set, the node can be directly removed from the set, and the time complexity is O(1); when searching for a node in the set, it can be directly determined whether the node exists in the set, and the time complexity is O(1).

[0087] Furthermore, the total cost is represented by the following formula:

[0088] f(n) = g(n) + h(n)g(n)

[0089] Wherein, g(n) is the actual cost from the starting node to the current node, and h(n) is a heuristic function used to estimate the cost from the current node to the target node;

[0090] h(n) is represented by the following formula:

[0091] h(n) = (D × straight_step + D2 × diagonal_step) × p

[0092] Wherein, D is the cost of moving in the horizontal or vertical direction, with a value of 1, and D2 is the cost of moving in the diagonal direction, with a value of straight_step is the number of steps in the horizontal or vertical direction in the shortest path from the current node to the target node, and diagonal_step is the number of steps in the diagonal direction in the shortest path from the current node to the target node; p is a weight parameter used to dynamically adjust the weight of the heuristic function, and its value range is 0.8 to 1.5.

[0093] In this embodiment, when the robot faces a scenario such as having a narrow passage or many obstacles and needs to quickly avoid obstacles, the value of the weight parameter p can be appropriately increased, and the value range of p can be adjusted to 1.2 or 1.5, so that the weight of the heuristic function is greater, so that the A* algorithm selects a path with a smaller estimated cost, speeds up the search speed, and helps the robot quickly avoid obstacles; when the robot faces a scenario such as having multiple precision instruments and needs to accurately avoid obstacles, it is necessary to ensure that the found path is as close as possible to the shortest path, and the value of the weight parameter p can be appropriately reduced, and p can be adjusted to 0.8 or 0.9, so that the A* algorithm pays more attention to the actual cost, thereby improving the accuracy of the path.

[0094] Based on the above steps, the path planning method based on the A* algorithm disclosed in this embodiment can enable the robot to flexibly avoid obstacles in various complex situations. By introducing the weight parameter, the weight of the heuristic function can be dynamically adjusted, thereby controlling the search direction and efficiency of the algorithm, continuously optimizing the control strategy, enabling the robot to sense the changes in the surrounding environment in real time, quickly adjust the motion trajectory, avoid collisions with workers, equipment and other obstacles, and at the same time maintain a high operating speed and accurately reach the target position.

[0095] The visual recognition module includes an identification unit, a display, and a wireless communication unit. The identification unit is used to identify and diagnose the real image of the device through an improved YOLO model to obtain the fault type of the device; the display is used to display the fault type of the device and the location of the device; the wireless communication unit is used to send the fault type of the device and the location of the device to the control terminal.

[0096] In this embodiment, the control terminal includes a host computer software. The user can view the fault type of the device and the location of the device through the control terminal; the fault types of the device include: the device has a problem with a Personal Computer (PC), the device has a problem with a Programmable Logic Controller (PLC), and the device has a problem with a Variable Frequency Drive (VFD);

[0097] Among them, the PC problems that occur in the device include, but are not limited to, the display screen of the PC connected to the device showing a blue screen and not being able to light up normally;

[0098] The PLC problems that occur in the device include, but are not limited to, the PLC fault indicator light on the PLC control panel turning red, and the power indicator light and I / O indicator light on the PLC control panel not being able to light up normally;

[0099] The VFD problems that occur in the device include, but are not limited to, the display screen on the VFD control panel showing a fault code, going black, or flashing.

[0100] Since the YOLO model, as a deep learning-based object detection model, has a simple structure and a relatively fast detection speed, and is suitable for real-time scenarios, this application uses the YOLOV8n model to identify and diagnose the real image of the device. Compared with the traditional YOLO model, the YOLOV8n model has been significantly improved in multiple aspects, effectively improving the accuracy, speed, and adaptability of object detection.

[0101] Preferably, identifying and diagnosing the real image of the device through the improved YOLO model includes:

[0102] Step 201: Preprocess the sample image and divide the sample image into a first data set and a second data set.

[0103] Among them, the sample image includes images of devices with faults under different lighting conditions.

[0104] In this embodiment, the lighting conditions include: strong light, weak light, and backlight; the sample images have different sizes, and the size range of the sample images is 540mm * 540mm to 1080mm * 1080mm.

[0105] Step 202: Adjust the weight parameters of the convolutional kernels of the YOLOV8 model, combine the attention optimization mechanism, and adopt a knowledge distillation method based on multi-teacher model collaboration to optimize the YOLOV8n model through the first dataset and the YOLOV8 model.

[0106] Step 203: Train the YOLOV8n model through the second dataset, and test and validate the YOLOV8n model after training.

[0107] Among them, the second dataset includes a second training set, a second test set, and a second validation set.

[0108] Step 204: Use the tested and validated YOLOV8n model to identify and diagnose the real images of the device.

[0109] Using the tested and validated YOLOV8n model to identify and diagnose the real images of the device includes: using the tested and validated YOLOV8n model to identify and diagnose the real images of the device, and outputting the bounding box, confidence score, and fault category of the device.

[0110] Among them, the confidence score is F1, and the target categories include: PC problems, VFD problems, and PLC problems.

[0111] Furthermore, in step 201, the preprocessing of the sample image includes: processing the grayscale value of the sample image; normalizing the sample image using the normalization parameters.

[0112] In this embodiment, normalizing the sample image using the normalization parameters includes:

[0113] Step 301: Extract features from the sample image through the YOLOV8 model to obtain the feature representation of the sample image.

[0114] Step 302: By analyzing and processing the feature representation, calculate the mean and variance of each feature channel as the normalization parameters of the feature channel.

[0115] Step 303: Normalize each pixel value of the sample image through the normalization parameters of its corresponding feature channel to obtain the normalized sample image.

[0116] Combining the above steps, this embodiment preprocesses the sample image. By increasing the image grayscale value, the details and features in the sample image are made clearer; through the normalization of sample images under different scales and different lighting conditions, the detail information of the sample image is better retained, and the adaptability to lighting changes is stronger, thereby improving the accuracy and stability of image recognition.

[0117] In step 202, by using the gradient descent method to adjust the weight parameters of the convolutional kernels of the YOLOV8 model, the weights of the convolutional kernels can be iteratively adjusted to minimize the loss function.

[0118] Furthermore, in step 202, combined with the attention optimization mechanism, a knowledge distillation method based on the collaboration of multiple teacher models is adopted to optimize the YOLOV8n model through the first dataset and the YOLOV8 model, including:

[0119] Step 401: Set multiple YOLOV8 models as teacher models and the YOLOV8n model as the student model, and divide the first dataset into a first training set, a first test set, and a first validation set.

[0120] Step 402: Use the teacher models to extract features from the first training set, and input the extracted features as soft labels into the student model.

[0121] Step 403: Combine the cross-entropy loss function, the attention distillation loss function, and the feature distillation loss function to form the total loss function, and train the student model.

[0122] Step 404: Test and validate the student model through the first test set and the first validation set to optimize the performance of the student model.

[0123] Even further, the expression of the total loss function L total is as follows:

[0124] L total = LCE + λ1·LAD + λ2·LFD

[0125] In the formula, LCE is the cross-entropy loss function, LAD is the attention distillation loss function, and LFD is the feature distillation loss function; λ1 and λ2 are the weights of the attention distillation loss function and the feature distillation loss function respectively, which are used to balance the contributions of the cross-entropy loss function and the distillation loss function.

[0126] Specifically, the cross-entropy loss function is used to measure the difference between the predicted probability distribution and the true label. The expression of the cross-entropy loss function LCE is as follows:

[0127]

[0128] In the formula, C is the total number of target categories; y i is the one-hot encoding of the true label. If the model predicts that the sample belongs to the i-th category, then y i = 1, otherwise y i = 0; p i is the probability that the model predicts that the sample belongs to the i-th category.

[0129] The attention distillation loss function is used to transfer the attention map of the teacher model to the student model during the knowledge distillation process. The expression of the attention distillation loss function LAD is as follows:

[0130]

[0131] where N is the total number of feature maps; a i is the weight of the i-th feature map, used to balance the contributions of different feature maps; At i is the attention distribution of the teacher model on the i-th feature map; As i is the attention distribution of the student model on the i-th feature map; KL is the Kullback-Leibler divergence, used to measure the difference between two probability distributions.

[0132] The feature distillation loss function is used to learn the feature representation of the teacher model. The expression of the feature distillation loss function is as follows:

[0133]

[0134] where N is the total number of feature maps, F t i is the feature representation of the teacher model on the i-th feature map, is the feature representation of the student model on the i-th feature map; is the square of the Euclidean norm of the feature representations of the teacher model and the student model, used to measure the difference between feature maps.

[0135] According to the above expressions, combining the cross-entropy loss function and the distillation loss function based on the attention mechanism forms the total loss function. By minimizing the total loss function, the student model can inherit the performance of the teacher model as much as possible while maintaining a small model size.

[0136] Based on this, this embodiment adopts a knowledge distillation method based on the collaboration of multiple teacher models, transfers the knowledge of multiple YOLOV8 models to the YOLOV8n model, and uses the integrated knowledge of multiple teacher models to guide the learning of the student model, which is beneficial to significantly improving the generalization ability and robustness of the student model; while improving the accuracy of the YOLOV8n model in recognizing images, compresses and optimizes the YOLOV8n model, making the optimized YOLOV8n model size only 500 kB, which can be deployed on common single-chip microcontrollers without relying on high-performance computing devices such as graphics cards, greatly reducing energy consumption and improving battery life.

[0137] The power management module is provided with a monitor, a power supply unit, and a control and management unit. The monitor is used to monitor the power and working status of the robot in real time; the power supply unit is provided with a solar cell and a wireless charging battery for powering the robot; the control and management unit is used to manage the power supply of the power supply unit.

[0138] In this embodiment, the power supply unit supplies power to the drive motor of the motion control module, the single-chip microcomputer unit in the total control module, and the vision recognition module respectively;

[0139] Among them, the output voltage of the drive motor is 9V, and the maximum output current is 1A; the output voltage of the single-chip microcomputer unit is 3.3V, and the maximum output current is 0.5A; the output voltage of the vision recognition module is 5V, and the maximum output current is 2A.

[0140] Preferably, the monitor includes a power monitor and a working status monitor. The power monitor is used to monitor the power of the robot, and the working status monitor is used to monitor the working status of the robot;

[0141] The solar cell is provided with a solar panel to directly power the robot; the wireless charging battery can inductively couple with a charging device located in the wireless charging area to power the robot;

[0142] Preferably, the control and management unit manages the power supply of the power supply unit, including: the control and management unit adjusts the charging speed and / or charging power of the power supply unit to the robot according to the power and working status of the robot.

[0143] In this embodiment, the working status includes a high power consumption state, a standby state, and a night state.

[0144] Specifically, adjusting the charging speed and / or charging power of the power supply unit to the robot according to the power and working status of the robot includes:

[0145] When the robot is in a high power consumption state, the robot has a large working load and fast power consumption. At this time, the power supply unit will automatically increase the charging speed to ensure that the robot can replenish energy in time;

[0146] When the robot is in a standby state or has sufficient power, the power supply unit will appropriately reduce the charging power to extend the service life;

[0147] When the robot is in a night state, the power supply unit automatically charges the robot. When what conditions are met, the charging stops.

[0148] Combined with the above description, the power management module disclosed in this embodiment can effectively improve the charging efficiency of the robot through the charging management of the power supply unit, ensure that the robot can maintain a sufficient power reserve in various working environments, and meet the working requirements of the next day.

[0149] The working principle of the intelligent fault diagnosis robot based on deep learning provided by the embodiments of the present invention for device identification and diagnosis specifically includes:

[0150] The robot senses the position of obstacles in the surrounding environment through the ultrasonic sensor in the sensor module, and controls the robot body to move towards the target position through the motion control module; after reaching the target position, it takes pictures of the display connected to the device, the control panel of the PLC, and the control panel of the VFD through the camera sensor in the sensor module to obtain the actual image of the device, and uses the visual recognition module to recognize the actual image and diagnose the fault type of the faulty device. The recognition effect is as Figure 4 、 Figure 5 shown.

[0151] In the above process, the robot is charged through the power management module to ensure sufficient power during operation.

[0152] Figures 6 to 12 It is a schematic diagram for testing the YOLOV8n model applied in the visual recognition module of the intelligent fault diagnosis robot provided in this embodiment.

[0153] Figure 6 It is a schematic diagram of the confusion matrix of the YOLOV8n model after training but before normalization. As Figure 6 can be seen, the model shows relatively good performance in identifying PLC problems and VFD problems.

[0154] Figure 7 It is a schematic diagram of the confusion matrix of the YOLOV8n model after normalization. Combining Figure 5 , when the model classifies PC problems, VFD problems, and PLC problems, its accuracy reaches 100%, and there is no misclassification at all. The prediction results show a high degree of consistency with the actual results.

[0155] Figure 8 It is a schematic diagram of the F1-Confidence curve for showing the performance of the YOLOV8n model at different confidence levels. Among them, F1-Confidence represents the prediction confidence of the model for its F1 score. Combining Figure 8 , the F1 scores of PC problems, VFD problems, and PLC problems are close to 1, indicating that the classification effect of the model is excellent; and the comprehensive F1 score of all classes reaches 0.98 at a confidence level of 0.819, indicating that the overall performance of the model is excellent, with high accuracy and recall rate, and strong robustness.

[0156] Figure 9Schematic diagram of the precision-confidence curve for the YOLOV8n model. Combining Figure 9 with it, the model performs well at different confidence levels, especially in the high-confidence region where the precision is close to 1.0. Among them, all classes achieve perfect precision at a confidence level of 0.970, indicating the high accuracy and robustness of the model. This helps to select the optimal confidence threshold to optimize the model performance.

[0157] Figure 10 Schematic diagram of the precision-recall curve for the YOLOV8n model. Figure 10 It shows the high performance of the model on each class. The average precision and the mean average precision of all classes reach 0.995, that is, the model can maintain a high precision while maintaining a high recall rate.

[0158] Figure 11 Schematic diagram of the recall-confidence curve for the YOLOV8n model. Figure 11 It shows that the recall rate reaches 1.00 at low confidence levels, indicating that all positive examples can be recognized. As the confidence level increases, the recall rate gradually decreases, reflecting the trade-off between precision and recall of the model; and the performance of each class is consistent, indicating that the model has strong robustness.

[0159] Figures 12 to 14 Schematic diagrams of the training loss function, validation loss function and performance metrics curves for the YOLOV8n model respectively. Among them, as the number of training epochs increases continuously, both the training loss function curve and the validation loss function curve show an obvious downward trend. Thus, it can be seen that during the optimization process of the model, the ability to process and fit data is gradually enhanced; at the same time, the performance metrics curves, including the precision curve, recall curve and mean average precision curve, also gradually show an upward trend, fully indicating that the detection performance and learning ability of the model are gradually improving.

[0160] It should be noted that in this article, the terms "including", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0161] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any technician familiar with the technical field of the present invention can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or perform equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An intelligent fault diagnosis robot based on deep learning, characterized in that Including: A robot body, in which a control system is provided. The control system includes a total control module, a sensor module, a motion control module, a vision recognition module, and a power management module. The total control module is respectively connected to the sensor module, the motion control module, the vision recognition module, and the power management module; The total control module is provided with a first ESP8266 WIFI module, a second ESP8266 WIFI module, an STM32F401RCT6 single-chip microcomputer, and an L298N drive chip. The first ESP8266 WIFI module is respectively connected to an ultrasonic sensor and the STM32F401RCT6 single-chip microcomputer. The STM32F401RCT6 single-chip microcomputer is connected to the infrared sensor. The second ESP8266 WIFI module is connected to the L298N drive chip. The L298N drive chip is connected to the motion control module; The sensor module includes a camera sensor, an ultrasonic sensor, and an infrared sensor. The camera sensor is used to capture the actual images of the surrounding environment, including the actual images of the devices. The ultrasonic sensor and the infrared sensor are used to sense the positions of the objects within a set distance, including the positions of the devices within a set distance; The motion control module is provided with a drive motor. The motion control module is used to set the walking range of the robot and control the robot to reach the target position within the walking range through a path planning method based on the A* algorithm; The vision recognition module includes an identification unit, a display, and a wireless communication unit. The identification unit is used to identify and diagnose the real images of the devices through an improved YOLO model to obtain the fault types of the devices. The display is used to display the fault types of the devices and the positions of the devices. The wireless communication unit is used to send the fault types of the devices and the positions of the devices to the control terminal; The power management module is provided with a monitor, a power supply unit, and a control management unit. The monitor is used to monitor the power and working status of the robot in real time. The power supply unit is provided with a solar cell and a wireless charging battery for supplying power to the robot. The control management unit is used to manage the power supply of the power supply unit; 2. The intelligent fault diagnosis robot based on deep learning according to claim 1, wherein The path planning method based on the A* algorithm includes: Establish an environmental map with grids and initialize the open list and the closed list; Start searching for a path through a five-way search strategy based on the coordinate difference. Select the node with the minimum total cost from the open list as the current node and add the current node to the closed list; Judge whether the current node is the target node. If so, the path search is successful and the path is traced back through the parent node. If not, add the current node to the closed list and traverse all adjacent nodes of the current node; If the adjacent node is not in the open list and the closed list, add the adjacent node to the open list and record the parent node of the adjacent node as the current node; If the adjacent node is in the open list and the new actual cost is smaller, update the total cost and the parent node of the adjacent node; If the current node is the target node or the open list is empty, end the search path.

3. The intelligent fault diagnosis robot based on deep learning according to claim 2, wherein The open list uses a priority queue and is sorted according to the total cost of the nodes. The node with the smallest total cost is preferentially selected to store the nodes to be expanded. The closed list directly stores the expanded nodes using a set. When initializing the open list and the closed list, the starting node is added to the open list, and at the same time, the closed list is made empty.

4. The intelligent fault diagnosis robot based on deep learning according to claim 2, characterized in that, The total cost is represented by the following formula: f(n) = g(n) + h(n) g(n) Where g(n) is the actual cost from the starting node to the current node, and h(n) is a heuristic function used to estimate the cost from the current node to the target node; h(n) is represented by the following formula: h(n) = (D × straight_step + D2 × diagonal_step) × p Where D is the cost of moving in the horizontal or vertical direction, with a value of 1, and D2 is the cost of moving in the diagonal direction, with a value of straight_step is the number of steps in the horizontal or vertical direction in the shortest path from the current node to the target node, and diagonal_step is the number of steps in the diagonal direction in the shortest path from the current node to the target node; p is a weight parameter used to dynamically adjust the weight of the heuristic function, and its value range is 0.8 to 1.

5.

5. The intelligent fault diagnosis robot based on deep learning according to claim 1, characterized in that, Identify and diagnose the real image of the device through an improved YOLO model, including: Preprocess the sample image and divide the sample image into a first data set and a second data set; Adjust the weight parameters of the convolutional kernels of the YOLOV8 model, combine the attention optimization mechanism, and adopt a knowledge distillation method based on multi-teacher model collaboration to optimize the YOLOV8n model through the first data set and the YOLOV8 model; Train the YOLOV8n model through the second data set, and test and verify the YOLOV8n model after training. Use the YOLOV8n model after testing and verification to identify and diagnose the real image of the device.

6. The intelligent fault diagnosis robot based on deep learning according to claim 5, wherein The preprocessing of the sample image includes: processing the gray value of the sample image; normalizing the sample image using normalization parameters.

7. The intelligent fault diagnosis robot based on deep learning according to claim 5, characterized in that Combining the attention optimization mechanism, adopting a knowledge distillation method based on multi-teacher model collaboration to optimize the YOLOV8n model through the first data set and the YOLOV8 model, including: Set multiple YOLOV8 models as teacher models and the YOLOV8n model as the student model, and divide the first data set into a first training set, a first test set, and a first validation set. Use the teacher model to extract features from the first training set and input the extracted features as soft labels into the student model. Combine the cross-entropy loss function, the attention distillation loss function, and the feature distillation loss function to form a total loss function and train the student model. Test and verify the student model through the first test set and the first validation set to optimize the performance of the student model.

8. The intelligent fault diagnosis robot based on deep learning according to claim 7, characterized in that The total loss function L total has the following expression: L total = LCE + λ1·LAD + λ2·LFD Wherein, LCE is the cross-entropy loss function, LAD is the attention distillation loss function, and LFD is the feature distillation loss function; λ1 and λ2 are the weights of the attention distillation loss function and the feature distillation loss function respectively, which are used to balance the contributions of the cross-entropy loss function and the distillation loss function.

9. The intelligent fault diagnosis robot based on deep learning according to claim 1, characterized in that The monitor includes a power monitor and a working state monitor. The power monitor is used to monitor the power of the robot, and the working state monitor is used to monitor the working state of the robot. The solar cell is provided with a solar panel, which directly powers the robot; the wireless charging battery can be inductively coupled with a charging device located in the wireless charging area, thereby powering the robot.

10. The intelligent fault diagnosis robot based on deep learning according to claim 1, wherein The control and management unit performs power supply management on the power supply unit, including: the control and management unit adjusts the charging speed and / or charging power of the power supply unit to the robot according to the power and working state of the robot.

Images