Industrial robot remote monitoring system based on AI vision

Abstract

The invention discloses an industrial robot remote monitoring system based on AI vision, and belongs to the technical field of industrial robot monitoring. The system comprises an image acquisition module, an image processing module, an AI visual analysis module, a fault judgment module and a remote monitoring platform. The image acquisition module acquires an original operation image set; the image processing module performs noise removal, distortion correction and ROI extraction on the original operation image set to obtain a robot operation image set; the AI visual analysis module extracts a robot operation track based on the robot operation image set, calculates the fitting degree of the robot operation track and a robot operation standard track, and judges whether the fault judgment module is triggered or not; the fault judgment module carries out action sequence analysis and fault analysis on the operation track of the robot to obtain a fault diagnosis result; and the remote monitoring platform is used for performing action logic correction or mechanical fault detection determination by a user. According to the invention, precise monitoring and fault diagnosis of the operation of the industrial robot are realized, and the monitoring efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial robot monitoring, and specifically relates to an industrial robot remote monitoring system based on AI vision. Background Art

[0002] With the development of intelligent manufacturing technology, industrial robots have become widely used in automated production scenarios. Their operating accuracy and stability directly impact product quality and production efficiency. Therefore, real-time monitoring, fault diagnosis, and remote management of industrial robots' operating processes have become critical to ensuring the continuous operation of production lines.

[0003] Currently, industrial robot monitoring primarily relies on manual inspections, sensor data collection, and traditional visual monitoring, all of which have significant limitations. Manual inspections rely on experience, have poor real-time performance, and struggle to detect subtle deviations in high-speed motion, leading to missed detections and misjudgments. This makes them particularly unsuitable for multi-robot collaboration or hazardous environments. Sensor data collection is susceptible to mechanical interference and only reflects the robot's own perception state, failing to correlate actual trajectory with environmental details and accurately locating the source of deviations. While traditional visual monitoring uses camera-generated images to extract trajectories, image preprocessing fails to effectively address noise and distortion, resulting in insufficient accuracy. Fit calculations rely on single-dimensional deviations, making it difficult to comprehensively assess motion consistency. Furthermore, fault diagnosis lacks systematic logic, making it difficult to effectively distinguish between logical errors in motion sequences and mechanical failures, requiring on-site disassembly and analysis by professionals, which is time-consuming and labor-intensive. Furthermore, existing remote monitoring systems primarily focus on data display, lacking the ability to intelligently analyze motion sequences and accurately locate mechanical faults. This makes remote closed-loop correction difficult, resulting in long fault response cycles and impacting production continuity.

[0004] Therefore, in response to the problems of poor real-time monitoring, insufficient trajectory analysis accuracy, low fault diagnosis efficiency and weak remote management capabilities in existing technologies, there is an urgent need for an industrial robot remote monitoring system based on AI vision to achieve high-precision extraction of industrial robot operation trajectories, rapid fault differentiation and remote and efficient processing, thereby improving the automation and intelligence level of monitoring. Summary of the Invention

[0005] The purpose of the present invention is to provide an industrial robot remote monitoring system based on AI vision to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] An AI vision-based industrial robot remote monitoring system, including an image acquisition module, an image processing module, an AI vision analysis module, a fault judgment module and a remote monitoring platform;

[0008] The image acquisition module is used to obtain a set of original operation images of a complete operation task of the industrial robot, wherein the set of original operation images is composed of a plurality of continuous frames of original operation images with a fixed time interval;

[0009] The image processing module is used to perform image preprocessing on each original operation image in the original operation image set to obtain a robot operation image set;

[0010] The AI ​​visual analysis module is used to extract the robot operation trajectory from the robot operation image set through AI visual technology, and calculate the fit of the robot operation trajectory based on the robot standard operation trajectory to determine whether to trigger the fault judgment module;

[0011] The fault judgment module is used to perform action sequence analysis and fault analysis on the robot's operation trajectory to obtain a fault diagnosis result;

[0012] The remote monitoring platform is used for users to modify action logic or detect and determine mechanical faults based on fault diagnosis results.

[0013] Preferably, the image acquisition module obtains a set of original operation images of a complete operation task of the industrial robot through a global industrial shutter camera, wherein the global industrial shutter camera is selected because it can realize synchronous exposure of the entire image, effectively avoiding the problem of dynamic image blur caused by the rolling shutter when the industrial robot moves at high speed, and ensuring that the details of the key parts of the robot in the original operation image are clearly distinguishable; the complete operation task covers the entire process of the industrial robot from the start of the operation task to the completion of the operation task, including all preset action steps; the original operation image set is composed of several frames of continuous original operation images with a fixed time interval, wherein the time interval can be set based on the operation speed of the robot.

[0014] Preferably, the process of performing image preprocessing on each original job image in the original job image set is:

[0015] The noise of each original image in the original image set is removed by a block matching 3D filtering algorithm. The block matching 3D filtering algorithm searches for similar blocks in the original image and performs three-dimensional transform domain filtering. It can effectively suppress the salt and pepper noise and Gaussian noise in the industrial environment while retaining the detailed features of the key parts of the industrial robot to the greatest extent. At the same time, the camera intrinsic parameters of the global industrial shutter camera, including focal length, principal point coordinates, radial distortion coefficient and tangential distortion coefficient, are obtained through the Zhang Zhengyou calibration method. Based on the camera intrinsic parameters, each original image after noise removal is subjected to distortion correction processing. The radial distortion correction is used to eliminate the image edge offset caused by the non-ideal spherical characteristics of the lens, and the tangential distortion correction is used to correct the pixel offset caused by the tilt of the lens and the imaging plane. This ensures that the pixel coordinates of the robot in each original image after noise removal can be accurately mapped to the physical world coordinates.

[0016] Based on this, each original operation image after distortion correction is subjected to ROI extraction through the semantic segmentation model U-Net. U-Net performs pixel-level classification on each original operation image after distortion correction through the encoding-decoding structure, segments the relevant areas of the industrial robot operation, and eliminates invalid pixel interference from the background environment. Finally, a robot operation image set consisting of several consecutive frames of image-preprocessed original operation images with fixed time intervals is obtained.

[0017] Preferably, the robot operation trajectory is composed of a number of trajectory points containing timestamps, 3D coordinates and attitude angles at continuous and fixed time intervals, wherein the continuous and fixed time intervals mean that the trajectory points are generated sequentially according to a uniform time step to ensure that the trajectory points can fully reflect the continuous motion process of the industrial robot from the start to the end of the operation task, and the timestamp contained in each trajectory point provides a reference for time alignment with the robot operation trajectory; 3D coordinates refer to the X, Y and Z axis spatial position data of the industrial robot end effector or key operating part in the robot base coordinate system, which is used to quantify position deviation; attitude angles include the Roll angle around the X axis, the Pitch angle around the Y axis and the Yaw angle around the Z axis, which are used to describe the spatial rotation state of the industrial robot operating part;

[0018] The robot's standard operating trajectory is composed of several standard trajectory points with continuous and fixed time intervals, including timestamps, 3D standard coordinates and standard attitude angles, which serve as the benchmark for judging the actual robot's operating trajectory. The standard trajectory points are ideal motion trajectories preset based on the industrial robot's operating tasks, and their time intervals are consistent with the robot's operating trajectory to ensure that the two can correspond one-to-one in the time dimension; the 3D standard coordinates and standard attitude angles are the ideal spatial position and attitude that the robot should achieve at each timestamp, respectively.

[0019] Preferably, the process of calculating the degree of fit of the robot's operating trajectory and determining whether to trigger a fault judgment module is:

[0020] Based on the timestamp of the trajectory point, each trajectory point in the robot's operating trajectory is associated with the standard trajectory point in the robot's standard operating trajectory in a one-to-one correspondence. Since both are composed of trajectory points at continuous and fixed time intervals, the timestamp can ensure that the trajectory point at the same moment is accurately matched with the standard trajectory point, forming several trajectory point comparison pairs. For each trajectory point comparison pair, the position deviation formula, attitude deviation formula and speed deviation formula are used to calculate the position deviation of each trajectory point comparison pair. , posture deviation and speed deviation , based on this, the number of trajectory point pairs with a speed deviation of 0 and the maximum position deviation are counted to obtain the number of trajectory point pairs with zero speed deviation and the maximum position deviation. At the same time, the mean of the posture deviations corresponding to all trajectory point pairs is calculated to obtain the average posture deviation. The number of trajectory point pairs with zero speed deviation, the maximum position deviation and the global average posture deviation are then brought into the weighted fitting formula to calculate the fitting degree;

[0021] The position deviation formula is:

[0022] ;

[0023] in, 、 and is the 3D coordinate of the trajectory point at timestamp t, 、 and is the 3D standard coordinate of the standard trajectory point at time stamp t;

[0024] The posture deviation formula is:

[0025] ;

[0026] in, 、 and is the attitude angle of the trajectory point at timestamp t, 、 and is the 3D standard coordinate of the standard trajectory point at time stamp t;

[0027] The speed deviation formula is:

[0028] ;

[0029] in, is a fixed time interval;

[0030] The weighted fitting formula is:

[0031] ;

[0032] in, 、 and is the weight, and ;

[0033] If the degree of fit is greater than the preset degree of fit threshold, the deviation between the robot's operating trajectory and the robot's standard operating trajectory exceeds the acceptable range, indicating a possible fault, triggering the fault judgment module for in-depth diagnosis.

[0034] If the degree of fit is less than or equal to the preset degree of fit threshold, the deviation between the robot's operating trajectory and the robot's standard operating trajectory is within an acceptable range. There is no need to trigger fault diagnosis. The image acquisition module continues to acquire a new set of original operating images to achieve continuous monitoring.

[0035] Preferably, the method for performing motion sequence analysis on the robot's operating trajectory:

[0036] S1. The robot's trajectory is processed through a temporal segmentation model based on a long short-term memory (LSTM) or a temporal convolutional network (TCN). This model automatically identifies the boundary points of the robot's movement start and end points and uses these boundary points as segmentation markers to divide the continuous robot trajectory into several continuous segments. Each segment corresponds to a complete action unit of the industrial robot.

[0037] S2. Count the 3D coordinate range and attitude angle range of the trajectory points in each trajectory segment, reflecting the spatial range of the action unit and the rotation range of the actuator. Calculate the speed of each trajectory point in each trajectory segment by the 3D coordinate difference of the trajectory points and the fixed time interval, and count the speed range of the trajectory points in the trajectory segment. Based on this, obtain the motion characteristics of each trajectory segment, which include the 3D coordinate range, attitude angle range, and speed range.

[0038] S3. Pre-collect standard trajectories for all motion types of the industrial robot. Extract the 3D coordinate range, attitude angle range, and velocity range of each standard trajectory using the method in S2 to form a structured motion feature library. Each motion type in the motion feature library is associated with a unique motion feature. Use the cosine similarity algorithm to match the motion feature of each trajectory segment with the motion features of all motion types in the motion feature library to determine the motion type corresponding to each trajectory segment. Arrange the identified motion types in chronological order of the trajectory segments to form a motion sequence for the robot's operating trajectory.

[0039] S4. A standard operation task action sequence is preset based on the operation process of the industrial robot, which includes fixed action steps, sequence and dependency relationships. The edit distance algorithm is used to compare and analyze the action sequence of the robot operation trajectory with the standard operation task action sequence to obtain an action sequence analysis result with a logical error in the action sequence or a logically complete action sequence. The logical error in the action sequence is one or more of the following: missing action steps, reversed action sequence and presence of redundant actions. Missing action steps means that the action sequence lacks necessary action steps in the standard operation task action sequence. Reversed action sequence means that there is a sequence of action steps in the action sequence that violates the dependency relationship. The presence of redundant actions means that the action sequence contains meaningless redundant action steps in the standard operation task action sequence.

[0040] Preferably, the process of performing fault analysis on the robot's operating trajectory:

[0041] Based on the timestamp, each track point in the robot's operating trajectory is associated with each standard track point in the robot's standard operating trajectory to obtain several track point comparison pairs. For each track point comparison pair, the axis position deviation of each track point comparison pair is calculated using the axis position deviation formula. The axis position deviation includes the X-axis position deviation. , Y-axis position deviation and Z-axis position deviation The significance of calculating the position deviation of each axis separately is that the mechanical failure of the industrial robot can be manifested as a single axis offset, and the failure needs to be located by split-axis analysis; the mean and standard deviation of the position deviation of each axis of all trajectory points are calculated to obtain the average position deviation of each axis. and And the standard deviation of the position deviation of each axis and The average deviation of each axis position is used to reflect the overall offset direction and size of each axis position. The standard deviation of each axis position deviation is used to reflect the degree of deviation dispersion of each axis position. At the same time, the proportion of trajectory points with positive position deviations of each axis is counted to obtain the proportion of consistent signs of the position deviations of each axis, which is used to reflect the fixity of the offset direction.

[0042] The shaft position deviation formula is:

[0043] ;

[0044] ;

[0045] ;

[0046] If the average deviation of each axis position is not 0, the standard deviation of each axis position deviation is less than the preset position deviation standard deviation threshold, and the proportion of consistent signs of each axis position deviation is greater than 80%, there is a systematic deviation, which can represent that the overall robot operation trajectory is stably deviated in a certain fixed direction, which is usually caused by systematic errors of mechanical structure;

[0047] If the standard deviation of each axis position deviation is greater than the preset position deviation standard deviation threshold, and the proportion of consistent signs of each axis position deviation is less than 60%, there is a random deviation, which can represent that the robot operation trajectory randomly fluctuates in the positive and negative directions without a fixed deviation trend, which is usually caused by sensor noise or unstable driving system;

[0048] If the deviation of each axis position of a certain trajectory point pair is greater than 4 times the average deviation of each axis position, there is a sudden deviation, which can represent that the robot operation trajectory suddenly deviates greatly at a certain moment, which is usually caused by instantaneous mechanical abnormalities;

[0049] If the deviation of each axis position of at least 5 consecutive adjacent trajectory point pairs is greater than 3 times the average deviation of each axis position, there is a local deviation, which can represent that there is a continuous large deviation in a certain trajectory segment of the robot operation trajectory, and other trajectory segments are normal, which is usually related to the failure of the actuator corresponding to the specific trajectory segment;

[0050] Accordingly, the deviation feature analysis result is obtained, and the deviation feature analysis result is mapped to obtain a preliminary mechanical fault type through a fault type matching rule library, the deviation feature analysis result is one or more of systematic deviation, random deviation, sudden deviation and local deviation, and the fault type matching rule library is used to store the preliminary mechanical fault type corresponding to the deviation feature analysis result.

[0051] Preferably, the precondition for the fault analysis of the robot operation trajectory is that the action sequence analysis result is an action sequence logic integrity, and if the action sequence analysis result is an action sequence logic error, the fault diagnosis result is an action sequence logic error, otherwise, the fault diagnosis result is a preliminary mechanical fault type.

[0052] Preferably, the remote monitoring platform comprises a visual display sub-module, an analysis data input module, an action logic correction sub-module and a mechanical fault detection sub-module;

[0053] The visualization display submodule is used to display the robot operation trajectory, the robot standard operation trajectory and the fault diagnosis results through a 3D visualization interface, and at the same time provides an interactive button for correcting the action logic and an interactive button for determining the mechanical fault target. The interactive button for correcting the action logic is used to trigger the action logic correction submodule, and the interactive button for determining the mechanical fault target is used to receive the mechanical fault target input by the user according to the preliminary mechanical fault type and synchronously trigger the mechanical fault detection submodule;

[0054] The action logic correction submodule is used to re-edit the action logic parameters through a visual flowchart editor and send the action logic parameters to the controller of the industrial robot through the OPCUA protocol;

[0055] The mechanical fault detection submodule is used to automatically control the global industrial shutter camera to capture an image of the mechanical fault target based on the mechanical fault target, and identify the cause of the mechanical fault using a pre-trained defect detection model. The pre-trained defect detection model is a classification model based on EfficientNet.

[0056] The analysis data input module is used to add, delete, modify and check the preliminary mechanical fault types corresponding to the deviation feature analysis results in the fault type matching rule library, the robot standard operation trajectory and the action features corresponding to various action types in the action feature library.

[0057] Due to the adoption of the above technical solution, the present invention has the following technical advancements compared to the prior art:

[0058] 1. The present invention realizes the automated and precise identification of logical errors in the robot's motion sequence, breaking through the limitation of traditional reliance on manual visual trajectory comparison. Traditional methods rely on human experience to judge the step integrity and sequence compliance of robot operation actions, which is prone to omissions and misjudgments due to fatigue or experience differences. The present invention automatically divides trajectory segments through an AI time series segmentation model, combines cosine similarity matching with the motion feature library to realize motion type recognition, and then compares with the standard motion sequence through a sequence alignment algorithm. It can objectively and quantitatively identify logical errors such as missing steps and reversed sequence, improve recognition accuracy, and significantly reduce the cost of manual intervention. It is suitable for flexible production scenarios with multiple varieties and high frequency.

[0059] 2. The present invention solves the problems of low efficiency and high cost of traditional on-site troubleshooting by constructing a remote characterization and positioning mechanism for mechanical faults. Traditional mechanical fault diagnosis requires technicians to go to the site to disassemble and inspect, making it difficult to quickly locate the source of the fault. The present invention automatically identifies four types of offset patterns: systematic, random, sudden, and local, by extracting the statistical features of the average deviation, standard deviation, and sign-consistent ratio of each axis position deviation. Combined with a preset rule base, it maps these to preliminary mechanical fault types, enabling remote derivation of trajectory features to fault types, shortening the fault location cycle from hours to minutes, and reducing downtime losses.

[0060] 3. The present invention realizes remote closed-loop management from diagnosis to correction to detection, improving the intelligent level of industrial robot operation and maintenance. In traditional systems, action logic correction and mechanical fault detection are mostly independent links and lack a collaborative mechanism. The remote monitoring platform of the present invention integrates visualization display, action logic correction and mechanical fault detection sub-modules, supports drag-and-drop editing and encrypted distribution of logic parameters, as well as directional image acquisition and AI recognition of mechanical fault targets, realizing full process automation from fault diagnosis to remote processing, reducing on-site debugging requirements, and adapting to the development needs of remote operation and maintenance of Industry 4.0. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0062] Figure 1 It is a schematic diagram of the system function modules of the present invention. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0064] Examples, such as Figure 1 The invention discloses an AI vision-based industrial robot remote monitoring system, which includes an image acquisition module, an image processing module, an AI vision analysis module, a fault judgment module and a remote monitoring platform, which work together to complete the remote monitoring of the industrial robot.

[0065] The image acquisition module is used to obtain a set of original operation images of a complete operation task of the industrial robot, wherein the set of original operation images is composed of a plurality of continuous frames of original operation images with a fixed time interval;

[0066] The image processing module is used to perform image preprocessing on each original operation image in the original operation image set to obtain a robot operation image set;

[0067] The AI ​​visual analysis module is used to extract the robot operation trajectory from the robot operation image set through AI visual technology, and calculate the fit of the robot operation trajectory based on the robot standard operation trajectory to determine whether to trigger the fault judgment module;

[0068] The fault judgment module is used to perform action sequence analysis and fault analysis on the robot's operation trajectory to obtain a fault diagnosis result;

[0069] The remote monitoring platform is used for users to modify action logic or detect and determine mechanical faults based on fault diagnosis results.

[0070] Furthermore, the working principle of the present invention is described below by way of examples:

[0071] This embodiment deploys an AI vision-based industrial robot remote monitoring system for the 6-axis industrial robot in the automatic assembly production line of mobile phone shells. The system deploys two global shutter industrial cameras above the industrial robot's operating area, capturing the original image of the operation at an interval of 20ms, covering the complete field of view of the robot's end effector (gripper) and the assembly station. At the same time, the image processing module and AI vision analysis module are deployed on the edge computing node, integrating the trajectory extraction model and the fitting calculation unit; the fault judgment module is connected to the edge node via industrial Ethernet, and has a built-in action sequence analysis engine and a mechanical fault feature recognition library; the remote monitoring platform is deployed on the factory cloud server, realizing visual monitoring and remote operation through a web interface, and supporting OPCUA communication with the robot controller PLC.

[0072] When the robot executes the standard complete operation task process of "material picking → alignment → assembly → exit", the image acquisition module continuously captures the original operation images at 20ms intervals, forming an operation original image set containing 500 frames of images, corresponding to a complete operation task cycle of 10 seconds. The image processing module preprocesses each frame of the operation original image, removes salt and pepper noise through the BM3D algorithm, corrects lens distortion based on Zhang Zhengyou calibration method, and crops the ROI area centered on the gripper, and outputs the robot operation image set.

[0073] The AI ​​visual analysis module uses the DeepSORT algorithm to track feature points in the robot's work image set, identify the 3D coordinates and attitude angles of the gripper end, and generate a robot work trajectory consisting of 500 track points, each with a timestamp, 3D coordinates, and attitude angles. The module then calls the standard robot work trajectory preset by offline programming and calculates the fit of the robot work trajectory to be 0.75. Since this fit is less than the preset fit threshold of 0.8, the fault diagnosis module is triggered. The fault diagnosis module uses an LSTM-based temporal segmentation model to segment the robot's work trajectory and identify four segments: "material collection → alignment → assembly → exit." These segments are consistent with the standard action sequence. The module extracts the action features of each segment, such as the 3D coordinate range, attitude angle range, and speed range of the alignment segment. The module then uses the cosine similarity algorithm to match the segment with the action feature library to determine the action type of each segment. The identified action types are arranged in chronological order to form an action sequence for the robot's work trajectory. This action sequence is then compared with the action sequence of the standard task task to ensure the logical integrity of the action sequence.

[0074] Since the result of the motion sequence analysis shows that the motion sequence is logically complete, a mechanical fault analysis is performed on the robot's operating trajectory. For example, the calculated average deviation of the X-axis position is +0.6mm, the standard deviation of the X-axis position deviation is 0.1mm, and the proportion of consistent X-axis position deviation signs is 90%. Assuming that there are no other types of deviations, it is determined that there is a systematic offset of the X-axis. The fault type matching rule library is matched, and the output fault diagnosis result is suspected wear of the X-axis joint reducer.

[0075] The remote monitoring platform receives the fault diagnosis results and displays them in a 3D visualization interface. Engineers then use the "Mechanical Fault Target Interaction" button to specify the X-axis joint as the mechanical fault target. This triggers the global industrial shutter camera to capture an image of the X-axis joint. The EfficientNet-based classification model identifies the X-axis joint image and determines that the cause of the mechanical failure is annular wear marks on the joint surface.

[0076] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An industrial robot remote monitoring system based on AI vision, characterized in that: include: An image acquisition module is used to obtain a set of original operation images of a complete operation task of the industrial robot, wherein the set of original operation images consists of a plurality of continuous frames with fixed time intervals; An image processing module is used to perform image preprocessing on each original image in the original image set to obtain a robot operation image set; The AI ​​visual analysis module is used to extract the robot's operating trajectory from the robot's operating image set using AI visual technology, and calculate the fit of the robot's operating trajectory based on the robot's standard operating trajectory to determine whether to trigger the fault judgment module; The fault judgment module is used to perform action sequence analysis and fault analysis on the robot's operation trajectory to obtain fault diagnosis results; The remote monitoring platform is used for users to modify action logic or detect and determine mechanical faults based on fault diagnosis results.

2. The AI ​​vision-based industrial robot remote monitoring system according to claim 1, characterized in that: The image acquisition module obtains a set of original operation images of a complete operation task of the industrial robot through a global industrial shutter camera.

3. The AI ​​vision-based industrial robot remote monitoring system according to claim 2, characterized in that: The process of performing image preprocessing on each job original image in the job original image set: Each original image in the original image set is subjected to noise removal using a block matching 3D filtering algorithm. At the same time, the camera intrinsic parameters of the global industrial shutter camera are obtained using the Zhang Zhengyou calibration method. Based on the camera intrinsic parameters, each original image after noise removal is subjected to distortion correction processing including radial distortion correction and tangential distortion correction. Each original image after distortion correction is subjected to ROI extraction using the semantic segmentation model U-Net, resulting in a robot operation image set consisting of several consecutive frames of image-preprocessed original images with fixed time intervals.

4. The AI ​​vision-based industrial robot remote monitoring system according to claim 3, characterized in that: The robot operation trajectory is composed of several trajectory points with continuous and fixed time intervals, including timestamps, 3D coordinates and attitude angles. The robot standard operation trajectory is composed of several standard trajectory points with continuous and fixed time intervals, including timestamps, 3D standard coordinates and standard attitude angles.

5. The AI ​​vision-based industrial robot remote monitoring system according to claim 4, characterized in that: The process of calculating the degree of fit of the robot's operating trajectory to determine whether to trigger the fault judgment module: Based on the timestamp, each trajectory point in the robot's operating trajectory is associated with the standard trajectory point in the robot's standard operating trajectory to obtain several trajectory point comparison pairs. For each trajectory point comparison pair, the position deviation, posture deviation and speed deviation of each trajectory point comparison pair are calculated using the position deviation formula, posture deviation formula and speed deviation formula respectively. Based on this, the number of trajectory point comparison pairs with a speed deviation of 0 and the maximum position deviation are counted to obtain the number of trajectory point comparison pairs with zero speed deviation and the maximum position deviation. At the same time, the mean of the posture deviations corresponding to all trajectory point comparison pairs is calculated to obtain the average posture deviation. The number of trajectory point comparison pairs with zero speed deviation, the maximum position deviation and the global average posture deviation are then substituted into the weighted fitting formula to calculate the fitting degree. If the degree of fit is greater than a preset degree of fit threshold, the fault judgment module is triggered; otherwise, the image acquisition module is triggered to obtain a new set of original operation images.

6. The AI ​​vision-based industrial robot remote monitoring system according to claim 5, characterized in that: The method for analyzing the motion sequence of the robot's operation trajectory: S1. Processing the robot's operating trajectory through a temporal segmentation model to obtain boundary trajectory points, and segmenting the robot's operating trajectory according to the boundary trajectory points to obtain several continuous trajectory segments; S2. Count the 3D coordinate range and attitude angle range of the trajectory points in each trajectory segment, calculate the speed corresponding to each trajectory point in each trajectory segment, and count the speed range of the trajectory points in the trajectory segment, thereby obtaining the motion characteristics of each trajectory segment, the motion characteristics including the 3D coordinate range, attitude angle range and speed range; S3. The motion features of each trajectory segment are matched with the motion features of all motion types in the motion feature library by the cosine similarity algorithm to determine the motion type corresponding to each trajectory segment. Based on this, the motion sequence of the robot's operating trajectory is obtained. The motion feature library is used to store motion features corresponding to various motion types of industrial robots; S4. Perform sequence comparison analysis on the action sequence of the robot's operation trajectory and the action sequence of the standard operation task to obtain the action sequence analysis results of action sequence logic errors or action sequence logic completeness. The action sequence logic errors are one or more of the following: missing action steps, reversed action order, and the presence of redundant actions.

7. The AI ​​vision-based industrial robot remote monitoring system according to claim 6, characterized in that: The process of fault analysis on the robot's operation trajectory: Based on the timestamp, each trajectory point in the robot's operating trajectory is associated with each standard trajectory point in the robot's standard operating trajectory to obtain several trajectory point comparison pairs. For each trajectory point comparison pair, the axis position deviation of each trajectory point comparison pair is calculated using the axis position deviation formula. The mean and standard deviation of the axis position deviation of all trajectory point comparison pairs are calculated to obtain the average position deviation and standard deviation of the axis position deviation. At the same time, the proportion of trajectory point comparison pairs with positive axis position deviation is counted to obtain the proportion of consistent signs of the axis position deviations. If the average position deviation of each axis is not 0, the standard deviation of the position deviation of each axis is less than the preset position deviation standard deviation threshold, and the proportion of consistent signs of the position deviation of each axis is greater than 80%, then there is a systematic offset; If the standard deviation of the position deviation of each axis is greater than the preset position deviation standard deviation threshold and the proportion of the position deviation signs of each axis is less than 60%, there is a random offset; If the position deviation of a certain trajectory point from the comparison of each axis is greater than 4 times the average deviation of the position of each axis, there is a sudden offset; If there are at least 5 consecutive adjacent trajectory points whose position deviations of each axis are greater than 3 times the average deviation of each axis, there is a local offset; Based on this, the deviation feature analysis result is obtained, and the deviation feature analysis result is mapped through the fault type matching rule library to obtain a preliminary mechanical fault type. The deviation feature analysis result is one or more of systematic offset, random offset, sudden offset and local offset. The position deviations of each axis include X-axis position deviation, Y-axis position deviation and Z-axis position deviation. The fault type matching rule library is used to store the preliminary mechanical fault type corresponding to the deviation feature analysis result.

8. The AI ​​vision-based industrial robot remote monitoring system according to claim 7, characterized in that: The prerequisite for performing fault analysis on the robot's operation trajectory is that the action sequence analysis result is that the action sequence logic is complete, and if the action sequence analysis result is that the action sequence logic is wrong, the fault diagnosis result is an action sequence logic error, otherwise, the fault diagnosis result is a preliminary mechanical fault type.

9. The AI ​​vision-based industrial robot remote monitoring system according to claim 8, characterized in that: The remote monitoring platform includes a visualization display submodule, an analysis data input module, an action logic correction submodule and a mechanical fault detection submodule; The visualization display submodule is used to display the robot operation trajectory, the robot standard operation trajectory and the fault diagnosis results through a 3D visualization interface, and at the same time provides an interactive button for correcting the action logic and an interactive button for determining the mechanical fault target. The interactive button for correcting the action logic is used to trigger the action logic correction submodule, and the interactive button for determining the mechanical fault target is used to receive the mechanical fault target input by the user according to the preliminary mechanical fault type and synchronously trigger the mechanical fault detection submodule; The action logic correction submodule is used to re-edit the action logic parameters through a visual flowchart editor and send the action logic parameters to the controller of the industrial robot through the OPCUA protocol; The mechanical fault detection submodule is used to automatically control the global industrial shutter camera to capture the mechanical fault target image based on the mechanical fault target, and identify the cause of the mechanical fault through the mechanical fault target image through the pre-trained defect detection model; The analysis data input module is used to add, delete, modify and check the preliminary mechanical fault types corresponding to the deviation feature analysis results in the fault type matching rule library, the robot standard operation trajectory and the action features corresponding to various action types in the action feature library.

Images