Operation monitoring method and system of industrial robots based on digital twins

Abstract

The present invention relates to the technical field of robot safety supervision, and specifically discloses a method and system for monitoring the operation of an industrial robot based on a digital twin. The method comprises acquiring visual information of the industrial robot, constructing and displaying a digital twin model containing a robot model based on the visual information; acquiring the robot's working data based on sensors installed inside the industrial robot, identifying the working data, and expanding an influence range on the robot model based on the identification results; querying safety targets within the influence range, querying target information of the safety targets, introducing negative influence points within the influence range based on the target information; and generating control instructions directed to the industrial robot based on the negative influence points. The present invention determines the influence range based on the robot model, determines control instructions based on the intersection of the influence range and the target to be protected, introduces a safety protection control mechanism, and greatly improves safety.

Description

Technical Field

[0001] The present invention relates to the technical field of robot safety supervision, and specifically to an operation monitoring method and system for an industrial robot based on digital twins. Background Art

[0002] With the continuous progress and development of science and technology in my country, the application of robots in the manufacturing and transportation industries has become more and more extensive at this stage. For example, the AGV unmanned guided vehicle in the generation scene can effectively improve the economic benefits of the enterprise and is of great significance to the intelligent development of the enterprise.

[0003] While industrial robots bring convenience, they also bring certain dangers. Once problems occur, they are likely to cause damage to surrounding equipment and personnel. For example, due to loss of control or component damage, some parts of the industrial robot may detach from the industrial robot. The mass of the industrial robot is very large, and once detached, it also has a certain initial velocity, which can easily cause damage to the equipment and personnel in the scene. Therefore, it is necessary to improve the safety of its working process; existing solutions to improve safety are mostly restrictive conditions. For example, the working process of the industrial robot must be away from equipment and personnel, and the distance must not be less than a preset value. This method is too simple and has poor intuitiveness. It is easy for staff to ignore the distance requirements. There is a technology in the existing technology called digital twin technology, which is used to construct three-dimensional models in real time and is highly intuitive. How to apply digital twin technology to prompt risks and improve the intuitiveness of the hazard display process is the technical problem that the technical solution of the present invention wants to solve. Summary of the Invention

[0004] The purpose of the present invention is to provide an operation monitoring method and system for an industrial robot based on digital twins to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for monitoring the operation of an industrial robot based on digital twins, the method comprising:

[0007] Acquire visual information of the industrial robot, and construct and display a digital twin model containing the robot model based on the visual information;

[0008] Acquire the robot's working data based on sensors installed inside the industrial robot, identify the working data, and expand the influence range on the robot model based on the identification results;

[0009] Querying security targets within the influence range, querying target information of the security targets, and introducing negative impact points within the influence range based on the target information;

[0010] A control instruction directed to the industrial robot is generated according to the negative impact point.

[0011] As a further solution of the present invention: the step of obtaining visual information of the industrial robot and constructing and displaying a digital twin model containing the robot model based on the visual information includes:

[0012] Obtain the real-time position of the industrial robot based on the locator built into the industrial robot;

[0013] Query the camera system for cameras within a preset range centered on the real-time location;

[0014] Generate an acquisition instruction directed to the camera to obtain a robot image containing camera parameters; the camera parameters include camera direction, camera position, camera wide angle and camera time;

[0015] Update the robot model based on the robot image and its camera parameters;

[0016] The robot model is inserted into the BIM model of the industrial area based on the real-time location to obtain a digital twin model.

[0017] As a further solution of the present invention: the step of updating the robot model based on the robot image and its camera parameters includes:

[0018] Count the robot images taken at the same time, and determine the image shooting distance based on the shooting position and real-time position of each robot image; the same shooting time means that the difference in shooting time between two robot images is less than a preset difference threshold;

[0019] Determine the cone range according to the image shooting distance, the shooting direction and the shooting wide angle, calculate the intersection of the cone range and the surface of the robot model, and obtain the mapping range;

[0020] Perform contour recognition on the robot image to obtain the actual contour of the robot, and correct the boundary of the robot model in the mapping range based on the actual contour;

[0021] Perform color value recognition on the robot image to obtain color value parameters within the mapping range, and modify the display parameters of the robot model within the mapping range according to the color value parameters;

[0022] Among them, when the correction source is not unique, the mean is used in the correction process.

[0023] As a further solution of the present invention, the steps of obtaining the working data of the robot according to the sensor installed inside the industrial robot, identifying the working data, and extending the influence range on the robot model according to the identification result include:

[0024] Obtaining the robot's working data based on the sensors installed inside the industrial robot;

[0025] Sort the working data of each sensor in time series to obtain a data sequence;

[0026] Inputting the data sequence into a trained anomaly recognition model to obtain anomaly probability;

[0027] The impact range is expanded on the robot model based on the abnormal probability of each sensor.

[0028] As a further solution of the present invention, the step of expanding the influence range on the robot model based on the abnormal probability of each sensor includes:

[0029] Query the components corresponding to each sensor to obtain the component's influence vector; the direction of the influence vector is the direction of movement of the component when it leaves the robot, and the modulus of the influence vector is the average initial velocity of the component when it leaves the robot;

[0030] Reading the abnormality probability, and correcting the modulus of the impact vector according to the abnormality probability;

[0031] Simulate the motion trajectory of the corresponding component according to the corrected influence vector;

[0032] Determine the impact range of the component based on the motion trajectory;

[0033] Count and fit the influence ranges of all components to obtain the final influence range;

[0034] Among them, the correction process of the modulus length is:

[0035] Where, is the corrected modulus length, is the module length before correction, is the abnormal probability, is the preset basic abnormal probability, is a preset constant.

[0036] As a further solution of the present invention, the steps of querying the security targets within the influence range, querying the target information of the security targets, and introducing negative impact points within the influence range according to the target information include:

[0037] Real-time query of safety targets within the impact range in the digital twin model; the safety targets include equipment and personnel;

[0038] When the security target is a device, query the importance of the device;

[0039] When the security target is personnel, set the importance to the preset maximum value;

[0040] Create negative impact points using the location of the security target as the center and the importance as the radius;

[0041] The step of generating a control instruction directed to the industrial robot according to the negative impact point includes:

[0042] When the influence range intersects with any negative influence point, the component corresponding to the intersection in the robot model is queried, and a control instruction directed to the component is generated, wherein the control instruction is used to adjust the separation direction of the component.

[0043] The technical solution of the present invention also provides an operation monitoring system for an industrial robot based on digital twins, the system comprising:

[0044] A twin model construction module, used to obtain visual information of the industrial robot, and to construct and display a digital twin model containing the robot model based on the visual information;

[0045] An influence range expansion module is used to obtain the robot's working data based on sensors installed inside the industrial robot, identify the working data, and expand the influence range on the robot model based on the identification results;

[0046] A negative impact point insertion module is used to query the safety target within the impact range, query the target information of the safety target, and introduce negative impact points within the impact range according to the target information;

[0047] A control instruction generating module is used to generate a control instruction directed to the industrial robot according to the negative impact point.

[0048] As a further solution of the present invention: the twin model construction module includes:

[0049] A real-time position acquisition unit, used to acquire the real-time position of the industrial robot based on a locator built into the industrial robot;

[0050] A camera query unit, used to query cameras within a preset range centered on a real-time position in the camera system;

[0051] An image acquisition unit, configured to generate an acquisition instruction directed to a camera to acquire a robot image containing camera parameters; the camera parameters include camera direction, camera position, camera wide angle, and camera time;

[0052] A robot modeling unit, configured to update a robot model based on the robot image and its camera parameters;

[0053] The model insertion unit is used to insert the robot model into the BIM model of the industrial area based on the real-time position to obtain a digital twin model.

[0054] As a further solution of the present invention: the impact range expansion module includes:

[0055] A working data acquisition unit, used to acquire working data of the robot based on sensors installed inside the industrial robot;

[0056] A data sorting unit is used to sort the working data of each sensor in time sequence to obtain a data sequence;

[0057] An anomaly recognition unit, configured to input the data sequence into a trained anomaly recognition model to obtain an anomaly probability;

[0058] The extended execution unit is used to extend the influence range on the robot model based on the abnormal probability of each sensor.

[0059] As a further solution of the present invention: the negative impact point insertion module includes:

[0060] A first query unit is used to query the security targets within the influence range in real time in the digital twin model; the security targets include equipment and personnel;

[0061] The second query unit is used to query the importance of the device when the security target is a device;

[0062] A default setting unit is used to set the importance to a preset maximum value when the security target is a person;

[0063] A negative impact point creation unit is used to create negative impact points using the location of the safety target as the center of the circle and the importance as the radius;

[0064] The control instruction generation module includes:

[0065] An execution unit is generated, which is used to query the component corresponding to the intersection in the robot model when the influence range intersects with any negative influence point, and generate a control instruction pointing to the component, wherein the control instruction is used to adjust the separation direction of the component.

[0066] Compared with the existing technology, the beneficial effects of the present invention are: the present invention uses existing cameras in industrial scenes to build a digital twin model containing a robot model in real time, obtains the working status of the robot according to the sensors built into the robot, and then determines the influence range based on the robot model. The control instructions are determined according to the intersection of the influence range and the target to be protected, and a safety protection control mechanism is introduced, which greatly improves safety. At the same time, with the help of the intuitiveness of the digital twin model, workers can observe dangerous situations more clearly. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention.

[0068] Figure 1 This is a flowchart of the operation monitoring method of industrial robots based on digital twins.

[0069] Figure 2 This is the first sub-process flowchart of the operation monitoring method of an industrial robot based on digital twins.

[0070] Figure 3 This is the second sub-process flowchart of the operation monitoring method of an industrial robot based on digital twins.

[0071] Figure 4 This is the third sub-process flowchart of the operation monitoring method of industrial robots based on digital twins.

[0072] Figure 5 This is the fourth sub-process flowchart of the operation monitoring method of an industrial robot based on digital twins.

[0073] Figure 6 This is a structural block diagram of the operation monitoring system of an industrial robot based on digital twins. DETAILED DESCRIPTION

[0074] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0075] Figure 1 This is a flowchart of a method for monitoring the operation of an industrial robot based on digital twins. In an embodiment of the present invention, a method for monitoring the operation of an industrial robot based on digital twins includes:

[0076] Step S100: Acquire visual information of the industrial robot, and construct and display a digital twin model containing the robot model based on the visual information;

[0077] Industrial robots generally operate in industrial areas. Cameras are installed in these areas. Videos of industrial robots at work are captured through the cameras, which is called visual information. By identifying this visual information, we can build a robot model and a scene model of the industrial area where the robot is located, collectively referred to as a digital twin model.

[0078] Step S200: obtaining the robot's working data according to sensors installed inside the industrial robot, identifying the working data, and expanding the influence range on the robot model according to the identification result;

[0079] There are multiple sensors inside the industrial robot. These sensors are used to obtain the work data generated during the working process. By identifying the work data, the working status of each part of the industrial robot can be obtained, and then the degree of risk can be determined. Based on the judgment results, the radiation range of the robot model can be determined in the digital twin model, which is called the impact range.

[0080] It should be noted that the visual information of the technical solution of the present invention is obtained in real time, and the working data is also obtained in real time. The visual information and working data at each moment are different. Therefore, the robot model and the influence range are different. It is equivalent to introducing a dynamic robot model and its influence range in the scene model.

[0081] Step S300: querying security targets within the influence range, querying target information of the security targets, and introducing negative impact points within the influence range according to the target information;

[0082] The impact obtained by the technical solution of the present invention is the impact on equipment or personnel. The safety targets within the impact range are queried in real time to obtain the target information of the safety targets. The target information includes the equipment type and personnel type, etc. The target information is analyzed to introduce negative impact points within the impact range of the robot model in the digital twin model; the negative impact point is generally a spherical area, and it is not an actual point.

[0083] Step S400: generating a control instruction directed to the industrial robot according to the negative impact point;

[0084] A control instruction directed to the industrial robot is generated according to the negative impact point, so as to adjust the impact direction of the industrial robot and avoid its impact.

[0085] Figure 2 This is a block diagram of the first sub-process of the operation monitoring method of an industrial robot based on digital twins. The steps of obtaining visual information of the industrial robot and constructing and displaying a digital twin model containing a robot model based on the visual information include:

[0086] Step S101: obtaining the real-time position of the industrial robot based on a locator built into the industrial robot;

[0087] Step S102: querying the camera system for cameras within a preset range centered on the real-time position;

[0088] Step S103: generating an acquisition instruction directed to the camera to obtain a robot image containing camera parameters; the camera parameters include camera direction, camera position, camera wide angle, and camera time;

[0089] Step S104: updating the robot model based on the robot image and its camera parameters;

[0090] Step S105: Insert the robot model into the industrial area BIM model based on the real-time position to obtain a digital twin model.

[0091] In an example of the technical solution of the present invention, the generation process of the digital twin model is explained. The position of the industrial robot is obtained in real time based on the locator built into the industrial robot to obtain the position at each moment. Then, the camera within a preset range centered on the real-time position is queried in the camera system, and an acquisition instruction is generated and sent to the camera. At this time, the camera performs acquisition. In fact, many cameras are running in real time. In addition to triggering the acquisition process, the acquisition instruction can also improve the acquisition clarity. In addition, the cameras in the present invention include multiple types, some of which are cameras installed on the ceiling, and some are cameras installed on the equipment. These can all be used to collect images. Therefore, when obtaining the robot image, it is necessary to record the camera parameters, which include camera direction, camera position, camera wide angle and camera time; based on the robot image and its camera parameters, the existing robot model is updated, and the initial state of the robot model is known; based on the real-time position, the robot model is inserted into the industrial area BIM model to obtain a digital twin model.

[0092] As a preferred embodiment of the technical solution of the present invention, the step of updating the robot model based on the robot image and its camera parameters includes:

[0093] Count the robot images taken at the same time, and determine the image shooting distance based on the shooting position and real-time position of each robot image; the same shooting time means that the difference in shooting time between two robot images is less than a preset difference threshold;

[0094] Determine the cone range according to the image shooting distance, the shooting direction and the shooting wide angle, calculate the intersection of the cone range and the surface of the robot model, and obtain the mapping range;

[0095] Perform contour recognition on the robot image to obtain the actual contour of the robot, and correct the boundary of the robot model in the mapping range based on the actual contour;

[0096] Perform color value recognition on the robot image to obtain color value parameters within the mapping range, and correct the display parameters of the robot model within the mapping range according to the color value parameters.

[0097] In one embodiment of the technical solution of the present invention, statistics are collected for each robot image with a sufficiently small time difference. The image shooting distance is determined based on the camera position and real-time position of each robot image. A conical range is determined based on the image shooting distance, camera direction, and camera wide angle. The intersection of the conical range and the surface of the robot model is calculated to obtain a mapping range. The mapping range indicates which areas of the robot model the robot image corresponds to. Contour recognition is performed on the robot image to obtain the actual contour of the robot. The boundary of the robot model within the mapping range is corrected based on the actual contour. Color value recognition is performed on the robot image to obtain color value parameters within the mapping range. Display parameters of the robot model within the mapping range are corrected based on the color value parameters.

[0098] It is worth mentioning that the number of robot images corresponding to a certain moment may not be unique. The robot image corresponding to a certain moment is the correction source. At this time, the correction results will appear in multiple ways. When the correction source is not unique, the correction process uses the mean of the correction results. For example, when correcting the contour, the mean of the correction amplitude is calculated, and when correcting the color value, the mean of the correction color value is calculated.

[0099] Figure 3 This is a block diagram of the second sub-process of the operation monitoring method of an industrial robot based on digital twins. The steps of obtaining the robot's operating data based on sensors installed inside the industrial robot, identifying the operating data, and expanding the influence range on the robot model based on the identification results include:

[0100] Step S201: obtaining the robot's working data based on sensors installed inside the industrial robot;

[0101] Step S202: sorting the working data of each sensor in time sequence to obtain a data sequence;

[0102] Step S203: inputting the data sequence into the trained abnormality recognition model to obtain abnormality probability;

[0103] Step S204: Expand the influence range on the robot model based on the abnormal probability of each sensor.

[0104] In one example of the technical solution of the present invention, the robot's working data is obtained based on the sensors installed inside the industrial robot. The working data contains time tags. The working data of each sensor is sorted in time series to obtain a data sequence. The data sequence is input into a trained anomaly recognition model to obtain anomaly probability. The anomaly recognition model can use an existing data recognition algorithm. Generally, the data size and data change rate are independent variables; the greater the anomaly probability, the greater the possibility of an anomaly; and the influence range is expanded on the robot model based on the anomaly probability of each sensor.

[0105] As a preferred embodiment of the technical solution of the present invention, the step of expanding the influence range on the robot model based on the abnormal probability of each sensor includes:

[0106] Query the components corresponding to each sensor to obtain the component's influence vector; the direction of the influence vector is the direction of movement of the component when it leaves the robot, and the modulus of the influence vector is the average initial velocity of the component when it leaves the robot;

[0107] Reading the abnormality probability, and correcting the modulus of the impact vector according to the abnormality probability;

[0108] Simulate the motion trajectory of the corresponding component according to the corrected influence vector;

[0109] Determine the impact range of the component based on the motion trajectory;

[0110] The influence ranges of all components are counted and fitted to obtain the final influence range.

[0111] In one example of the technical solution of the present invention, the components corresponding to each sensor are queried, and the movement direction of the component when it detaches from the robot is queried. The historical data is queried to find out the average initial rate of detachment once the component detaches. These two parameters are combined to obtain the influence vector; the abnormality probability corresponding to the sensor is read, and the abnormality probability of the component is calculated (if a component corresponds to multiple sensors, the maximum abnormality probability is selected), and the modulus of the influence vector is corrected according to the abnormality probability. At this time, the corrected influence vector represents the direction and degree of the possible impact on the surrounding area.

[0112] The motion trajectory of the corresponding component is simulated according to the corrected influence vector. The simulation process is generally a free fall with initial velocity. The component model is extended along the motion trajectory to obtain the influence range; the influence ranges of all components are counted and fitted to obtain the final influence range.

[0113] Among them, the correction process of the modulus length is:

[0114] Where, is the corrected modulus length, is the module length before correction, is the abnormal probability, is the preset basic abnormal probability, is a preset constant.

[0115] Regarding the process of correcting the modulus length, an exponential function with the natural exponential as the base is used, and its value is greater than zero, which means that the corrected modulus length will not be negative; the difference between the abnormal probability and the basic abnormal probability is calculated, and then the difference is amplified. If the abnormal probability is large, the modulus length will become larger, and if the abnormal probability is small, the modulus length will become smaller, thereby realizing the modulus length adjustment process based on the abnormal probability.

[0116] Figure 4 This is a block diagram of the third sub-process of the operation monitoring method of an industrial robot based on digital twins. The steps of querying the safety target within the influence range, querying the target information of the safety target, and introducing a negative impact point within the influence range according to the target information include:

[0117] Step S301: querying the security targets within the impact range in real time in the digital twin model; the security targets include equipment and personnel;

[0118] Step S302: When the security target is a device, query the importance of the device;

[0119] Step S303: When the security target is a person, the importance is set to a preset maximum value;

[0120] Step S304: Create negative impact points using the location of the safety target as the center of the circle and the importance as the radius.

[0121] In an example of the technical solution of the present invention, the equipment and personnel that need to be protected within the impact range are queried in real time in the digital twin model, which is called a safety target. When the safety target is a device, the importance of the device is queried. The importance is pre-stored in a preset table and can be directly read in actual application. When the security target is a person, the importance is set to a preset maximum value; the position of the safety target is used as the center of the circle, and the importance is used as the radius to create a negative impact point. The state of the created negative impact point in the model is actually a sphere, so it can also be called a negative impact sphere.

[0122] Figure 5 This is a fourth sub-flow chart of the operation monitoring method of an industrial robot based on digital twins, wherein the step of generating a control instruction directed to the industrial robot according to the negative impact point includes:

[0123] Step S401: When the influence range intersects with any negative influence point, the component corresponding to the intersection in the robot model is queried, and a control instruction directed to the component is generated, wherein the control instruction is used to adjust the separation direction of the component.

[0124] In one example of the technical solution of the present invention, when the influence range intersects with any negative influence point, the component corresponding to the intersection in the robot model is queried, and a control instruction pointing to the component is generated. The control instruction is used to adjust the detachment direction of the component. The simplest control instruction is a rotation instruction, which turns the direction of the component to the ground without affecting the surroundings. Another control instruction is actually a protection instruction pointing to the component, which is used to control protective equipment such as safety valves to block the detachment process of the component.

[0125] Figure 6 The following is a structural block diagram of an industrial robot operation monitoring system based on digital twins. In an embodiment of the present invention, an industrial robot operation monitoring system based on digital twins is provided. The system 10 includes:

[0126] A twin model construction module 11 is used to obtain visual information of the industrial robot, and to construct and display a digital twin model containing the robot model based on the visual information;

[0127] The influence range expansion module 12 is used to obtain the robot's working data based on the sensor installed inside the industrial robot, identify the working data, and expand the influence range on the robot model based on the identification result;

[0128] A negative impact point insertion module 13 is used to query the security targets within the impact range, query the target information of the security targets, and introduce negative impact points within the impact range according to the target information;

[0129] The control instruction generating module 14 is configured to generate a control instruction directed to the industrial robot according to the negative impact point.

[0130] Furthermore, the twin model construction module 11 includes:

[0131] A real-time position acquisition unit, used to acquire the real-time position of the industrial robot based on a locator built into the industrial robot;

[0132] A camera query unit, used to query cameras within a preset range centered on a real-time position in the camera system;

[0133] An image acquisition unit, configured to generate an acquisition instruction directed to a camera to acquire a robot image containing camera parameters; the camera parameters include camera direction, camera position, camera wide angle, and camera time;

[0134] A robot modeling unit, configured to update a robot model based on the robot image and its camera parameters;

[0135] The model insertion unit is used to insert the robot model into the BIM model of the industrial area based on the real-time position to obtain a digital twin model.

[0136] Specifically, the impact range expansion module 12 includes:

[0137] A working data acquisition unit, used to acquire working data of the robot based on sensors installed inside the industrial robot;

[0138] A data sorting unit is used to sort the working data of each sensor in time sequence to obtain a data sequence;

[0139] An anomaly recognition unit, configured to input the data sequence into a trained anomaly recognition model to obtain an anomaly probability;

[0140] The extended execution unit is used to extend the influence range on the robot model based on the abnormal probability of each sensor.

[0141] Furthermore, the negative impact point insertion module 13 includes:

[0142] A first query unit is used to query the security targets within the influence range in real time in the digital twin model; the security targets include equipment and personnel;

[0143] The second query unit is used to query the importance of the device when the security target is a device;

[0144] A default setting unit is used to set the importance to a preset maximum value when the security target is a person;

[0145] A negative impact point creation unit is used to create negative impact points using the location of the safety target as the center of the circle and the importance as the radius;

[0146] The control instruction generation module includes:

[0147] An execution unit is generated, which is used to query the component corresponding to the intersection in the robot model when the influence range intersects with any negative influence point, and generate a control instruction pointing to the component, wherein the control instruction is used to adjust the separation direction of the component.

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for monitoring the operation of an industrial robot based on digital twins, characterized in that: The method comprises: Acquire visual information of the industrial robot, and construct and display a digital twin model containing the robot model based on the visual information; Acquire the robot's working data based on sensors installed inside the industrial robot, identify the working data, and expand the influence range on the robot model based on the identification results; Querying security targets within the influence range, querying target information of the security targets, and introducing negative impact points within the influence range based on the target information; generating a control instruction directed to the industrial robot according to the negative impact point; The steps of acquiring the working data of the robot according to the sensor installed inside the industrial robot, identifying the working data, and expanding the influence range on the robot model according to the identification result include: Obtaining the robot's working data based on the sensors installed inside the industrial robot; Sort the working data of each sensor in time series to obtain a data sequence; Inputting the data sequence into a trained anomaly recognition model to obtain anomaly probability; Expand the impact range on the robot model based on the abnormal probability of each sensor; The step of expanding the influence range on the robot model based on the abnormal probability of each sensor includes: Query the components corresponding to each sensor to obtain the component's influence vector; the direction of the influence vector is the direction of movement of the component when it leaves the robot, and the modulus of the influence vector is the average initial velocity of the component when it leaves the robot; Reading the abnormality probability, and correcting the modulus of the impact vector according to the abnormality probability; Simulate the motion trajectory of the corresponding component according to the corrected influence vector; Determine the impact range of the component based on the motion trajectory; Count and fit the influence ranges of all components to obtain the final influence range; Among them, the correction process of the modulus length is: Where, is the corrected modulus length, is the module length before correction, is the abnormal probability, is the preset basic abnormal probability, is a preset constant.

2. The operation monitoring method of an industrial robot based on digital twin according to claim 1, characterized in that: The steps of obtaining visual information of the industrial robot and constructing and displaying a digital twin model containing the robot model based on the visual information include: Obtain the real-time position of the industrial robot based on the locator built into the industrial robot; Query the camera system for cameras within a preset range centered on the real-time location; Generate an acquisition instruction directed to the camera to obtain a robot image containing camera parameters; the camera parameters include camera direction, camera position, camera wide angle and camera time; Update the robot model based on the robot image and its camera parameters; The robot model is inserted into the BIM model of the industrial area based on the real-time location to obtain a digital twin model.

3. The operation monitoring method of an industrial robot based on digital twin according to claim 2, characterized in that: The step of updating the robot model based on the robot image and its camera parameters includes: Count the robot images taken at the same time, and determine the image shooting distance based on the shooting position and real-time position of each robot image; the same shooting time means that the difference in shooting time between two robot images is less than a preset difference threshold; Determine the cone range according to the image shooting distance, the shooting direction and the shooting wide angle, calculate the intersection of the cone range and the surface of the robot model, and obtain the mapping range; Perform contour recognition on the robot image to obtain the actual contour of the robot, and correct the boundary of the robot model in the mapping range based on the actual contour; Perform color value recognition on the robot image to obtain color value parameters within the mapping range, and modify the display parameters of the robot model within the mapping range according to the color value parameters; Among them, when the correction source is not unique, the mean is used in the correction process.

4. The operation monitoring method of an industrial robot based on digital twin according to claim 1, characterized in that: The steps of querying the security target within the influence range, querying the target information of the security target, and introducing a negative impact point within the influence range according to the target information include: Real-time query of safety targets within the impact range in the digital twin model; the safety targets include equipment and personnel; When the security target is a device, query the importance of the device; When the security target is personnel, set the importance to the preset maximum value; Create negative impact points using the location of the security target as the center and the importance as the radius; The step of generating a control instruction directed to the industrial robot according to the negative impact point includes: When the influence range intersects with any negative influence point, the component corresponding to the intersection in the robot model is queried, and a control instruction directed to the component is generated, wherein the control instruction is used to adjust the separation direction of the component.

5. An industrial robot operation monitoring system based on digital twins, characterized in that: The system is used to implement the operation monitoring method of an industrial robot based on digital twin according to any one of claims 1 to 4, and the system includes: A twin model construction module, used to obtain visual information of the industrial robot, and to construct and display a digital twin model containing the robot model based on the visual information; An influence range expansion module is used to obtain the robot's working data based on sensors installed inside the industrial robot, identify the working data, and expand the influence range on the robot model based on the identification results; A negative impact point insertion module is used to query the safety target within the impact range, query the target information of the safety target, and introduce negative impact points within the impact range according to the target information; A control instruction generating module, configured to generate a control instruction directed to the industrial robot according to the negative impact point; The impact range expansion module includes: A working data acquisition unit, used to acquire working data of the robot based on sensors installed inside the industrial robot; A data sorting unit is used to sort the working data of each sensor in time sequence to obtain a data sequence; An anomaly recognition unit, configured to input the data sequence into a trained anomaly recognition model to obtain an anomaly probability; The extended execution unit is used to extend the influence range on the robot model based on the abnormal probability of each sensor.

6. The operation monitoring system of an industrial robot based on digital twin according to claim 5, characterized in that: The twin model building module includes: A real-time position acquisition unit, used to acquire the real-time position of the industrial robot based on a locator built into the industrial robot; A camera query unit, used to query cameras within a preset range centered on a real-time position in the camera system; An image acquisition unit, configured to generate an acquisition instruction directed to a camera to acquire a robot image containing camera parameters; the camera parameters include camera direction, camera position, camera wide angle, and camera time; A robot modeling unit, configured to update a robot model based on the robot image and its camera parameters; The model insertion unit is used to insert the robot model into the BIM model of the industrial area based on the real-time position to obtain a digital twin model.

7. The operation monitoring system of an industrial robot based on digital twin according to claim 5, characterized in that: The negative impact point insertion module includes: A first query unit is used to query the security targets within the influence range in real time in the digital twin model; the security targets include equipment and personnel; The second query unit is used to query the importance of the device when the security target is a device; A default setting unit is used to set the importance to a preset maximum value when the security target is a person; A negative impact point creation unit is used to create negative impact points using the location of the safety target as the center of the circle and the importance as the radius; The control instruction generation module includes: An execution unit is generated, which is used to query the component corresponding to the intersection in the robot model when the influence range intersects with any negative influence point, and generate a control instruction pointing to the component, wherein the control instruction is used to adjust the separation direction of the component.

Images