Multi-vision machine pose adjusting system and method based on binocular vision

Abstract

The invention relates to the technical field of machine position control, and discloses a multi-vision machine pose adjustment system and method based on binocular vision, and the system comprises an industrial robot pose model construction module, an industrial robot pose monitoring module, and an industrial robot pose adjustment module. According to standard industrial robot real-time pose image parameters acquired based on binocular machine vision, an intelligent recognition algorithm and scientifically preset industrial robot space pose model parameters are combined to intelligently construct an industrial robot real-time space pose three-dimensional model, and dynamic digital construction of the industrial robot space pose model is achieved; and according to the industrial robot real-time space pose abnormal state analysis parameters, intelligent matching of industrial robot real-time space pose abnormal adjustment parameters is carried out in combination with an intelligent search algorithm and standard stored industrial robot abnormal space pose adjustment parameters, and dynamic construction of industrial robot space pose fault calibration control parameters is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine position control, and specifically to a multi-vision machine pose adjustment system and method based on binocular vision. Background Art

[0002] Industrial robots have characteristics such as programmability, strong versatility, high repeatability, and human-robot collaboration. Through the application of industrial robots, enterprises can achieve the automation of production lines, improve production efficiency, reduce production costs, and meet personalized and diversified production needs. Binocular stereo vision is an important form of machine vision. It is a method based on the principle of parallax and uses imaging devices to obtain two images of the object to be measured from different positions, and obtains the three-dimensional geometric information of the object by calculating the position deviation between corresponding points in the images. By fusing the images obtained by the two eyes and observing the differences between them, we can obtain an obvious sense of depth, establish the corresponding relationship between features, and correspond the image points of the same spatial physical point in different images. In the operation process of existing industrial robots, the pose state control is all carried out through the set motion trajectory. The industrial robot cannot autonomously and efficiently judge its own current abnormal pose state during the operation process, nor can it accurately and intelligently adjust the current abnormal pose state, which reduces the reliability and safety of the operation control of the industrial robot.

[0003] The Chinese invention patent with the publication number CN111427370B discloses a Gmapping mapping method for a mobile robot based on sparse pose adjustment, which realizes the pose adjustment of the mobile robot through technical solutions such as initializing particle poses and distributions, scan matching, calculating the target distribution of the sampling position, calculating the Gaussian approximation, updating the weight of the i-th particle, updating the particle map and pose map to construct a closed-loop constraint; however, the above technical solutions cannot realize the intelligent adjustment of the abnormal pose state of the mobile robot during the operation process. Summary of the Invention

[0004] (1) Technical Problems to be Solved

[0005] To solve the problem that the industrial robot cannot autonomously and efficiently judge its own current abnormal pose state during the operation process, nor can it accurately and intelligently adjust the current abnormal pose state, which reduces the reliability and safety of the operation control of the industrial robot, and achieve the purposes of dynamically constructing the spatial pose model of the above industrial robot, scientifically analyzing the abnormal state of the industrial robot's spatial pose, accurately matching the abnormal adjustment parameters of the industrial robot's spatial pose, intelligently adjusting the abnormal state of the industrial robot's spatial pose, and improving the reliability and intelligence of the operation control of the industrial robot.

[0006] (2) Technical Solutions

[0007] The present invention is realized through the following technical solutions: A multi-vision robot pose adjustment method based on binocular vision, the method comprising the following steps:

[0008] S1. Collect real-time pose image data of an industrial robot;

[0009] S2. Perform preprocessing on the real-time pose image of the industrial robot based on the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot;

[0010] S3. Perform processing on the construction of a three-dimensional model of the real-time spatial pose of the industrial robot according to the standard real-time pose image data of the industrial robot and the spatial pose model data of the industrial robot to generate real-time spatial pose model data of the industrial robot;

[0011] S4. Perform analysis processing on the abnormal state of the real-time spatial pose of the industrial robot based on the real-time spatial pose model data of the industrial robot and the abnormal spatial pose model data of the industrial robot to generate analysis data on the abnormal state of the real-time spatial pose of the industrial robot; when normal, end the current industrial robot pose adjustment operation;

[0012] S5. When abnormal, perform matching processing on the abnormal adjustment parameters of the real-time spatial pose of the industrial robot according to the analysis data on the abnormal state of the real-time spatial pose of the industrial robot and the abnormal spatial pose adjustment data of the industrial robot to generate abnormal adjustment data of the real-time spatial pose of the industrial robot;

[0013] S6. Execute the industrial robot spatial pose adjustment operation according to the abnormal adjustment data of the real-time spatial pose of the industrial robot;

[0014] S7. Construct monitoring data for the real-time spatial pose adjustment of the industrial robot and execute the industrial robot spatial pose adjustment feedback operation.

[0015] Preferably, the operation steps for collecting the real-time pose image data of the industrial robot are as follows:

[0016] S11. Simultaneously collect spatial pose image information of the operating state of the industrial robot from two different spatial positions through a binocular vision imaging processing device, and generate real-time pose image data of the industrial robot A = (a1, a2), where a1 represents the real-time pose image data of the first view of the industrial robot, and a2 represents the real-time pose image data of the second view of the industrial robot. The binocular vision imaging processing device includes any one of the binocular stereo cameras of BASLER and the Gemini binocular structured light 3D camera of Orbbec.

[0017] The present invention dynamically collects the real-time pose image parameters of an industrial robot through a binocular vision imaging processing device, achieving the effect of efficiently collecting the real-time pose image parameters of an industrial robot based on binocular machine vision.

[0018] Preferably, the operation steps for preprocessing the real-time pose image of the industrial robot based on the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot are as follows:

[0019] S21. Use the BM3D algorithm to perform noise reduction preprocessing on the real-time pose image data a1 of the first view of the industrial robot and the real-time pose image data a2 of the second view of the industrial robot in the real-time pose image data A of the industrial robot, and generate standard real-time pose image data A'=(a'1, a'2), where a'1 represents the standard real-time pose image data of the first view of the industrial robot, and a'2 represents the standard real-time pose image data of the second view of the industrial robot.

[0020] The present invention performs noise reduction preprocessing on the real-time pose image of the industrial robot by combining the real-time pose image parameters of the industrial robot with the BM3D algorithm, achieving the effect of accurately collecting the real-time pose image parameters of the industrial robot.

[0021] Preferably, the operation steps for constructing a three-dimensional model of the real-time spatial pose of the industrial robot based on the standard real-time pose image data of the industrial robot and the spatial pose model data of the industrial robot to generate real-time spatial pose model data of the industrial robot are as follows:

[0022] S31. Establish a set M=(m1,…,m g ,…,m κ ) of spatial pose model data of the industrial robot, where g = 1, 2, 3, …, κ; where m g represents the spatial pose model data of the industrial robot corresponding to the g-th type of binocular vision industrial robot pose image, and κ represents the maximum value of the number of types of binocular vision industrial robot pose images; the binocular vision industrial robot pose image represents the image information of different types of running postures of the industrial robot collected online from two different spatial positions simultaneously by the binocular vision imaging processing device; the spatial pose model data of the industrial robot represents the standard three-dimensional entity model parameters of the spatial pose of the industrial robot set according to the characteristics of the binocular vision industrial robot pose image;

[0023] S32. Compare the standard real-time pose image data a'1 of the first view of the industrial robot and the standard real-time pose image data a'2 of the second view of the industrial robot in the standard real-time pose image data A' of the industrial robot with the spatial pose model data m of the industrial robot in the set M of spatial pose model data of the industrial robotg Perform industrial robot pose image feature matching to search for the industrial robot spatial pose model data m that matches the real-time pose image data a'1 of the first view of the standard industrial robot and the real-time pose image data a'2 of the second view of the standard industrial robot g and construct the real-time spatial pose model data m of the industrial robot A Execute to generate the specific operation steps of the real-time spatial pose model data m of the industrial robot A are as follows:

[0024] S321. Initialize the algorithm parameters, the population size N, and the maximum number of iterations T;

[0025] S322. Initialize the population, calculate the fitness, and determine the pose search pathfinder and the pose search follower;

[0026] S323. Update the position of the pose search pathfinder in the search space of the industrial robot spatial pose model data set M according to the position formula where t represents the current iteration generation of the algorithm; represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model data set M after the t-th iteration, represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model data set M after the (t - 1)-th iteration, represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model data set M after the (t + 1)-th iteration, is the step size factor for the movement of the pose search pathfinder, obeys a uniform distribution within [0, 1];

[0027] S324. Update the position of the pose search follower in the search space of the industrial robot spatial pose model data set M according to the position formula where represents the position of the pose search follower i in the search space of the industrial robot spatial pose model data set M after the t-th iteration, represents the position of the pose search follower i in the search space of the industrial robot spatial pose model data set M after the (t + 1)-th iteration; represents the position of other pose search followers j in the search space of the industrial robot spatial pose model data set M after the t-th iteration, and the position of the pose search follower moves not only related to the position of the pose search pathfinder but also affected by the positions of other pose search followers Regarding the influence, R1 represents the position distance parameter between pose search followers in the search space of the industrial robot spatial pose model data set M, and R2 represents the position distance parameter between the pose search pathfinder and the pose search followers in the search space of the industrial robot spatial pose model data set M. R1 = ◇×r1, R2 = ∫×r2; ◇ represents the interaction coefficient between pose search followers, and ∫ represents the attraction coefficient of the pose search pathfinder to the pose search followers. Both ◇ and ∫ follow a uniform distribution in [1, 2]; r1 is the step factor for the pose search followers to move relative to other pose search followers, and r2 is the step factor for the pose search followers and the pose search pathfinder to move. Both r1 and r2 are random numbers within the range of [0, 1];

[0028] S325. Calculate the fitness values of all the industrial robot spatial pose model data m in the search space of the industrial robot spatial pose model data set M for the first-view real-time pose image data a'1 of the standard industrial robot and the second-view real-time pose image data a'2 of the standard industrial robot, and update the industrial robot spatial pose model data m with the maximum fitness value for the first-view real-time pose image data a'1 of the standard industrial robot and the second-view real-time pose image data a'2 of the standard industrial robot. g as the global optimal value; g

[0029] S326. When the maximum iteration number T is reached, output the industrial robot spatial pose model data m that is the most matched with the first-view real-time pose image data a'1 of the standard industrial robot and the second-view real-time pose image data a'2 of the standard industrial robot searched in step S325 g and generate the industrial robot real-time spatial pose model data m through data identification. A .

[0030] The present invention intelligently constructs a three-dimensional model of the industrial robot real-time spatial pose by combining the real-time pose image parameters of the standard industrial robot collected based on binocular machine vision with the artificial intelligence pathfinder optimization algorithm and the scientifically preset industrial robot spatial pose model parameters, achieving the effect of dynamically digital construction of the industrial robot spatial pose model.

[0031] Preferably, based on the industrial robot real-time spatial pose model data and the industrial robot abnormal spatial pose model data, perform analysis and processing on the abnormal state of the industrial robot real-time spatial pose to generate industrial robot real-time spatial pose abnormal state analysis data; when normal, the operation steps to end the current industrial robot pose adjustment operation are as follows:

[0032] S41. Establish an industrial robot spatial pose model data set U = (u1,..., ub , …, u λ ), b = 1, 2, 3, …, λ; where u b represents the industrial robot spatial pose model data corresponding to the b-th type of abnormal spatial pose of the industrial robot, and λ represents the maximum value of the number of types of abnormal spatial poses of the industrial robot; the types of abnormal spatial poses of the industrial robot include the spatial pose with the industrial robot joint movement exceeding the limit, the spatial pose with the industrial robot joint reaching the singularity point, and the deviation of the industrial robot movement posture; the industrial robot spatial pose model data represents the standard spatial pose three-dimensional entity model parameters set for different types of abnormal spatial poses of the industrial robot.

[0033] S42. Use the ARAP search algorithm to match the real-time spatial pose model data m of the industrial robot A with the industrial robot spatial pose model data u in the industrial robot spatial pose model data set U b for pose model feature matching, and generate the real-time spatial pose abnormal state analysis data U of the industrial robot based on the pose model feature matching result fenxi ;

[0034] When the pose model features of m A and u b do not match successfully, it means that the current spatial movement posture of the industrial robot conforms to the safety standard posture, then output the real-time spatial pose abnormal state analysis data U of the industrial robot fenxi as normal, and directly end the current industrial robot pose adjustment operation at this time;

[0035] When the pose model features of m A and u b match successfully, it means that the current spatial movement posture of the industrial robot does not conform to the safety standard posture, then output the real-time spatial pose abnormal state analysis data U of the industrial robot fenxi as abnormal, and at this time output the text information of the type of abnormal spatial pose of the industrial robot corresponding to the industrial robot spatial pose model data u b .

[0036] The present invention conducts scientific analysis of the real-time spatial pose abnormal state of the industrial robot by combining the real-time spatial pose model parameters of the industrial robot with the ARAP search algorithm and the abnormal spatial pose model parameters of the industrial robot based on big data storage, achieving the effect of dynamically and scientifically identifying the spatial pose fault state of the industrial robot.

[0037] Preferably, when an abnormality occurs, the operation steps for matching the real-time spatial pose abnormality adjustment parameters of the industrial robot and generating the real-time spatial pose abnormality adjustment data of the industrial robot according to the real-time spatial pose abnormality state analysis data of the industrial robot and the industrial robot abnormal spatial pose adjustment data are as follows:

[0038] S51. When the real-time spatial pose abnormality state analysis data U of the industrial robot fenxi is abnormal, a set K=(k1,…,k b ,…,k λ ) of industrial robot abnormal spatial pose adjustment data is established, where k b represents the industrial robot abnormal spatial pose adjustment data corresponding to the bth type of industrial robot abnormal spatial pose, and the industrial robot abnormal spatial pose adjustment data represents the industrial robot standard spatial pose abnormal correction motion trajectory coordinate parameters set for different industrial robot abnormal spatial pose types;

[0039] S52. The BERT language model algorithm is used to perform character matching of the industrial robot abnormal spatial pose types of the real-time spatial pose abnormality state analysis data U of the industrial robot fenxi with the industrial robot abnormal spatial pose adjustment data k b in the industrial robot abnormal spatial pose adjustment data set K, search for the industrial robot abnormal spatial pose adjustment data k fenxi corresponding to the real-time spatial pose abnormality state analysis data U of the industrial robot b , and generate the real-time spatial pose abnormality adjustment data k shishi through data identification.

[0040] The present invention intelligently matches the real-time spatial pose abnormality adjustment parameters of the industrial robot by combining the real-time spatial pose abnormality state analysis parameters of the industrial robot with the BERT language model algorithm and the industrial robot abnormal spatial pose adjustment parameters stored in the standard, achieving the effect of dynamically constructing the industrial robot spatial pose fault calibration control parameters.

[0041] Preferably, the operation steps for performing the industrial robot spatial pose adjustment operation according to the real-time spatial pose abnormality adjustment data of the industrial robot are as follows:

[0042] S61. The industrial robot control terminal controls the industrial robot to perform the industrial robot spatial pose adjustment operation according to the spatial pose abnormality adjustment parameters corresponding to the real-time spatial pose abnormality adjustment data k shishi of the industrial robot.

[0043] The present invention adjusts parameters according to the real-time spatial pose anomaly of an industrial robot and combines with the control end of the industrial robot to autonomously and efficiently execute the industrial robot spatial pose adjustment operation, achieving the effect of improving the response speed of the industrial robot spatial pose adjustment operation.

[0044] Preferably, the operation steps of constructing the real-time spatial pose adjustment monitoring data of the industrial robot and executing the industrial robot spatial pose adjustment feedback operation are as follows:

[0045] S71. Combine the real-time pose image data A of the industrial robot, the real-time spatial pose model data m A , the real-time spatial pose anomaly state analysis data U fenxi , the real-time spatial pose anomaly adjustment data k shishi to construct the real-time spatial pose adjustment monitoring data O of the industrial robot, where O = (A, m A , U fenxi , k shishi );

[0046] S72. Transmit the real-time spatial pose adjustment monitoring data O of the industrial robot to the industrial robot supervision platform through the industrial Internet of Things for online transmission to execute the industrial robot spatial pose adjustment feedback operation.

[0047] The present invention scientifically constructs the real-time spatial pose adjustment monitoring data of the industrial robot based on the real-time spatial pose image, real-time spatial pose model, real-time spatial pose anomaly analysis result, and real-time spatial pose adjustment parameters of the industrial robot, and at the same time combines with the industrial robot supervision platform to efficiently and dynamically execute the industrial robot spatial pose adjustment feedback operation, achieving the effect of realizing the online feedback of the industrial robot pose adjustment result.

[0048] A multi-vision machine pose adjustment system based on binocular vision is used to implement the multi-vision machine pose adjustment method based on binocular vision. The system includes an industrial robot pose model construction module, an industrial robot pose monitoring module, and an industrial robot pose adjustment module;

[0049] The industrial robot pose model construction module includes an industrial robot real-time pose image preprocessing unit, an industrial robot real-time pose image preprocessing unit, an industrial robot spatial pose model storage unit, and an industrial robot real-time spatial pose model generation unit;

[0050] The real-time pose image preprocessing unit of the industrial robot collects the real-time pose image data of the industrial robot through a binocular vision imaging processing device; the real-time pose image preprocessing unit of the industrial robot performs preprocessing on the real-time pose image of the industrial robot based on the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot; the industrial robot spatial pose model storage unit is used to store industrial robot spatial pose model data; the real-time spatial pose model generation unit of the industrial robot constructs a three-dimensional model of the real-time spatial pose of the industrial robot according to the standard real-time pose image data of the industrial robot and the industrial robot spatial pose model data to generate real-time spatial pose model data of the industrial robot;

[0051] The industrial robot pose monitoring module includes an industrial robot abnormal spatial pose model storage unit, an industrial robot real-time spatial pose abnormal state analysis unit, an industrial robot abnormal spatial pose adjustment parameter storage unit, and an industrial robot real-time spatial pose abnormal adjustment parameter matching unit;

[0052] The industrial robot abnormal spatial pose model storage unit is used to store industrial robot abnormal spatial pose model data; the industrial robot real-time spatial pose abnormal state analysis unit performs an analysis process on the real-time spatial pose abnormal state of the industrial robot based on the real-time spatial pose model data of the industrial robot and the industrial robot abnormal spatial pose model data to generate real-time spatial pose abnormal state analysis data of the industrial robot; the industrial robot abnormal spatial pose adjustment parameter storage unit is used to store industrial robot abnormal spatial pose adjustment data; the industrial robot real-time spatial pose abnormal adjustment parameter matching unit performs a matching process on the real-time spatial pose abnormal adjustment parameters of the industrial robot according to the real-time spatial pose abnormal state analysis data of the industrial robot and the industrial robot abnormal spatial pose adjustment data to generate real-time spatial pose abnormal adjustment data of the industrial robot;

[0053] The industrial robot pose adjustment module includes an industrial robot real-time spatial pose abnormal adjustment operation execution unit and an industrial robot real-time spatial pose abnormal adjustment feedback unit;

[0054] The industrial robot real-time spatial pose abnormal adjustment operation execution unit performs an industrial robot spatial pose adjustment operation in combination with the industrial robot control end according to the real-time spatial pose abnormal adjustment data of the industrial robot; the industrial robot real-time spatial pose abnormal adjustment feedback unit is used to construct real-time spatial pose adjustment monitoring data of the industrial robot and perform an industrial robot spatial pose adjustment feedback operation in combination with the industrial robot supervision platform.

[0055] (III) Beneficial effects

[0056] The present invention provides a multi-vision machine pose adjustment system and method based on binocular vision, having the following beneficial effects:

[0057] First, the binocular vision imaging processing device dynamically collects the real-time pose image parameters of the industrial robot, realizing the efficient collection of the real-time pose image parameters of the industrial robot based on binocular machine vision; according to the real-time pose image parameters of the industrial robot, combined with the data noise reduction algorithm, preprocessing the noise reduction of the real-time pose image of the industrial robot, realizing the accurate collection of the real-time pose image parameters of the industrial robot, and improving the collection accuracy of the real-time pose image of the industrial robot; according to the standard real-time pose image parameters of the industrial robot collected based on binocular machine vision, combined with the intelligent recognition algorithm and the scientifically preset industrial robot spatial pose model parameters, intelligently constructing the three-dimensional model of the real-time spatial pose of the industrial robot, realizing the dynamic digital construction of the industrial robot spatial pose model, and improving the efficiency and accuracy of the real-time spatial pose adjustment of the industrial robot.

[0058] Second, through the combination of the real-time spatial pose model parameters of the industrial robot, the intelligent search algorithm, and the abnormal spatial pose model parameters of the industrial robot stored based on big data, scientifically analyzing the abnormal state of the real-time spatial pose of the industrial robot, realizing the dynamic scientific identification of the fault state of the industrial robot spatial pose, and improving the real-time performance and accuracy of the industrial robot spatial pose adjustment; according to the analysis parameters of the abnormal state of the real-time spatial pose of the industrial robot, combined with the intelligent search algorithm and the standard stored abnormal spatial pose adjustment parameters of the industrial robot, intelligently matching the abnormal adjustment parameters of the real-time spatial pose of the industrial robot, realizing the dynamic construction of the industrial robot spatial pose fault calibration control parameters, and improving the safety and reliability of the industrial robot spatial pose adjustment.

[0059] Third, by autonomously and efficiently executing the industrial robot spatial pose adjustment operation according to the abnormal adjustment parameters of the real-time spatial pose of the industrial robot combined with the industrial robot control end, improving the response speed of the industrial robot spatial pose adjustment operation; scientifically constructing the real-time spatial pose adjustment monitoring data of the industrial robot based on the real-time spatial pose image, real-time spatial pose model, real-time spatial pose abnormal analysis result, and real-time spatial pose adjustment parameters of the industrial robot, and at the same time, combined with the industrial robot supervision platform, efficiently and dynamically executing the industrial robot spatial pose adjustment feedback operation, realizing the online feedback of the industrial robot pose adjustment result, and improving the digitization of the industrial robot pose adjustment. Description of the Drawings

[0060] Figure 1 It is a schematic diagram of the modules of a multi-vision machine pose adjustment system based on binocular vision provided by the present invention;

[0061] Figure 2 It is a flowchart of a multi-vision machine pose adjustment method based on binocular vision provided by the present invention. Specific Embodiments

[0062] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0063] The embodiments of a multi-vision robot pose adjustment system and method based on binocular vision are as follows:

[0064] Embodiment 1:

[0065] Please refer to Figure 1 - Figure 2 , a multi-vision robot pose adjustment method based on binocular vision, the method includes the following steps:

[0066] S1. Collect real-time pose image data of the industrial robot;

[0067] S2. Perform preprocessing on the real-time pose image of the industrial robot based on the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot;

[0068] S3. Perform processing on the construction of the three-dimensional model of the real-time spatial pose of the industrial robot according to the standard real-time pose image data of the industrial robot and the spatial pose model data of the industrial robot to generate real-time spatial pose model data of the industrial robot;

[0069] S4. Perform analysis processing on the abnormal state of the real-time spatial pose of the industrial robot based on the real-time spatial pose model data of the industrial robot and the abnormal spatial pose model data of the industrial robot to generate analysis data on the abnormal state of the real-time spatial pose of the industrial robot; when it is normal, end the current industrial robot pose adjustment operation;

[0070] S5. When it is abnormal, perform matching processing on the abnormal adjustment parameters of the real-time spatial pose of the industrial robot according to the analysis data on the abnormal state of the real-time spatial pose of the industrial robot and the abnormal spatial pose adjustment data of the industrial robot to generate abnormal adjustment data of the real-time spatial pose of the industrial robot;

[0071] S6. Perform the industrial robot spatial pose adjustment operation according to the abnormal adjustment data of the real-time spatial pose of the industrial robot;

[0072] S7. Construct the monitoring data of the real-time spatial pose adjustment of the industrial robot and perform the industrial robot spatial pose adjustment feedback operation.

[0073] Furthermore, please refer to Figure 1 -Figure 2 , the operation steps for collecting real-time pose image data of an industrial robot are as follows:

[0074] S11. Simultaneously collect spatial pose image information of the operating state of the industrial robot from two different spatial positions through a binocular vision imaging processing device, and generate real-time pose image data of the industrial robot A = (a1, a2), where a1 represents the real-time pose image data from the first perspective of the industrial robot, and a2 represents the real-time pose image data from the second perspective of the industrial robot. The binocular vision imaging processing device includes any one of the BASLER binocular stereo cameras and the Orbbec Gemini binocular structured light 3D cameras.

[0075] The operation steps for preprocessing the real-time pose image of the industrial robot based on the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot are as follows:

[0076] S21. Use the BM3D algorithm to perform noise reduction preprocessing on the real-time pose image data a1 from the first perspective of the industrial robot and the real-time pose image data a2 from the second perspective of the industrial robot in the real-time pose image data A of the industrial robot, and generate standard real-time pose image data of the industrial robot A' = (a'1, a'2), where a'1 represents the standard real-time pose image data from the first perspective of the industrial robot, and a'2 represents the standard real-time pose image data from the second perspective of the industrial robot.

[0077] The operation steps for constructing a three-dimensional model of the real-time spatial pose of the industrial robot based on the standard real-time pose image data of the industrial robot and the spatial pose model data of the industrial robot to generate real-time spatial pose model data of the industrial robot are as follows:

[0078] S31. Establish a set M of spatial pose model data of the industrial robot M = (m1,…,m g ,…,m κ ), g = 1, 2, 3,…, κ; where m g represents the spatial pose model data of the industrial robot corresponding to the g-th type of binocular vision industrial robot pose image, and κ represents the maximum value of the number of types of binocular vision industrial robot pose images; the binocular vision industrial robot pose image represents the image information of different types of operating postures of the industrial robot collected simultaneously from two different spatial positions through a binocular vision imaging processing device; the spatial pose model data of the industrial robot represents the standard three-dimensional entity model parameters of the spatial pose of the industrial robot set according to the characteristics of the binocular vision industrial robot pose image;

[0079] S32. Compare the real-time pose image data a'1 of the first view and the real-time pose image data a'2 of the second view of the standard industrial robot in the real-time pose image data A' of the standard industrial robot with the industrial robot spatial pose model data m in the industrial robot spatial pose model data set M g Perform industrial robot pose image feature matching to search for the industrial robot spatial pose model data m that matches the real-time pose image data a'1 of the first view and the real-time pose image data a'2 of the second view of the standard industrial robot g , and construct the real-time spatial pose model data m of the industrial robot A , and execute the specific operation steps for generating the real-time spatial pose model data m of the industrial robot A are as follows:

[0080] S321. Initialize the algorithm parameters, the population size N, and the maximum number of iterations T;

[0081] S322. Initialize the population, calculate the fitness, and determine the pose search pathfinder and the pose search follower;

[0082] S323. Update the position of the pose search pathfinder in the search space of the industrial robot spatial pose model data set M according to the position formula , where t represents the current iteration generation of the algorithm; represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model data set M after the t-th iteration, represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model data set M after the (t - 1)-th iteration, represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model data set M after the (t + 1)-th iteration, is the step size factor for the movement of the pose search pathfinder, obeys a uniform distribution within [0, 1];

[0083] S324. Update the position of the pose search follower in the search space of the industrial robot spatial pose model data set M according to the position formula , where represents the position of the pose search follower i in the search space of the industrial robot spatial pose model data set M after the t-th iteration, represents the position of the pose search follower i in the search space of the industrial robot spatial pose model data set M after the (t + 1)-th iteration; represents the position of other pose search followers j in the search space of the industrial robot spatial pose model data set M after the t-th iteration, and the position of the pose search follower The movement is not only related to the position of the pose search pathfinder but also affected by the positions of other pose search followers In the industrial robot spatial pose model data set M search space, R1 represents the position distance parameter between pose search followers, and R2 represents the position distance parameter between the pose search pathfinder and the pose search followers. R1 = ◇×r1, R2 = ∫×r2; ◇ represents the interaction coefficient between pose search followers, and ∫ represents the attraction coefficient of the pose search pathfinder to the pose search followers. Both ◇ and ∫ follow a uniform distribution in [1, 2]; r1 is the step factor for the movement of a pose search follower and other pose search followers, and r2 is the step factor for the movement of a pose search follower and the pose search pathfinder. Both r1 and r2 are random numbers within the range of [0, 1];

[0084] S325. Calculate the fitness values of the real-time pose image data a'1 of the first view of the standard industrial robot and the real-time pose image data a'2 of the second view of the standard industrial robot with all the industrial robot spatial pose model data m in the industrial robot spatial pose model data set M search space g and update the industrial robot spatial pose model data m with the maximum fitness value searched out for the real-time pose image data a'1 of the first view of the standard industrial robot and the real-time pose image data a'2 of the second view of the standard industrial robot g as the global optimal value;

[0085] S326. When the maximum iteration number T is satisfied, output the industrial robot spatial pose model data m that is the most matched with the real-time pose image data a'1 of the first view of the standard industrial robot and the real-time pose image data a'2 of the second view of the standard industrial robot searched out in step S325 g and generate the industrial robot real-time spatial pose model data m through data identification A .

[0086] Through the real-time pose image preprocessing unit of the industrial robot, the binocular vision imaging processing device is used to dynamically collect the real-time pose image parameters of the industrial robot, realizing the efficient acquisition of the real-time pose image parameters of the industrial robot based on binocular machine vision; the real-time pose image preprocessing unit of the industrial robot performs noise reduction preprocessing on the real-time pose image of the industrial robot according to the real-time pose image parameters of the industrial robot combined with the data noise reduction algorithm, realizing the accurate acquisition of the real-time pose image parameters of the industrial robot and improving the acquisition accuracy of the real-time pose image of the industrial robot; the real-time spatial pose model generation unit of the industrial robot intelligently constructs the three-dimensional model of the real-time spatial pose of the industrial robot according to the standard real-time pose image parameters of the industrial robot collected based on binocular machine vision combined with the intelligent recognition algorithm and the scientifically preset spatial pose model parameters of the industrial robot, realizing the dynamic digital construction of the spatial pose model of the industrial robot and improving the efficiency and accuracy of the real-time spatial pose adjustment of the industrial robot.

[0087] Further, please refer to Figure 1 - Figure 2 , perform real-time spatial pose abnormal state analysis and processing on the industrial robot based on the real-time spatial pose model data of the industrial robot and the abnormal spatial pose model data of the industrial robot, and generate real-time spatial pose abnormal state analysis data of the industrial robot; when it is normal, the operation steps to end the current industrial robot pose adjustment operation are as follows:

[0088] S41. Establish a set U of industrial robot spatial pose model data = (u1,..., u b ,..., u λ ), b = 1, 2, 3,..., λ; where u b represents the industrial robot spatial pose model data corresponding to the b-th type of abnormal spatial pose of the industrial robot, and λ represents the maximum value of the number of types of abnormal spatial poses of the industrial robot; the types of abnormal spatial poses of the industrial robot include the spatial pose with the industrial robot joint movement exceeding the limit, the spatial pose with the industrial robot joint reaching the singularity point, and the deviation of the industrial robot movement posture; the industrial robot spatial pose model data represents the standard spatial pose three-dimensional entity model parameters set for different types of abnormal spatial poses of the industrial robot.

[0089] S42. Use the ARAP search algorithm to perform pose model feature matching between the real-time spatial pose model data m A of the industrial robot and the industrial robot spatial pose model data u b in the industrial robot spatial pose model data set U, and generate real-time spatial pose abnormal state analysis data U fenxi according to the pose model feature matching result;

[0090] When m A and u bIf the pose model feature matching fails, it indicates that the current spatial motion pose of the industrial robot conforms to the safety standard pose, and then output the analysis data U of the abnormal state of the real-time spatial pose of the industrial robot. fenxi If it is normal, directly end the current pose adjustment operation of the industrial robot at this time;

[0091] When m A and u b If the pose model feature matching is successful, it indicates that the current spatial motion pose of the industrial robot does not conform to the safety standard pose, and then output the analysis data U of the abnormal state of the real-time spatial pose of the industrial robot. fenxi If it is abnormal, output the spatial pose model data u of the industrial robot at this time. b The corresponding text information of the abnormal spatial pose type of the industrial robot.

[0092] When it is abnormal, perform the matching process of the real-time spatial pose abnormal adjustment parameters of the industrial robot according to the analysis data of the real-time spatial pose abnormal state of the industrial robot and the abnormal spatial pose adjustment data of the industrial robot. The operation steps for generating the real-time spatial pose abnormal adjustment data of the industrial robot are as follows:

[0093] S51. When the analysis data U of the real-time spatial pose abnormal state of the industrial robot fenxi is abnormal, establish a set K of abnormal spatial pose adjustment data of the industrial robot = (k1,..., k b ,..., k λ ), where k b represents the abnormal spatial pose adjustment data of the industrial robot corresponding to the b-th type of abnormal spatial pose of the industrial robot. The abnormal spatial pose adjustment data of the industrial robot represents the coordinate parameters of the standard spatial pose abnormal correction motion trajectory set for different types of abnormal spatial poses of the industrial robot;

[0094] S52. Use the BERT language model algorithm to match the text information of the abnormal spatial pose type of the industrial robot corresponding to the analysis data U of the real-time spatial pose abnormal state of the industrial robot fenxi with the abnormal spatial pose adjustment data k in the set K of abnormal spatial pose adjustment data of the industrial robot b to perform character matching of the abnormal spatial pose type of the industrial robot, search for the abnormal spatial pose adjustment data k of the industrial robot corresponding to the analysis data U of the real-time spatial pose abnormal state of the industrial robot fenxi , and generate the real-time spatial pose abnormal adjustment data k of the industrial robot through data identification b . shishi .

[0095] Through the real-time spatial pose abnormal state analysis unit of the industrial robot, based on the real-time spatial pose model parameters of the industrial robot, combined with the intelligent search algorithm and the abnormal spatial pose model parameters of the industrial robot based on big data storage, scientific analysis of the real-time spatial pose abnormal state of the industrial robot is carried out, realizing the dynamic scientific identification of the spatial pose fault state of the industrial robot, and improving the real-time performance and accuracy of the spatial pose adjustment of the industrial robot; the real-time spatial pose abnormal adjustment parameter matching unit of the industrial robot, based on the real-time spatial pose abnormal state analysis parameters of the industrial robot, combined with the intelligent search algorithm and the abnormal spatial pose adjustment parameters of the industrial robot stored in the standard, conducts intelligent matching of the real-time spatial pose abnormal adjustment parameters of the industrial robot, realizing the dynamic construction of the spatial pose fault calibration control parameters of the industrial robot, and improving the safety and reliability of the spatial pose adjustment of the industrial robot.

[0096] Further, please refer to Figure 1 - Figure 2 According to the real-time spatial pose abnormal adjustment data of the industrial robot, the operation steps for performing the spatial pose adjustment operation of the industrial robot are as follows:

[0097] S61. The control end of the industrial robot controls the industrial robot to perform the spatial pose adjustment operation of the industrial robot according to the spatial pose abnormal adjustment parameter corresponding to the real-time spatial pose abnormal adjustment data k of the industrial robot shishi of the industrial robot.

[0098] The operation steps for constructing the real-time spatial pose adjustment monitoring data of the industrial robot and performing the spatial pose adjustment feedback operation of the industrial robot are as follows:

[0099] S71. Combine the real-time pose image data A of the industrial robot, the real-time spatial pose model data m A , the real-time spatial pose abnormal state analysis data U fenxi , and the real-time spatial pose abnormal adjustment data k shishi of the industrial robot to construct the real-time spatial pose adjustment monitoring data O of the industrial robot, where O = (A, m A , U fenxi , k shishi );

[0100] S72. Transmit the real-time spatial pose adjustment monitoring data O of the industrial robot to the industrial robot supervision platform through the industrial Internet of Things for online transmission to perform the spatial pose adjustment feedback operation of the industrial robot.

[0101] The operation execution unit for real-time spatial pose anomaly adjustment of an industrial robot autonomously and efficiently executes the spatial pose adjustment operation of the industrial robot based on the real-time spatial pose anomaly adjustment parameters of the industrial robot in combination with the control end of the industrial robot, improving the response speed of the spatial pose adjustment operation of the industrial robot; the real-time spatial pose anomaly adjustment feedback unit of the industrial robot scientifically constructs the monitoring data for the real-time spatial pose adjustment of the industrial robot based on the real-time spatial pose image, real-time spatial pose model, real-time spatial pose anomaly analysis result, and real-time spatial pose adjustment parameters of the industrial robot. At the same time, it efficiently and dynamically executes the spatial pose adjustment feedback operation of the industrial robot in combination with the industrial robot supervision platform, realizes the online feedback of the pose adjustment result of the industrial robot, and improves the digitization of the pose adjustment of the industrial robot.

[0102] Embodiment 2:

[0103] Please refer to Figure 1 - Figure 2 , a multi-vision machine pose adjustment system based on binocular vision, used to implement a multi-vision machine pose adjustment method based on binocular vision. The system includes an industrial robot pose model construction module, an industrial robot pose monitoring module, and an industrial robot pose adjustment module;

[0104] The industrial robot pose model construction module includes an industrial robot real-time pose image preprocessing unit, an industrial robot real-time pose image preprocessing unit, an industrial robot spatial pose model storage unit, and an industrial robot real-time spatial pose model generation unit;

[0105] The industrial robot real-time pose image preprocessing unit collects industrial robot real-time pose image data through binocular vision imaging processing equipment; the industrial robot real-time pose image preprocessing unit performs preprocessing on the real-time pose image of the industrial robot based on the industrial robot real-time pose image data to generate standard industrial robot real-time pose image data; the industrial robot spatial pose model storage unit is used to store industrial robot spatial pose model data; the industrial robot real-time spatial pose model generation unit performs processing on the real-time spatial pose three-dimensional model construction of the industrial robot according to the standard industrial robot real-time pose image data and the industrial robot spatial pose model data to generate industrial robot real-time spatial pose model data;

[0106] The industrial robot pose monitoring module includes an industrial robot abnormal spatial pose model storage unit, an industrial robot real-time spatial pose abnormal state analysis unit, an industrial robot abnormal spatial pose adjustment parameter storage unit, and an industrial robot real-time spatial pose abnormal adjustment parameter matching unit;

[0107] The storage unit for the abnormal spatial pose model of the industrial robot is used to store the data of the abnormal spatial pose model of the industrial robot; the real-time spatial pose abnormal state analysis unit of the industrial robot performs the analysis and processing of the real-time spatial pose abnormal state of the industrial robot based on the real-time spatial pose model data of the industrial robot and the abnormal spatial pose model data of the industrial robot, and generates the analysis data of the real-time spatial pose abnormal state of the industrial robot; the storage unit for the adjustment parameters of the abnormal spatial pose of the industrial robot is used to store the adjustment data of the abnormal spatial pose of the industrial robot; the matching unit for the adjustment parameters of the real-time spatial pose abnormal of the industrial robot performs the matching process of the adjustment parameters of the real-time spatial pose abnormal of the industrial robot according to the analysis data of the real-time spatial pose abnormal state of the industrial robot and the adjustment data of the abnormal spatial pose of the industrial robot, and generates the adjustment data of the real-time spatial pose abnormal of the industrial robot.

[0108] The pose adjustment module of the industrial robot includes the execution unit for the abnormal adjustment operation of the real-time spatial pose of the industrial robot and the feedback unit for the abnormal adjustment of the real-time spatial pose of the industrial robot.

[0109] The execution unit for the abnormal adjustment operation of the real-time spatial pose of the industrial robot performs the adjustment operation of the spatial pose of the industrial robot according to the adjustment data of the real-time spatial pose abnormal of the industrial robot in combination with the control end of the industrial robot; the feedback unit for the abnormal adjustment of the real-time spatial pose of the industrial robot is used to construct the monitoring data of the real-time spatial pose adjustment of the industrial robot and perform the feedback operation of the spatial pose adjustment of the industrial robot in combination with the supervision platform of the industrial robot.

[0110] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirits of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-vision machine pose adjustment method based on binocular vision, characterized in that, The method includes the following steps: S1. Collect real-time pose image data of the industrial robot; S2. Preprocess the real-time pose image of the industrial robot based on the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot; S3. Perform processing for constructing a three-dimensional model of the real-time spatial pose of the industrial robot according to the standard real-time pose image data of the industrial robot and the spatial pose model data of the industrial robot to generate real-time spatial pose model data of the industrial robot; S4. Perform analysis processing on the abnormal state of the real-time spatial pose of the industrial robot based on the real-time spatial pose model data of the industrial robot and the abnormal spatial pose model data of the industrial robot to generate analysis data on the abnormal state of the real-time spatial pose of the industrial robot; when it is normal, end the current industrial robot pose adjustment operation; S5. When it is abnormal, perform processing for matching abnormal adjustment parameters of the real-time spatial pose of the industrial robot according to the analysis data on the abnormal state of the real-time spatial pose of the industrial robot and the abnormal spatial pose adjustment data of the industrial robot to generate abnormal adjustment data of the real-time spatial pose of the industrial robot; S6. Execute the industrial robot spatial pose adjustment operation according to the abnormal adjustment data of the real-time spatial pose of the industrial robot; S7. Construct real-time spatial pose adjustment monitoring data of the industrial robot and execute the industrial robot spatial pose adjustment feedback operation.

2. A multi-vision machine pose adjustment method based on binocular vision according to claim 1, characterized in that: The S1 includes the following steps: S11. Simultaneously collect spatial pose image information of the operating state of the industrial robot from two different spatial positions through a binocular vision imaging processing device, and generate real-time pose image data of the industrial robot A=(a1, a2), where a1 represents the real-time pose image data of the first view of the industrial robot, and a2 represents the real-time pose image data of the second view of the industrial robot.

3. A multi-vision robot pose adjustment method based on binocular vision according to claim 2, characterized in that: The S2 includes the following steps: S21. Use the BM3D algorithm to perform noise reduction preprocessing on the real-time pose image of the industrial robot for a1 and a2 in the real-time pose image data A of the industrial robot, and generate standard real-time pose image data of the industrial robot A'=(a'1, a'2), where a'1 represents the standard real-time pose image data of the first view of the industrial robot, and a'2 represents the standard real-time pose image data of the second view of the industrial robot.

4. A multi-vision machine pose adjustment method based on binocular vision according to claim 3, characterized in that: The S3 includes the following steps: S31. Establish the industrial robot spatial pose model data set M = (m1, …, m g , …, m κ ), where g = 1, 2, 3, …, κ; where m g represents the industrial robot spatial pose model data corresponding to the g-th binocular vision industrial robot pose image type, and κ represents the maximum value of the number of binocular vision industrial robot pose image types; S32. Match the a'1 and a'2 in the A' with the m in the M g Perform industrial robot pose image feature matching to search for the m that matches the a'1 and a'2 g , and construct the industrial robot real-time spatial pose model data m A , and execute the specific operation steps for generating the industrial robot real-time spatial pose model data m A as follows: S321. Initialize the algorithm parameters, the population size N, and the maximum number of iterations T; S322. Initialize the population, calculate the fitness, and determine the pose search pathfinder and the pose search follower; S323. Update the position of the pose search pathfinder in the M search space; S324. Update the position of the pose search follower in the M search space; S325. Calculate the fitness values of the a'1 and a'2 with all the m in the M search space, and update and search for the m with the maximum fitness values of the a'1 and a'2 g as the global optimal value; g ​ S326. When the maximum number of iterations T is satisfied, output the m that best matches the a'1 and the a'2 searched in step S325 g And generate the industrial robot real-time spatial pose model data m through data identification A .

5. A multi-vision robot pose adjustment method based on binocular vision according to claim 4, characterized in that: The S4 includes the following steps: S41. Establish a data set U = (u1, …, u b , …, u λ ) of the spatial pose models of industrial robots, where b = 1, 2, 3, …, λ; where u b represents the spatial pose model data of the industrial robot corresponding to the b-th type of abnormal spatial pose of the industrial robot, and λ represents the maximum value of the number of types of abnormal spatial poses of the industrial robot; S42. Use the ARAP search algorithm to match the m A with the u in the U b for pose model feature matching, and generate industrial robot real-time spatial pose abnormal state analysis data U based on the pose model feature matching results fenxi ; When m A and u b do not match successfully in terms of pose model features, then output that the U fenxi is normal, and directly end the current industrial robot pose adjustment operation at this time; When m A successfully matches the pose model features with u b output the U fenxi as abnormal, and at this time output the u b corresponding industrial robot abnormal spatial pose type text information.

6. A method for adjusting the pose of a multi-vision machine based on binocular vision according to claim 5, characterized in that: The S5 includes the following steps: S51. When the U fenxi is abnormal, establish an industrial robot abnormal spatial pose adjustment data set K = (k1, …, k b , …, k λ ), where k b represents the industrial robot abnormal spatial pose adjustment data corresponding to the b-th type of industrial robot abnormal spatial pose; S52. Use the BERT language model algorithm to process the U fenxi corresponding industrial robot abnormal spatial pose type text information and the k in the K b to perform character matching of the industrial robot abnormal spatial pose type, and search for the U fenxi corresponding k b , and generate real-time spatial pose abnormal adjustment data k of the industrial robot through data identification shishi .

7. A multi-vision machine pose adjustment method based on binocular vision according to claim 6, characterized in that: The S6 includes the following steps: S61. The control end of the industrial robot controls the industrial robot to perform the industrial robot spatial pose adjustment operation according to the corresponding spatial pose abnormal adjustment parameter of the k shishi ​ 8. A method for adjusting the pose of a multi-vision machine based on binocular vision according to claim 7, characterized in that: The S7 includes the following steps: S71. Combine the real-time pose image data A of the industrial robot, the m A , the U fenxi , the k shishi to construct the real-time spatial pose adjustment monitoring data O of the industrial robot; S72. Transmit the O online through the industrial Internet of Things to the industrial robot supervision platform to execute the industrial robot spatial pose adjustment feedback operation.

9. A multi-vision machine pose adjustment system based on binocular vision, which is used to implement a multi-vision machine pose adjustment method based on binocular vision according to any one of claims 1-8, characterized in that: The system includes an industrial robot pose model construction module, an industrial robot pose monitoring module, and an industrial robot pose adjustment module.

Images