A multi-vision machine pose adjustment system and method based on binocular vision

Abstract

The application relates to the technical field of machine position control, and discloses a multi-vision machine pose adjustment system and method based on binocular vision, which comprises an industrial robot pose model construction module, an industrial robot pose monitoring module and an industrial robot pose adjustment module; according to standard industrial robot real-time pose image parameters collected based on binocular machine vision, intelligent recognition algorithms and scientifically preset industrial robot space pose model parameters are combined to intelligently construct an industrial robot real-time space pose three-dimensional model, so that dynamic digital construction of the industrial robot space pose model is realized; according to industrial robot real-time space pose abnormal state analysis parameters, intelligent search algorithms and standard stored industrial robot abnormal space pose adjustment parameters are combined to intelligently match industrial robot real-time space pose abnormal adjustment parameters, so that dynamic construction of industrial robot space pose fault calibration control parameters is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine position control, in particular to a multi-vision machine pose adjustment system and method based on binocular vision. BACKGROUND

[0002] Industrial robots have programmability, strong universality, high repeatability, and human-machine collaboration. Through the application of industrial robots, enterprises can realize the automation of production lines, improve production efficiency, reduce production costs, and meet personalized and diversified production needs. Binocular stereo vision is an important form of machine vision. It is based on the principle of parallax and uses imaging devices to obtain two images of the measured object from different positions. By calculating the positional deviation between corresponding points in the images, the three-dimensional geometric information of the object can be obtained. Fusing the images obtained by both eyes and observing the differences between them allows us to obtain a clear sense of depth and establish a correspondence between features. The same physical point in space is mapped to different image points in different images. Existing industrial robots control the pose state during operation through a set motion trajectory. The industrial robot cannot autonomously and efficiently determine its current abnormal pose state during operation, nor can it accurately and intelligently adjust the current abnormal pose state, which reduces the reliability and safety of industrial robot operation control.

[0003] Chinese invention patent CN111427370B discloses a Gmapping mapping method for mobile robots based on sparse pose adjustment. The technical solutions of initializing particle pose and distribution, scan matching, calculating the target distribution of the sampling position, calculating the Gaussian approximation, updating the weight of the i-th particle, and updating the particle ground pose map are used to realize the pose adjustment of the mobile robot. However, the above technical solutions cannot achieve intelligent adjustment of the abnormal pose state of the mobile robot during operation. SUMMARY

[0004] (I) Technical problems solved

[0005] To solve the problem that the industrial robot cannot autonomously and efficiently determine its current abnormal pose state during operation, nor can it accurately and intelligently adjust the current abnormal pose state, which reduces the reliability and safety of industrial robot operation control, the above industrial robot space pose model is dynamically constructed, the abnormal state of the industrial robot space pose is scientifically analyzed, the abnormal adjustment parameters of the industrial robot space pose are accurately matched, the abnormal state of the industrial robot space pose is intelligently adjusted, and the reliability and intelligence of the industrial robot operation control are improved.

[0006] (II) Technical solutions

[0007] The application is implemented by the following technical solutions: a multi-vision machine pose adjustment method based on binocular vision, comprising the following steps:

[0008] S1, collecting industrial robot real-time pose image data;

[0009] S2, performing real-time pose image preprocessing of the industrial robot according to the industrial robot real-time pose image data, to generate standard industrial robot real-time pose image data;

[0010] S3, performing real-time spatial pose three-dimensional model construction processing of the industrial robot according to the standard industrial robot real-time pose image data and the industrial robot spatial pose model data, to generate industrial robot real-time spatial pose model data;

[0011] S4, performing real-time spatial pose abnormal state analysis processing of the industrial robot based on the industrial robot real-time spatial pose model data and the industrial robot abnormal spatial pose model data, to generate industrial robot real-time spatial pose abnormal state analysis data; when normal, ending the industrial robot pose adjustment operation;

[0012] S5, when abnormal, performing real-time spatial pose abnormal adjustment parameter matching processing of the industrial robot according to the industrial robot real-time spatial pose abnormal state analysis data and the industrial robot abnormal spatial pose adjustment data, to generate industrial robot real-time spatial pose abnormal adjustment data;

[0013] S6, performing industrial robot spatial pose adjustment operation according to the industrial robot real-time spatial pose abnormal adjustment data;

[0014] S7, constructing industrial robot real-time spatial pose adjustment monitoring data and performing industrial robot spatial pose adjustment feedback operation.

[0015] Preferably, the operation steps of collecting industrial robot real-time pose image data are as follows:

[0016] S11, simultaneously collecting spatial pose image information of the industrial robot running state from two different spatial positions by a binocular vision imaging processing device, and generating industrial robot real-time pose image data A=(a1, a2), wherein a1 represents industrial robot first-view real-time pose image data, and a2 represents industrial robot second-view real-time pose image data, the binocular vision imaging processing device comprising any one of BASLER binocular stereo camera and Obitonguang Gemini binocular structured light 3D camera.

[0017] The application achieves the effect of high-efficiency collection of real-time pose image parameters of an industrial robot based on binocular machine vision by dynamically collecting real-time pose image parameters of the industrial robot through a binocular vision imaging processing device.

[0018] Preferably, real-time pose image preprocessing of the industrial robot is performed according to the real-time pose image data of the industrial robot, and the operation steps of generating standard real-time pose image data of the industrial robot are as follows:

[0019] S21, real-time pose image denoising preprocessing of the industrial robot is performed on the first-view real-time pose image data a1 and the second-view real-time pose image data a2 of the industrial robot in the real-time pose image data A of the industrial robot by using the BM3D algorithm, and standard real-time pose image data A'=(a'1, a'2) of the industrial robot is generated, wherein a'1 represents standard first-view real-time pose image data of the industrial robot, and a'2 represents standard second-view real-time pose image data of the industrial robot.

[0020] The application achieves the effect of accurate collection of real-time pose image parameters of an industrial robot by performing real-time pose image denoising preprocessing of the industrial robot according to real-time pose image parameters of the industrial robot and a BM3D algorithm.

[0021] Preferably, real-time spatial pose three-dimensional model construction processing of the industrial robot is performed according to the standard real-time pose image data of the industrial robot and spatial pose model data of the industrial robot, and the operation steps of generating real-time spatial pose model data of the industrial robot are as follows:

[0022] S31, a set of spatial pose model data M=(m1,…,m g ,…,m κ ) of the industrial robot is established, g=1, 2, 3,…,κ; wherein m g represents spatial pose model data of the industrial robot corresponding to the gth type of binocular vision industrial robot pose image, and κ represents the maximum value of the number of types of binocular vision industrial robot pose images; the binocular vision industrial robot pose image represents image information of different types of running postures of the industrial robot collected online from two different spatial positions at the same time through a binocular vision imaging processing device; and the spatial pose model data of the industrial robot represents standard spatial pose three-dimensional entity model parameters set for features of the binocular vision industrial robot pose image.

[0023] S32, the standard first-view real-time pose image data a'1 and the standard second-view real-time pose image data a'2 in the standard real-time pose image data A' of the industrial robot are matched with the spatial pose model data mg Perform industrial robot pose image feature matching to search for the industrial robot spatial pose model data m that matches the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2. g And construct the real-time spatial pose model data m of the industrial robot. A Execute the generation of the real-time spatial pose model data m of the industrial robot. A The specific operating steps are as follows:

[0024] S321. Initialize the algorithm parameters: population size N, maximum number of iterations T;

[0025] S322. Initialize the population, calculate fitness, and determine the pose search pathfinders and pose search followers;

[0026] S323, According to the position formula The position of the pose search pathfinder is updated in the search space of the industrial robot spatial pose model data set M, where t represents the current iteration number of the algorithm; This represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model dataset M after the t-th iteration. This represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model dataset M after the (t-1)th iteration. This represents the position of the pose search pathfinder ω in the search space of the industrial robot spatial pose model dataset M after the (t+1)th iteration. The step size factor for the pathfinder's movement in pose search. It follows a uniform distribution in [0,1].

[0027] S324, According to the position formula Update the position of the pose search follower in the search space of the industrial robot spatial pose model data set M, where This represents the position of pose search follower i in the search space of the industrial robot spatial pose model dataset M after the t-th iteration. This represents the position of pose search follower i in the search space of the industrial robot spatial pose model data set M after the (t+1)th iteration; This represents the position of other pose search followers j in the search space of the industrial robot spatial pose model data set M after the t-th iteration. Movement is not only related to pose search Pathfinder location Related, and influenced by the position of other poses search followers The influence of R1, where R1 represents the positional distance parameter between pose search followers in the search space of the industrial robot spatial pose model data set M, and R2 represents the positional distance parameter between pose search pathfinder and pose search followers in the search space of the industrial robot spatial pose model data set M, R1 = ◇ × r1, R2 = ∫ × r2; ◇ represents the interaction coefficient between pose search followers, and ∫ represents the attraction coefficient of pose search pathfinder to pose search followers. Both ◇ and ∫ follow a uniform distribution in [1,2]. r1 is the step size factor for the movement of pose search followers and other pose search followers, and r2 is the step size factor for the movement of pose search followers and pose search pathfinder. Both r1 and r2 are random numbers in the range [0,1].

[0028] S325. Calculate the real-time pose image data a'1 of the standard industrial robot from the first perspective and the real-time pose image data a'2 of the standard industrial robot from the second perspective, and search the space of all the spatial pose model data m of the industrial robot in the spatial pose model data set M of the industrial robot. g The fitness value is used to update the industrial robot spatial pose model data m that has the largest fitness value with the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2. g Global optimum;

[0029] S326. When the maximum number of iterations T is satisfied, output the industrial robot spatial pose model data m that best matches the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2 found in step S325. g And through data identification, real-time spatial pose model data of industrial robots is generated. A .

[0030] This invention achieves the effect of dynamic digital construction of industrial robot spatial pose model by intelligently constructing a three-dimensional model of industrial robot real-time spatial pose based on the real-time pose image parameters of standard industrial robot acquired by binocular machine vision, combined with the AI ​​Pathfinder optimization algorithm and scientifically preset industrial robot spatial pose model parameters.

[0031] Preferably, based on the real-time spatial pose model data and the abnormal spatial pose model data of the industrial robot, the real-time spatial pose anomaly state analysis of the industrial robot is performed to generate real-time spatial pose anomaly state analysis data of the industrial robot; when normal, the operation steps to end the current industrial robot pose adjustment operation are as follows:

[0032] S41. Establish the spatial pose model data set of the industrial robot U = (u1, ..., u1)b ,…,u λ ), b = 1, 2, 3, …, λ; wherein u b represents the industrial robot space pose model data corresponding to the bth industrial robot abnormal space pose type, λ represents the maximum value of the number of industrial robot abnormal space pose types; the industrial robot abnormal space pose type includes industrial robot joint motion over-limit space pose, industrial robot joint reaching singular point space pose and industrial robot motion posture deviation; the industrial robot space pose model data represents the standard space pose three-dimensional entity model parameters set for different industrial robot abnormal space pose types;

[0033] S42, the industrial robot real-time space pose model data m A is searched by using the ARAP search algorithm, and is matched with the industrial robot space pose model data u b in the industrial robot space pose model data set U, and the industrial robot real-time space pose abnormal state analysis data U fenxi is generated according to the pose model feature matching result.

[0034] When m A and u b are not matched, it indicates that the current space motion posture of the industrial robot conforms to the safety standard posture, and the industrial robot real-time space pose abnormal state analysis data U fenxi is normal, and the industrial robot pose adjustment operation is directly ended at this time.

[0035] When m A and u b are matched, it indicates that the current space motion posture of the industrial robot does not conform to the safety standard posture, and the industrial robot real-time space pose abnormal state analysis data U fenxi is abnormal, and the industrial robot space pose model data u b corresponding to the industrial robot abnormal space pose type text information is output.

[0036] The present application realizes the effect of dynamic scientific identification of industrial robot space pose fault state by scientifically analyzing the industrial robot real-time space pose abnormal state based on the industrial robot real-time space pose model parameters combined with the ARAP search algorithm and the industrial robot abnormal space pose model parameters based on big data storage.

[0037] Preferably, when the anomaly, according to the industrial robot real-time spatial pose anomaly state analysis data and industrial robot abnormal spatial pose adjustment data of the industrial robot real-time spatial pose anomaly adjustment parameter matching processing, the operation steps of generating industrial robot real-time spatial pose anomaly adjustment data are as follows:

[0038] S51, when the industrial robot real-time spatial pose anomaly state analysis data U fenxi is abnormal, the industrial robot abnormal spatial pose adjustment data set K=(k1,…,k b ,…,k λ ) is established, wherein k b indicates the industrial robot abnormal spatial pose adjustment data corresponding to the bth industrial robot abnormal spatial pose type, and the industrial robot abnormal spatial pose adjustment data indicates the industrial robot standard spatial pose anomaly correction motion trajectory coordinate parameter set for different industrial robot abnormal spatial pose types;

[0039] S52, the BERT language model algorithm is adopted to perform industrial robot abnormal spatial pose type character matching on the industrial robot real-time spatial pose anomaly state analysis data U fenxi and the industrial robot abnormal spatial pose adjustment data k b in the industrial robot abnormal spatial pose adjustment data set K corresponding to the industrial robot abnormal spatial pose type text information. fenxi The corresponding industrial robot abnormal spatial pose adjustment data k b is searched out, and the industrial robot real-time spatial pose anomaly adjustment data k shishi is generated through data identification.

[0040] The present application realizes the effect of dynamic construction of industrial robot spatial pose fault calibration control parameter by combining the industrial robot real-time spatial pose anomaly state analysis parameter with the BERT language model algorithm and the standard stored industrial robot abnormal spatial pose adjustment parameter to perform intelligent matching of the industrial robot real-time spatial pose anomaly adjustment parameter.

[0041] Preferably, the operation steps of performing industrial robot spatial pose adjustment operation according to the industrial robot real-time spatial pose anomaly adjustment data are as follows:

[0042] S61, the industrial robot control end controls the industrial robot to perform industrial robot spatial pose adjustment operation according to the industrial robot real-time spatial pose anomaly adjustment data k shishi corresponding to the spatial pose anomaly adjustment parameter.

[0043] The application combines real-time spatial pose abnormality adjustment parameters of industrial robots with autonomous and efficient execution of industrial robot spatial pose adjustment work by the control end of the industrial robot to improve the response speed of the industrial robot spatial pose adjustment work.

[0044] Preferably, the operation steps of constructing the industrial robot real-time spatial pose adjustment monitoring data and executing the industrial robot spatial pose adjustment feedback work are as follows:

[0045] S71, the industrial robot real-time pose image data A, the industrial robot real-time spatial pose model data m A , the industrial robot real-time spatial pose abnormality state analysis data U fenxi , the industrial robot real-time spatial pose abnormality adjustment data k shishi are combined to construct the industrial robot real-time spatial pose adjustment monitoring data O, wherein O=(A, m A ,U fenxi ,k shishi );

[0046] S72, the industrial robot real-time spatial pose adjustment monitoring data O is transmitted online to the industrial robot supervision platform through the industrial Internet of Things to execute the industrial robot spatial pose adjustment feedback work.

[0047] The application constructs the industrial robot real-time spatial pose adjustment monitoring data based on the real-time spatial pose image, the real-time spatial pose model, the real-time spatial pose abnormality analysis result and the real-time spatial pose adjustment parameter of the industrial robot, and simultaneously combines the efficient and dynamic execution of the industrial robot spatial pose adjustment feedback work by the industrial robot supervision platform to realize the online feedback of the industrial robot pose adjustment result.

[0048] A multi-vision machine pose adjustment system based on binocular vision is used to realize the multi-vision machine pose adjustment method based on binocular vision, and the system comprises an industrial robot pose model construction module, an industrial robot pose monitoring module and an industrial robot pose adjustment module.

[0049] The industrial robot pose model construction module comprises an industrial robot real-time pose image preprocessing unit, an industrial robot real-time pose image preprocessing unit, an industrial robot spatial pose model storage unit and an industrial robot real-time spatial pose model generation unit.

[0050] The industrial robot real-time pose image preprocessing unit collects industrial robot real-time pose image data through a binocular vision imaging processing device; the industrial robot real-time pose image preprocessing unit performs real-time pose image preprocessing of the industrial robot according to the industrial robot real-time pose image data, and generates standard industrial robot real-time pose image data; the industrial robot spatial pose model storage unit is used for storing industrial robot spatial pose model data; the industrial robot real-time spatial pose model generation unit performs real-time spatial pose three-dimensional model construction processing of the industrial robot according to the standard industrial robot real-time pose image data and the industrial robot spatial pose model data, and generates industrial robot real-time spatial pose model data;

[0051] The industrial robot pose monitoring module includes an industrial robot abnormal spatial pose model storage unit, an industrial robot real-time spatial pose abnormal state analysis unit, an industrial robot abnormal spatial pose adjustment parameter storage unit, and an industrial robot real-time spatial pose abnormal adjustment parameter matching unit.

[0052] The industrial robot abnormal spatial pose model storage unit is used for storing industrial robot abnormal spatial pose model data; the industrial robot real-time spatial pose abnormal state analysis unit performs real-time spatial pose abnormal state analysis processing of the industrial robot based on the industrial robot real-time spatial pose model data and the industrial robot abnormal spatial pose model data, and generates industrial robot real-time spatial pose abnormal state analysis data; the industrial robot abnormal spatial pose adjustment parameter storage unit is used for storing industrial robot abnormal spatial pose adjustment data; and the industrial robot real-time spatial pose abnormal adjustment parameter matching unit performs real-time spatial pose abnormal adjustment parameter matching processing of the industrial robot according to the industrial robot real-time spatial pose abnormal state analysis data and the industrial robot abnormal spatial pose adjustment data, and generates industrial robot real-time spatial pose abnormal adjustment data.

[0053] The industrial robot pose adjustment module includes an industrial robot real-time spatial pose abnormal adjustment operation execution unit and an industrial robot real-time spatial pose abnormal adjustment feedback unit.

[0054] The industrial robot real-time spatial pose abnormal adjustment operation execution unit performs industrial robot spatial pose adjustment operations in combination with an industrial robot control end according to the industrial robot real-time spatial pose abnormal adjustment data; and the industrial robot real-time spatial pose abnormal adjustment feedback unit is used for constructing industrial robot real-time spatial pose adjustment monitoring data and performing industrial robot spatial pose adjustment feedback operations in combination with an industrial robot supervision platform.

[0055] (Three) beneficial effects

[0056] The application provides a multi-vision machine pose adjustment system and method based on binocular vision.

[0057] I. The real-time pose image parameters of the industrial robot are dynamically collected by the binocular vision imaging processing device, the real-time pose image parameters of the industrial robot are efficiently collected based on binocular machine vision, the real-time pose image of the industrial robot is preprocessed by combining the data denoising algorithm with the real-time pose image parameters of the industrial robot, the real-time pose image parameters of the industrial robot are accurately collected, and the collection accuracy of the real-time pose image of the industrial robot is improved; the intelligent construction of the real-time spatial pose three-dimensional model of the industrial robot is performed by combining the standard real-time pose image parameters of the industrial robot collected based on binocular machine vision with the intelligent recognition algorithm and the scientifically preset spatial pose model parameters of the industrial robot, the dynamic digital construction of the spatial pose model of the industrial robot is realized, and the efficiency and accuracy of the real-time spatial pose adjustment of the industrial robot are improved.

[0058] II. The real-time spatial pose abnormal state of the industrial robot is scientifically analyzed by combining the intelligent search algorithm with the abnormal spatial pose model parameters of the industrial robot based on big data storage, the dynamic scientific identification of the spatial pose fault state of the industrial robot is realized, and the real-time performance and accuracy of the spatial pose adjustment of the industrial robot are improved; the intelligent matching of the real-time spatial pose abnormal adjustment parameters of the industrial robot is performed by combining the intelligent search algorithm with the standard stored abnormal spatial pose adjustment parameters of the industrial robot according to the real-time spatial pose abnormal state analysis parameters of the industrial robot, the dynamic construction of the fault calibration control parameters of the spatial pose of the industrial robot is realized, and the safety and reliability of the spatial pose adjustment of the industrial robot are improved.

[0059] III. The response speed of the spatial pose adjustment work of the industrial robot is improved by combining the real-time spatial pose abnormal adjustment parameters of the industrial robot with the self-efficient execution of the spatial pose adjustment work of the industrial robot by the control end of the industrial robot; the real-time spatial pose adjustment monitoring data of the industrial robot is scientifically constructed based on the real-time spatial pose image, the real-time spatial pose model, the real-time spatial pose abnormal analysis result and the real-time spatial pose adjustment parameter of the industrial robot, the spatial pose adjustment feedback work of the industrial robot is efficiently and dynamically executed by combining the industrial robot supervision platform, the online feedback of the pose adjustment result of the industrial robot is realized, and the digitization of the pose adjustment of the industrial robot is improved. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 A module schematic diagram of a multi-vision machine pose adjustment system based on binocular vision is provided for the application.

[0061] Figure 2 A flowchart of a multi-vision machine pose adjustment method based on binocular vision is provided for the application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be apparently and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0063] The embodiments of the multi-vision machine pose adjustment system and method based on binocular vision are as follows:

[0064] Embodiment 1

[0065] Please refer to Figure 1 - Figure 2 A multi-vision machine pose adjustment method based on binocular vision, the method comprising the following steps:

[0066] S1, collecting real-time pose image data of an industrial robot;

[0067] S2, performing real-time pose image preprocessing of the industrial robot according to the real-time pose image data of the industrial robot, to generate standard real-time pose image data of the industrial robot;

[0068] S3, performing real-time spatial pose three-dimensional model construction processing of the industrial robot according to the standard real-time pose image data of the industrial robot and spatial pose model data of the industrial robot, to generate real-time spatial pose model data of the industrial robot;

[0069] S4, performing real-time spatial pose abnormal state analysis processing of the industrial robot based on the real-time spatial pose model data of the industrial robot and abnormal spatial pose model data of the industrial robot, to generate real-time spatial pose abnormal state analysis data of the industrial robot; when normal, ending the current industrial robot pose adjustment work;

[0070] S5, when abnormal, performing real-time spatial pose abnormal adjustment parameter matching processing of the industrial robot according to the real-time spatial pose abnormal state analysis data of the industrial robot and abnormal spatial pose adjustment data of the industrial robot, to generate real-time spatial pose abnormal adjustment data of the industrial robot;

[0071] S6, performing spatial pose adjustment work of the industrial robot according to the real-time spatial pose abnormal adjustment data of the industrial robot;

[0072] S7, constructing real-time spatial pose adjustment monitoring data of the industrial robot and performing spatial pose adjustment feedback work of the industrial robot.

[0073] Further, please refer to Figure 1 -Figure 2 The operation steps of collecting real-time pose image data of the industrial robot are as follows:

[0074] S11, simultaneously collecting spatial pose image information of the running state of the industrial robot from two different spatial positions online through a binocular vision imaging processing device, and generating real-time pose image data A=(a1, a2) of the industrial robot, wherein a1 represents first-view real-time pose image data of the industrial robot, and a2 represents second-view real-time pose image data of the industrial robot, and the binocular vision imaging processing device includes any one of a BASLER binocular stereo camera and an OBI Zhongguang Gemini binocular structured light 3D camera.

[0075] The operation steps of performing real-time pose image preprocessing of the industrial robot according to the real-time pose image data of the industrial robot to generate standard real-time pose image data of the industrial robot are as follows:

[0076] S21, performing real-time pose image denoising preprocessing of the industrial robot on the first-view real-time pose image data a1 and the second-view real-time pose image data a2 in the real-time pose image data A of the industrial robot by using a BM3D algorithm, and generating standard real-time pose image data A'=(a'1, a'2) of the industrial robot, wherein a'1 represents standard first-view real-time pose image data of the industrial robot, and a'2 represents standard second-view real-time pose image data of the industrial robot.

[0077] The operation steps of performing real-time spatial pose three-dimensional model construction processing of the industrial robot according to the standard real-time pose image data of the industrial robot and the spatial pose model data of the industrial robot to generate real-time spatial pose model data of the industrial robot are as follows:

[0078] S31, establishing a spatial pose model data set M=(m1, …, mg, …, mk) of the industrial robot, g=1, 2, 3, …, κ; wherein mg represents industrial robot spatial pose model data corresponding to the gth binocular vision industrial robot pose image type, and κ represents the maximum value of the number of binocular vision industrial robot pose image types; the binocular vision industrial robot pose image represents image information of different types of running poses of the industrial robot collected online from two different spatial positions through a binocular vision imaging processing device; and the industrial robot spatial pose model data represents standard industrial robot spatial pose three-dimensional entity model parameters set for binocular vision industrial robot pose image features; g κ g

[0079] ​​​S32. Combine the standard industrial robot first-view real-time pose image data a'1 and standard industrial robot second-view real-time pose image data a'2 from the standard industrial robot real-time pose image data A' with the industrial robot spatial pose model data m from the industrial robot spatial pose model data set M. g Perform feature matching on industrial robot pose images to search for industrial robot spatial pose model data m that matches the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2. g And construct the real-time spatial pose model data m of the industrial robot. A Execute to generate real-time spatial pose model data m for industrial robots A The specific operating steps are as follows:

[0080] S321. Initialize the algorithm parameters: population size N, maximum number of iterations T;

[0081] S322. Initialize the population, calculate fitness, and determine the pose search pathfinders and pose search followers;

[0082] S323, According to the position formula Update the position of the pose search pathfinder in the search space of the industrial robot spatial pose model dataset M, where t represents the current iteration number of the algorithm; This represents the position of the pose search pathfinder ω in the industrial robot spatial pose model dataset M after the t-th iteration. This represents the position of the pose search pathfinder ω in the industrial robot spatial pose model dataset M after the (t-1)th iteration. This represents the position of the pose search pathfinder ω in the industrial robot spatial pose model dataset M after the (t+1)th iteration. The step size factor for the pathfinder's movement in pose search. It follows a uniform distribution in [0,1].

[0083] S324, According to the position formula Update the position of the pose search follower in the search space of the industrial robot spatial pose model dataset M, where... This represents the position of pose search follower i in the search space of the industrial robot spatial pose model dataset M after the t-th iteration. This represents the position of pose search follower i in the search space of the industrial robot spatial pose model data set M after the (t+1)th iteration; This represents the position of other pose search followers j in the industrial robot spatial pose model dataset M after the t-th iteration. Movement is not only related to pose search Pathfinder location Related, and influenced by the position of other poses search followers The influence of R1 and R2 is given by R1 = ◇ × r1 and R2 = ∫ × r2. ◇ represents the interaction coefficient between pose search followers and ∫ represents the attraction coefficient of the pose search pathfinder to the pose search follower. Both ◇ and ∫ follow a uniform distribution in [1,2]. r1 is the step size factor for the movement of the pose search follower to other pose search followers, and r2 is the step size factor for the movement of the pose search follower to the pose search pathfinder. Both r1 and r2 are random numbers in the range [0,1].

[0084] S325. Calculate the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2, and search the space space of the industrial robot spatial pose model data set M for all industrial robot spatial pose model data m. g The fitness value is calculated, and the industrial robot spatial pose model data m with the highest fitness value compared to the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2 is updated. g Global optimum;

[0085] S326. When the maximum number of iterations T is satisfied, output the industrial robot spatial pose model data m that best matches the standard industrial robot first-view real-time pose image data a'1 and the standard industrial robot second-view real-time pose image data a'2 found in step S325. g And through data identification, real-time spatial pose model data of industrial robots is generated. A .

[0086] The real-time pose image preprocessing unit of the industrial robot dynamically collects real-time pose image parameters of the industrial robot by using a binocular vision imaging device, so as to realize efficient collection of real-time pose image parameters of the industrial robot based on binocular machine vision. The real-time pose image preprocessing unit of the industrial robot performs real-time pose image denoising preprocessing of the industrial robot according to the real-time pose image parameters of the industrial robot in combination with a data denoising algorithm, so as to accurately collect real-time pose image parameters of the industrial robot and improve the collection accuracy of real-time pose images of the industrial robot. The real-time spatial pose model generation unit of the industrial robot intelligently constructs a real-time spatial pose three-dimensional model of the industrial robot according to the standard real-time pose image parameters of the industrial robot collected based on binocular machine vision in combination with an intelligent recognition algorithm and scientifically preset spatial pose model parameters of the industrial robot, so as to dynamically construct a spatial pose model of the industrial robot and improve the efficiency and accuracy of real-time spatial pose adjustment of the industrial robot.

[0087] Further, please refer to Figure 1 Figure 2 , based on the real-time spatial pose model data of the industrial robot and the abnormal spatial pose model data of the industrial robot, the real-time spatial pose abnormal state of the industrial robot is analyzed and processed to generate real-time spatial pose abnormal state analysis data of the industrial robot. When normal, the operation steps of this industrial robot pose adjustment operation are as follows:

[0088] S41, establish an industrial robot spatial pose model data set U=(u1,…,u b ,…,u λ ), b=1,2,3,…,λ; wherein u b represents the industrial robot spatial pose model data corresponding to the bth industrial robot abnormal spatial pose type, and λ represents the maximum value of the number of industrial robot abnormal spatial pose types; the industrial robot abnormal spatial pose type includes industrial robot joint motion over-limit spatial pose, industrial robot joint reaching singular point spatial pose, and industrial robot motion attitude deviation; the industrial robot spatial pose model data represents the standard spatial pose three-dimensional entity model parameters set for different industrial robot abnormal spatial pose types;

[0089] S42, using the ARAP search algorithm, the real-time spatial pose model data m A of the industrial robot is matched with the industrial robot spatial pose model data u b in the industrial robot spatial pose model data set U to generate industrial robot real-time spatial pose abnormal state analysis data U fenxi according to the pose model feature matching result;

[0090] When m A is matched with u b ​The pose model feature is not matched successfully, indicating that the current spatial motion posture of the industrial robot meets the safety standard posture, and the industrial robot real-time spatial pose abnormal state analysis data U is output fenxi is normal, and the industrial robot pose adjustment operation is directly ended at this time;

[0091] When m A and u b The pose model feature is matched successfully, indicating that the current spatial motion posture of the industrial robot does not meet the safety standard posture, and the industrial robot real-time spatial pose abnormal state analysis data U is output fenxi is abnormal, and the industrial robot spatial pose model data u b corresponding to the industrial robot abnormal spatial pose type text information is output.

[0092] When abnormal, the industrial robot real-time spatial pose abnormal adjustment parameter matching processing is performed according to the industrial robot real-time spatial pose abnormal state analysis data and the industrial robot abnormal spatial pose adjustment data, and the operation steps of generating the industrial robot real-time spatial pose abnormal adjustment data are as follows:

[0093] S51, when the industrial robot real-time spatial pose abnormal state analysis data U fenxi is abnormal, the industrial robot abnormal spatial pose adjustment data set K = (k1, …, k b , …, k λ ) is established, where k b represents the industrial robot abnormal spatial pose adjustment data corresponding to the bth industrial robot abnormal spatial pose type, and the industrial robot abnormal spatial pose adjustment data represents the industrial robot standard spatial pose abnormal correction motion trajectory coordinate parameters set for different industrial robot abnormal spatial pose types;

[0094] S52, the BERT language model algorithm is adopted to perform industrial robot abnormal spatial pose type character matching on the industrial robot real-time spatial pose abnormal state analysis data U fenxi corresponding to the industrial robot abnormal spatial pose type text information and the industrial robot abnormal spatial pose adjustment data k b in the industrial robot abnormal spatial pose adjustment data set K, and the industrial robot real-time spatial pose abnormal adjustment data k fenxi corresponding to the industrial robot real-time spatial pose abnormal state analysis data U b is searched out, and the data identification is performed to generate the industrial robot real-time spatial pose abnormal adjustment data k shishi .

[0095] The industrial robot real-time spatial pose abnormal state analysis unit, based on the industrial robot real-time spatial pose model parameters, combines intelligent search algorithm and industrial robot abnormal spatial pose model parameters based on big data storage to scientifically analyze the industrial robot real-time spatial pose abnormal state, realizes the dynamic scientific identification of the industrial robot spatial pose fault state, and improves the real-time and precision of the industrial robot spatial pose adjustment; The industrial robot real-time spatial pose abnormal adjustment parameter matching unit, according to the industrial robot real-time spatial pose abnormal state analysis parameter, combines intelligent search algorithm and standard stored industrial robot abnormal spatial pose adjustment parameter to intelligently match the industrial robot real-time spatial pose abnormal adjustment parameter, realizes the dynamic construction of the industrial robot spatial pose fault calibration control parameter, and improves the safety and reliability of the industrial robot spatial pose adjustment.

[0096] Further, please refer to Figure 1 - Figure 2 According to the operation steps of the industrial robot real-time spatial pose abnormal adjustment data to perform the industrial robot spatial pose adjustment operation as follows:

[0097] S61, the industrial robot control end according to the industrial robot real-time spatial pose abnormal adjustment data k shishi The corresponding spatial pose abnormal adjustment parameter controls the industrial robot to perform the industrial robot spatial pose adjustment operation.

[0098] The operation steps of constructing the industrial robot real-time spatial pose adjustment monitoring data and performing the industrial robot spatial pose adjustment feedback operation are as follows:

[0099] S71, the industrial robot real-time pose image data A, the industrial robot real-time spatial pose model data m A , the industrial robot real-time spatial pose abnormal state analysis data U fenxi , the industrial robot real-time spatial pose abnormal adjustment data k shishi Data combination to construct the industrial robot real-time spatial pose adjustment monitoring data O, wherein O=(A,m A ,U fenxi ,k shishi );

[0100] S72, the industrial robot real-time spatial pose adjustment monitoring data O is transmitted to the industrial robot supervision platform through the industrial internet online to perform the industrial robot spatial pose adjustment feedback operation.

[0101] The industrial robot real-time spatial pose abnormality adjustment job execution unit executes the industrial robot spatial pose adjustment job based on the industrial robot real-time spatial pose abnormality adjustment parameters and the industrial robot control end autonomously and efficiently, thereby improving the response speed of the industrial robot spatial pose adjustment job; the industrial robot real-time spatial pose abnormality adjustment feedback unit scientifically constructs the industrial robot real-time spatial pose adjustment monitoring data based on the real-time spatial pose image, the real-time spatial pose model, the real-time spatial pose abnormality analysis result and the real-time spatial pose adjustment parameter of the industrial robot, and simultaneously combines the industrial robot supervision platform to efficiently and dynamically execute the industrial robot spatial pose adjustment feedback job, thereby realizing online feedback of the industrial robot pose adjustment result and improving the digitization of the industrial robot pose adjustment.

[0102] Embodiment 2:

[0103] Please refer to Figure 1 - Figure 2 A multi-vision machine pose adjustment system based on binocular vision is used to realize a multi-vision machine pose adjustment method based on binocular vision. The system includes an industrial robot pose model construction module, an industrial robot pose monitoring module, and an industrial robot pose adjustment module.

[0104] The industrial robot pose model construction module includes an industrial robot real-time pose image preprocessing unit, an industrial robot real-time pose image preprocessing unit, an industrial robot spatial pose model storage unit, and an industrial robot real-time spatial pose model generation unit.

[0105] The industrial robot real-time pose image preprocessing unit collects industrial robot real-time pose image data through a binocular vision imaging processing device; the industrial robot real-time pose image preprocessing unit performs real-time pose image preprocessing of the industrial robot based on the industrial robot real-time pose image data to generate standard industrial robot real-time pose image data; the industrial robot spatial pose model storage unit is used to store industrial robot spatial pose model data; and the industrial robot real-time spatial pose model generation unit performs real-time spatial pose three-dimensional model construction processing of the industrial robot based on the standard industrial robot real-time pose image data and the industrial robot spatial pose model data to generate industrial robot real-time spatial pose model data.

[0106] The industrial robot pose monitoring module includes an industrial robot abnormal spatial pose model storage unit, an industrial robot real-time spatial pose abnormality state analysis unit, an industrial robot abnormal spatial pose adjustment parameter storage unit, and an industrial robot real-time spatial pose abnormality adjustment parameter matching unit.

[0107] The industrial robot abnormal space pose model storage unit is configured to store industrial robot abnormal space pose model data; the industrial robot real-time space pose abnormal state analysis unit is configured to perform real-time space pose abnormal state analysis processing of the industrial robot based on the industrial robot real-time space pose model data and the industrial robot abnormal space pose model data, and generate industrial robot real-time space pose abnormal state analysis data; the industrial robot abnormal space pose adjustment parameter storage unit is configured to store industrial robot abnormal space pose adjustment data; and the industrial robot real-time space pose abnormal adjustment parameter matching unit is configured to perform real-time space pose abnormal adjustment parameter matching processing of the industrial robot based on the industrial robot real-time space pose abnormal state analysis data and the industrial robot abnormal space pose adjustment data, and generate industrial robot real-time space pose abnormal adjustment data.

[0108] The industrial robot pose adjustment module comprises an industrial robot real-time space pose abnormal adjustment operation execution unit and an industrial robot real-time space pose abnormal adjustment feedback unit.

[0109] The industrial robot real-time space pose abnormal adjustment operation execution unit is configured to perform industrial robot space pose adjustment operation in combination with the industrial robot control end based on the industrial robot real-time space pose abnormal adjustment data; and the industrial robot real-time space pose abnormal adjustment feedback unit is configured to construct industrial robot real-time space pose adjustment monitoring data and perform industrial robot space pose adjustment feedback operation in combination with the industrial robot supervision platform.

[0110] Although the embodiments of the present application have been shown and described, it is to be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A multi-vision machine pose adjustment method based on binocular vision, characterized in that, The method comprises the following steps: S1, collecting real-time pose image data of an industrial robot; S2, performing real-time pose image preprocessing of the industrial robot according to the real-time pose image data of the industrial robot, to generate standard real-time pose image data of the industrial robot; S3, performing real-time spatial pose three-dimensional model construction processing of the industrial robot according to the standard real-time pose image data of the industrial robot and spatial pose model data of the industrial robot, to generate real-time spatial pose model data of the industrial robot; S4, performing real-time spatial pose abnormal state analysis processing of the industrial robot based on the real-time spatial pose model data of the industrial robot and abnormal spatial pose model data of the industrial robot, to generate real-time spatial pose abnormal state analysis data of the industrial robot; when normal, ending the current industrial robot pose adjustment operation; S5, when abnormal, performing real-time spatial pose abnormal adjustment parameter matching processing of the industrial robot according to the real-time spatial pose abnormal state analysis data of the industrial robot and abnormal spatial pose adjustment data of the industrial robot, to generate real-time spatial pose abnormal adjustment data of the industrial robot; S6, performing spatial pose adjustment operation of the industrial robot according to the real-time spatial pose abnormal adjustment data of the industrial robot; S7, constructing real-time spatial pose adjustment monitoring data of the industrial robot and performing spatial pose adjustment feedback operation of the industrial robot. 2.The multi-vision machine pose adjustment method based on binocular vision of claim 1, wherein: The S1 comprises the following steps: S11, simultaneously collecting spatial pose image information of an industrial robot running state from two different spatial positions online through a binocular vision imaging processing device, and generating industrial robot real-time pose image data wherein represents industrial robot first-view real-time pose image data, represents industrial robot second-view real-time pose image data. 3.The multi-vision machine pose adjustment method based on binocular vision of claim 2, wherein: The S2 comprises the following steps: S21, adopting the BM3D algorithm to the real-time pose image data of the industrial robot in the middle and the real-time pose image denoising preprocessing of the industrial robot is carried out, and standard real-time pose image data of the industrial robot is generated wherein the standard industrial robot first-view real-time pose image data is represented, the standard industrial robot second-view real-time pose image data is represented.

4. The multi-vision machine pose adjustment method based on binocular vision according to claim 3, characterized in that: The S3 comprises the following steps: S31, establish an industrial robot space pose model data set , ; wherein, represents the first binocular vision industrial robot pose image type corresponding to the industrial robot space pose model data, represents the maximum value of the number of binocular vision industrial robot pose image types; S32, the above The above and stated With the The process described above involves performing feature matching on the pose image of an industrial robot to search for features that match the described image. and stated The matching And construct real-time spatial pose model data for industrial robots. The process generates the real-time spatial pose model data of the industrial robot. The specific operating steps are as follows: S321, initializing algorithm parameters, population number N, and maximum iteration number T; S322, initializing population, calculating fitness, and determining pose search pathfinder and pose search follower; S323, performing the updating the pose search explorer position in the search space; S324, performing an updated pose search follower searches for a position in the search space; S325、calculate the fitness value of the and the with the search space of all the fitness value, and update the search with the and the fitness value of the global optimal value; S326、when the maximum iteration number T is satisfied, output the search result of step S325 that matches the and the most matching and generate industrial robot real-time spatial pose model data through data identification .

5. The multi-vision machine pose adjustment method based on binocular vision according to claim 4, characterized in that: The S4 comprises the following steps: S41, establish an industrial robot space pose model data set , ; wherein represents the first industrial robot abnormal space pose type corresponding to the industrial robot space pose model data, represents the maximum value of the number of industrial robot abnormal space pose types; S42, using the ARAP search algorithm to search the with the described in the Perform pose model feature matching, and generate industrial robot real-time spatial pose abnormal state analysis data according to the pose model feature matching result ; When With No pose model features are matched successfully, output the is normal, at which point the industrial robot pose adjustment job is directly ended this time; When With The pose model feature matching is successful, and the is an exception, and the corresponding industrial robot abnormal space pose type text information is output.

6. The multi-vision machine pose adjustment method based on binocular vision according to claim 5, characterized in that: The S5 comprises the following steps: S51、when the abnormal, establish an industrial robot abnormal space pose adjustment data set wherein indicates the industrial robot abnormal space pose type corresponding to the industrial robot abnormal space pose adjustment data S52, adopting a BERT language model algorithm to the text information of the abnormal space pose type of the industrial robot corresponding to the text information of the abnormal space pose type of the industrial robot and the text information of the abnormal space pose type of the industrial robot corresponding to the text information of the abnormal space pose type of the industrial robot corresponding to the text information of the abnormal space pose type of the industrial robot corresponding to the text information of the abnormal space pose type of the industrial robot corresponding to the text information of the abnormal space pose type of the industrial robot corresponding to the text information of the abnormal space pose type of the industrial robot 7. The multi-vision machine pose adjustment method based on binocular vision according to claim 6, characterized in that: The S6 comprises the following steps: S61、industrial robot control end adjusts the industrial robot according to the The corresponding spatial pose abnormality adjustment parameter controls the industrial robot to perform the industrial robot spatial pose adjustment work. 8.The multi-vision machine pose adjustment method based on binocular vision of claim 7, wherein: The S7 comprises the following steps: S71、combining data to construct industrial robot real-time spatial pose adjustment monitoring data combining data to construct industrial robot real-time spatial pose adjustment monitoring data ;​​​ S72, the above The industrial robot's spatial pose adjustment feedback is performed by transmitting data online to the industrial robot monitoring platform via the Industrial Internet of Things (IIoT).

9. A binocular vision based multi-vision machine pose adjustment system for implementing the binocular vision based multi-vision machine pose adjustment method of any one of claims 1-8, characterized in that: The system comprises an industrial robot pose model construction module, an industrial robot pose monitoring module, and an industrial robot pose adjustment module.

Images