Industrial robot motion control system based on computer vision

Abstract

The invention discloses an industrial robot motion control system based on computer vision, which relates to the technical field of robot control and comprises a computer vision module, a feature extraction module, a path planning module, a motion control module, a dynamic adjustment module and a wireless communication module. The computer vision module comprises a multi-image acquisition unit, an image processing unit and an image fusion unit, the path planning module obtains planning path information of the robot according to target point data analysis, and the image fusion unit is used for fusing multi-image data, integrating information of different sensors, compensating illumination and shielding problems, and improving the path planning precision. The influence of environmental factors on images is reduced, the image quality is improved, the accuracy of feature recognition is improved, and the effects of real-time track adjustment, working efficiency improvement and potential safety hazard reduction are achieved through real-time monitoring, actual and planned path comparison, threshold value setting in combination with time and re-planning when the threshold value is exceeded in robot movement through a dynamic adjustment module.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and in particular to an industrial robot motion control system based on computer vision. Background Art

[0002] Computer vision is the study of how computers can acquire, understand, and interpret information from images or videos. It mimics the human visual system, enabling computers to recognize objects, scenes, and actions by processing, analyzing, and understanding images. Industrial robots are automated machines used in industrial production, typically featuring a multi-degree-of-freedom mechanical structure and a programmable control system. Motion control systems precisely control the robot's movement, including position, velocity, and acceleration, ensuring it follows a predetermined trajectory and speed. This allows them to perform various tasks in industrial environments, such as handling, welding, assembly, and painting.

[0003] Currently, the images collected by sensors are easily affected by environmental factors such as lighting and occlusion, resulting in inaccurate information, reducing the quality of object recognition, and failing to provide a reliable basis for the robot's motion control, thereby reducing the accuracy of motion control. In addition, during the robot's movement, mechanical errors, environmental interference and other factors can easily lead to deviations in the motion trajectory. The current system lacks an effective real-time correction mechanism and is unable to correct trajectory deviations in a timely manner, which reduces the robot's work efficiency and increases safety hazards. Summary of the Invention

[0004] The purpose of the present invention is to provide an industrial robot motion control system based on computer vision, which solves the problems raised in the above background technology.

[0005] To achieve the above-mentioned object, the present invention provides the following technical solution: an industrial robot motion control system based on computer vision, comprising a computer vision module, a feature extraction module, a path planning module, a motion control module, a dynamic adjustment module and a wireless communication module;

[0006] The computer vision module includes a multi-image acquisition unit, an image processing unit and an image fusion unit;

[0007] The multi-image acquisition unit is used to install sensors at different locations on site, and acquire images through multiple sensors to obtain multi-image data;

[0008] The image processing unit is used to process the multi-image data to improve image quality and obtain processed multi-image data;

[0009] The image fusion unit is used to fuse the processed multi-image data to obtain fused image data;

[0010] The feature extraction module is used to analyze the fused image data to obtain gradient information and local binary information, combine the gradient information and the local binary information to obtain extracted features, and the feature extraction module includes template features, and combines the extracted features with the template features to obtain target point data;

[0011] The path planning module obtains the planned path information of the robot based on the target point data analysis, and after obtaining the planned path information, the robot movement can be controlled by the motion control module;

[0012] The dynamic adjustment module obtains actual path information by real-time monitoring of the robot motion data, compares the actual path information with the planned path information to obtain motion difference information, adjusts the robot motion data in real time according to the motion difference information, and sets a deviation threshold for the motion difference information. When the motion difference information exceeds the deviation threshold, the path information is replanned by the path planning module.

[0013] Optionally, the multi-image data includes image one data, image two data and image three data, and the image fusion unit first calculates the information entropy data of the multi-image data to obtain information entropy one, information entropy two and information entropy three, adds information entropy one, information entropy two and information entropy three to obtain total information entropy, compares the information entropy data of each image with the total information entropy to obtain each image weight, sums each image weight to obtain the total image weight, multiplies each image one data, image two data and image three data with the corresponding image weight respectively and sums them to obtain the total image data, and compares the total image data with the total image weight to obtain the fused image data.

[0014] Optionally, the feature extraction module is used to analyze the fused image data to obtain pixel value information of the coordinate points of the fused image, analyze the pixel value information of the coordinate points of the fused image to obtain gradient information in the x direction of the coordinate points and gradient information in the y direction of the coordinate points, combine the gradient information in the x direction of the coordinate points and the gradient information in the y direction of the coordinate points to obtain the gradient amplitude feature of the coordinate points, and then analyze the size relationship between the pixel value information of the coordinate points of the fused image and the neighboring pixels to obtain the LBP feature of the coordinate points, combine the gradient amplitude feature of the coordinate points with the LBP feature of the coordinate points to obtain the extracted feature, and the feature extraction module includes template features, and the extracted features are combined with the template features to obtain the target point data.

[0015] Optionally, the target object is determined by the target point data, and the target point coordinate information is determined by the position of the target object in the fused image data. The path planning module sets the position of the robot as the starting point to obtain the starting point coordinate information, and then combines the target point coordinate information with the starting point coordinate information for analysis to obtain the planned path information. The motion control module controls the robot movement according to the planned path information.

[0016] Optionally, the dynamic adjustment module obtains actual path information by real-time monitoring of the robot motion data, compares the actual path information with the planned path information to obtain motion difference information, sets a deviation threshold of one for the motion difference information, and when the motion difference information is not less than the deviation threshold of one, adjusts the robot in real time, and obtains the total motion difference information per unit time based on the motion difference information and combined with the time factor, sets a deviation threshold of two for the total motion difference information per unit time, and when the total motion difference information per unit time is not less than the deviation threshold of two, replans the path information through the motion control module.

[0017] Optionally, a data storage module is also included, which is used to store multi-image data, fused image data, template features, planned path information, actual path information and motion difference information for subsequent analysis.

[0018] Optionally, the wireless communication module is signal-connected to the computer vision module, feature extraction module, path planning module, motion control module and dynamic adjustment module. The wireless communication module is used to transmit data from each module to a terminal device, and management personnel remotely monitor the system through the terminal device.

[0019] Optionally, the image processing unit includes registration processing, filtering processing and denoising processing.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. The present invention fuses the processed multi-image data through an image fusion unit, fuses the information of each image, and obtains fused image data. The fused image data can integrate image information collected by different sensors, compensate for problems such as lighting and obstructions, make up for the shortcomings of a single image, and improve the integrity and accuracy of image information, so that the fused image can more comprehensively and accurately reflect the scene information, providing a more reliable basis for subsequent target object recognition and path planning, thereby reducing the impact of environmental factors on the system and improving the accuracy of robot motion control.

[0022] 2. During the robot's movement process, the present invention monitors the robot's motion data in real time through a dynamic adjustment module to obtain actual path information, compares the actual path information with the planned path information to obtain motion difference information, adjusts the robot's motion data in real time according to the motion difference information, and sets a deviation threshold for the motion difference information in combination with the time factor. When the motion difference information within the set time exceeds the deviation threshold, the path planning module replans the path information, thereby realizing real-time adjustment of the robot's motion trajectory, being able to correct trajectory deviations in a timely manner, avoiding large deviations of the motion trajectory, improving the robot's work efficiency, and reducing safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a block diagram of the system module of the present invention;

[0024] Figure 2 Schematic diagram of the system workflow of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0026] See also Figure 1 and Figure 2 ,This implementation provides an industrial robot motion control system based on computer ,vision, including a computer vision module, a feature extraction module, a path planning module, a motion ,control module and a dynamic adjustment module;

[0027] The computer vision module includes a multi-image acquisition unit, an image processing unit, and an image fusion unit;

[0028] The multi-image acquisition unit is used to install sensors at different locations on site, collect images through multiple sensors, and obtain multi-image data;

[0029] The image processing unit is used to process the multi-image data to improve the image quality and obtain processed multi-image data;

[0030] The image fusion unit is used to fuse the processed multi-image data to obtain fused image data;

[0031] The feature extraction module is used to analyze the fused image data to obtain gradient information and local binary information, and combine the gradient information and local binary information to obtain extracted features. The feature extraction module includes template features, and the extracted features are combined with the template features to obtain target point data.

[0032] The path planning module analyzes the target point data to obtain the robot's planned path information. After obtaining the planned path information, the robot's movements can be controlled through the motion control module.

[0033] The motion control module includes PLC controller, driver, motor, encoder, gyroscope, accelerometer, etc., which is used to realize the movement of the robot;

[0034] The dynamic adjustment module obtains the actual path information by real-time monitoring of the robot motion data, compares the actual path information with the planned path information to obtain the motion difference information, adjusts the robot motion data in real time according to the motion difference information, and sets a deviation threshold for the motion difference information. When the motion difference information exceeds the deviation threshold, the path information is replanned through the path planning module.

[0035] In this embodiment, first, multi-image data of the scene is acquired by the multi-image acquisition unit in the computer vision module. The multi-image data includes images acquired from different directions. Due to the complexity of the industrial environment, the overall image quality is poor. At this time, each image is processed by the image processing unit to improve the image quality to obtain processed multi-image data. Finally, the processed multi-image data is fused by the image fusion unit to fuse the information of each image to obtain fused image data. The fused image data can integrate the image information acquired by different sensors, make up for the shortcomings of a single image, and improve the integrity and accuracy of the image information, so that the fused image can reflect the scene information more comprehensively and accurately, providing a more reliable basis for subsequent target object recognition and path planning, thereby reducing the impact of environmental factors on the system and improving the accuracy of robot motion control.

[0036] After obtaining the fused image data, the fused image data is analyzed through the feature extraction module to obtain gradient information and local binary information. The gradient information and local binary information are combined to obtain extracted features. The feature extraction module includes template features. The extracted features are combined with the template features for analysis to obtain target point data. The path planning module analyzes the target point data and the current position of the robot to obtain the planned path information. The motion control module controls the robot movement through the planned path information.

[0037] During the robot's movement, the dynamic adjustment module monitors the robot's motion data in real time to obtain actual path information, compares the actual path information with the planned path information to obtain motion difference information, and adjusts the robot's motion data in real time according to the motion difference information. In combination with the time factor, a deviation threshold is set for the motion difference information. When the motion difference information within the set time exceeds the deviation threshold, the path planning module replans the path information, thereby realizing real-time adjustment of the robot's motion trajectory, being able to correct trajectory deviations in a timely manner, avoiding large deviations of the motion trajectory, improving the robot's work efficiency, and reducing safety hazards.

[0038] Furthermore, the multi-image data includes image one data, image two data, image three data to image n data. The image fusion unit first calculates the information entropy data of the multi-image data to obtain information entropy one, information entropy two, information entropy three to information entropy n, adds the information entropy one, information entropy two, information entropy three to information entropy n to obtain the total information entropy, and compares the total image data with the total image weight to obtain the fused image data. The fusion process of the image fusion unit is as follows:

[0039]

[0040] in;

[0041] W1 is the weight of image i, which indicates the importance of the image in the image fusion process. i )) and the total information entropy (i.e. ) to obtain the weight of each image (i.e. W i ), and then sum up the weights of each image to get the total image weight (i.e. );

[0042] P i (x, y) is the pixel value of image i at position (x, y), and the total image data (i.e. );

[0043] P all (x, y) is the pixel value of the fused image position (x, y), P all Represents the fused image, the total image data (i.e. ) and the total image weight (i.e. ) are compared to obtain the fused image data, which includes the pixel value P of the fused image position (x, y) all (x,y);

[0044] n is the number of images;

[0045] H(Pi ) is the information entropy of image i, which is used to measure the amount of information contained in the image. The larger the information entropy, the richer the image information and the higher its weight in image fusion;

[0046] The information entropy H(P i ) The process is as follows:

[0047]

[0048] k is the grayscale value of the pixel in the image. In a common 8-bit grayscale image, the grayscale value range is 0 to 255, so the value range of k is 0 to 255;

[0049] pk i is the probability of a pixel with grayscale value k appearing in image i. If the size of image i is M×N and the number of pixels with grayscale value k is A, then the probability of a pixel with grayscale value k appearing in image i is pk i Equal to pk i Divide by M×N;

[0050] log2(pk i ) indicates the pair pk i Take the base 2 logarithm;

[0051] Symbol - is used to ensure that the value of information entropy is non-negative, because the probability pk of a pixel with gray value k in image i appears i The value range is 0≤pk i ≤1, then log2(pk i ) is usually negative because 0<pk i When <1, log2(pk i ) is a negative number, so adding a negative sign can make the information entropy H(P i ) is a positive number, which conforms to the physical meaning of information entropy.

[0052] Specifically, the information entropy H(P i ) is used to measure the uncertainty or confusion of information in an image. The information entropy H(P i ) is higher, indicating that the image contains richer and more complex information;

[0053] The information entropy H(P i) is lower, indicating that the image information is more simple and ordered. In the case of multi-image fusion, information entropy can be used as an indicator to measure image quality and the amount of useful information. Images with high information entropy usually contain more valuable information, so it is necessary to increase the weight during fusion. For example, under different lighting conditions, images collected by various sensors may show differences in brightness, contrast, etc. Multi-image fusion can combine the advantages of different images, compensate for problems such as uneven lighting, increase the weight of images with better lighting and fewer obstructions, and improve the visual quality of the image;

[0054] Multiple sensors capture images from different angles, each containing a portion of the scene information. Multi-image fusion then integrates this information to form a more comprehensive and complete description of the scene. For example, on an industrial production line, sensors at different locations can capture different sides of a part. The fused images allow robots to more accurately understand the overall shape and size of the part, avoiding information loss due to the limitations of a single perspective. This improves the accuracy of the image information and facilitates subsequent feature recognition and target positioning.

[0055] Furthermore, the feature extraction module is used to analyze the fused image data to obtain the pixel value information of the fused image coordinate point, which is the pixel value P of the fused image position (x, y). all (x, y), analyze the pixel value information of the fused image coordinate point to obtain the gradient information G in the x direction of the coordinate point x (x,y) and the gradient information G in the y direction of the coordinate point y (x,y), G x (x,y) and G y The process of obtaining (x,y) is as follows:

[0056] G x (x,y)=P all (x+1,y)-P all (x-1,y)

[0057] G y (x,y)=P all (x,y+1)-P all (x,y-1)

[0058] Among them G x (x,y) is the gradient value of the coordinate point (x,y) in the x direction in the fused image, which represents the grayscale change rate in that direction of the fused image, that is, the gradient information of the coordinate point in the x direction;

[0059] G y (x,y) is the gradient value in the y direction of the coordinate point (x,y) in the fused image, that is, the gradient information in the y direction of the coordinate point;

[0060] P all (x+1,y) is the pixel value at the coordinate point (x+1,y) in the fused image;

[0061] P all (x-1, y) is the pixel value at the coordinate point (x-1, y) in the fused image;

[0062] P all (x, y+1) is the pixel value at the coordinate point (x, y+1) in the fused image;

[0063] P all (x, y-1) is the pixel value at the coordinate point (x, y-1) in the fused image;

[0064] The gradient information in the x direction of the coordinate point (i.e. G x (x,y)) and the gradient information in the y direction of the coordinate point (ie G y (x,y)) is combined with the analysis to obtain the coordinate point gradient amplitude feature G(x,y). The process of obtaining G(x,y) is as follows:

[0065]

[0066] G(x,y) is the gradient amplitude of the fused image at the coordinate point (x,y);

[0067] If the pixel value P at the fused image position (x, y) all (x, y) is the center coordinate point of the fused image, so the neighborhood pixel value is P all (x+c,y+j), where c and j range from 0 to 1, and both c and j are not 0, and the neighborhood pixel value is compared with P all (x, y) is analyzed to obtain the LBP features of the coordinate points (i.e. LBP (x, y)). The process of obtaining LBP (x, y) is as follows:

[0068]

[0069] Where LBP(x,y) is the LBP value of the fused image at the coordinate point (x,y), and the neighborhood pixel value is combined with the pixel value information of the coordinate point (i.e. P all (x,y)) is analyzed to obtain the LBP features of the coordinate points (i.e. LBP(x,y));

[0070] b is the number of the neighborhood pixel;

[0071] The LBP value LBP(x,y) of the fused image at the coordinate point (x,y) can describe the texture features of the local area of ​​the image. Different texture patterns can be used to distinguish different target objects.

[0072] The gradient amplitude feature of the coordinate point (i.e. G(x,y)) is combined with the LBP feature of the coordinate point (i.e. LBP(x,y)) to obtain the extracted feature (i.e. F(x,y)). The feature extraction module includes the template feature (i.e. T f (x,y)), extract the features (i.e. F(x,y)) and the template features (i.e. T f (x,y)) combined with analysis to obtain the target point data (i.e. f min ), f min The process is as follows:

[0073] f min =arg min f D f

[0074]

[0075] F(x,y)=[G(x,y),LBP(x,y]

[0076] Among them D f To extract feature F and the fth template feature T f The Euclidean distance between them is used to measure the similarity between the two, extracting the feature F and the f-th template feature T f The Euclidean distance D between f The smaller it is, the more features F and T are extracted from the fth module. f The more similar;

[0077] F(x,y) is the feature vector at the coordinate point (x,y) of the fused image;

[0078] T f (x, y) The feature vector at the template feature coordinate point (x, y) is obtained from the known data of the system, that is, it is obtained by pre-calculation of known objects;

[0079] R represents the extracted feature F and the fth template feature T f the area being compared;

[0080] f min D f The template number with the minimum value indicates that the object corresponding to the template is the target object. By comparing the extracted features (i.e. F(x, y)) with the template features (i.e. T f (x,y)) combined with analysis to obtain the target point data (i.e. f min );

[0081] arg min f Indicates the index operator corresponding to the minimum value, indicating that D f Minimum f-value;

[0082] By Df Minimum template number k min The template that is most similar to the extracted features is determined, and each template corresponds to an object, so that D f Minimum template number k min It can determine the target object in the fused image. By using Euclidean distance calculation, it can reduce the calculation time, improve the efficiency of target recognition, and is suitable for industrial needs.

[0083] Specifically, by combining the extracted features (i.e., F(x, y)) with the template features (i.e., T f (x, y)) is compared to determine the target object in the fused image. Other objects in the fused image, such as obstacles, can also be analyzed. By adding template features of known obstacles and then extracting them through the feature recognition module, obstacles and target objects can be marked separately in actual operation. In subsequent path planning, objects that block the robot can be avoided, ensuring the robot's work efficiency while reducing safety hazards.

[0084] Furthermore, through D f Minimum template number k min After the target object is determined, the target point coordinate information is determined by the position of the target object in the fused image data. The path planning module sets the robot's position as the starting point to obtain the starting point coordinate information. The target point coordinate information is combined with the starting point coordinate information for analysis to obtain the planned path information. The motion control module controls the robot's movement according to the planned path information, and the planned path information includes a number of planned trajectory points.

[0085] Several planned trajectory points are represented as (Q b1 ,Q b2 ,...Q be ,...Q bm ), where Q be The coordinates are (x be ,y be ).

[0086] Specifically, after determining the target object, we can get the fused image P all The coordinate information of the target object, that is, the coordinate information of the target point, and the coordinate information of obstacles can be obtained at the same time. When planning the robot path information, it is necessary to avoid the robot from colliding with obstacles while ensuring the shortest route to improve the robot's work efficiency.

[0087] Furthermore, the actual path information includes a number of actual trajectory points, which are represented by (Q ba1 ,Q ba2 ,...Q as ,...Q am), where .Q as The coordinates are (x as ,y as ), the dynamic adjustment module obtains the actual path information by real-time monitoring of the robot motion data, and compares the actual path information with the planned path information to obtain the motion difference information (i.e. D s ), is the motion difference information (ie D s ) Set the deviation threshold value to 1. When the motion difference information is not less than the deviation threshold value, the robot is adjusted in real time. s The process is as follows:

[0088]

[0089] D s is the sth point Q on the actual trajectory as and the bth point Q on the planned trajectory bs The Euclidean distance between the two points indicates the s-th point Q on the actual trajectory. as The deviation of the actual path information (i.e. Q ba1 ,Q ba2 ,...Q as ,...Q am ) and the planned path information (i.e. (Q b1 ,Q b2 ,...Q be ,...Q bm )) to compare and get the motion difference information (ie D s );

[0090] x as is the horizontal coordinate of the sth point on the actual trajectory;

[0091] y as is the ordinate of the sth point on the actual trajectory;

[0092] x bs is the horizontal coordinate of the bth point on the planned trajectory;

[0093] y bs is the ordinate of the bth point on the planned trajectory;

[0094] Set the sth point Q on the actual trajectory as and the bth point Q on the planned trajectory bs The Euclidean distance D between s The deviation threshold is Y1, when D s When the deviation threshold is greater than or equal to Y1, it indicates that the deviation of the point is too large. The robot is adjusted in real time to obtain a new actual trajectory point, which is replaced with the original actual trajectory point. The new actual trajectory point is then compared and analyzed with the planned trajectory point through the dynamic adjustment module.

[0095] Calculate the deviation per unit time. If the number of actual trajectory points passed within the time end Z is 5, add the Euclidean distances between the five actual trajectory points and the planned trajectory points to obtain the total deviation information DZ within the time period Z (i.e., the total motion difference information per unit time). Set the deviation threshold value Y2 of the total deviation information DZ within the time period Z. When DZ ≥ the deviation threshold value Y2, it means that the route deviation is too large. At this time, the path planning module uses the current robot position as the new starting point, the target point coordinates remain unchanged, and resets the planned path information, thereby realizing real-time adjustment of the robot's motion trajectory, and being able to correct the trajectory deviation in time. By calculating the total deviation of the robot within the set time period, it prevents the problem of excessive deviation caused by the robot's small deviation for a long time not being detected by the system in time, thereby improving the robot's work efficiency and reducing safety hazards.

[0096] Furthermore, a data storage module is used to store multi-image data, fused image data, template features, planned path information, actual path information, motion difference information, etc.

[0097] Specifically, by storing the main data during the operation of the system through the data storage module, it is possible to facilitate the subsequent tracing and analysis of the system's operation status. For example, by analyzing historical motion difference information, it is possible to find out the reasons why the robot is prone to deviations in certain specific scenarios, thereby optimizing the system. In addition, the stored template feature data can be continuously updated and improved as the system is used. By comparing and analyzing the newly collected target object features with the stored template features, the deficiencies in the template features can be discovered, the template can be updated in a timely manner, and the accuracy of target object recognition can be improved.

[0098] Furthermore, the image processing unit includes registration processing, filtering processing and denoising processing.

[0099] In order to accurately align the multi-image data collected by different sensors in space, the registration method based on feature point matching is adopted. In the filtering processing, the median filtering algorithm is used. For each pixel point in the multi-image data, a neighborhood window is selected with the pixel point as the center, and all pixel values ​​in the neighborhood window are sorted, and the middle value is taken as the new value of the pixel point.

[0100] Median filtering can effectively retain the edge information of the image and perform denoising. In order to smooth the image and reduce the high-frequency noise in the image, a Gaussian filtering algorithm is used. In this system, according to the characteristics of the multi-image data, the appropriate Gaussian kernel size and standard deviation are selected, and a convolution operation is performed on the processed multi-image data to remove the high-frequency noise in the image, making the image smoother and improving the image quality, which is convenient for subsequent image fusion and feature extraction operations;

[0101] Specifically, since the multi-image acquisition unit uses different sensors to capture images, these images may have position deviations in space. Through registration processing, the multi-image data captured by different sensors can be accurately aligned in space, so that subsequent image fusion and feature extraction operations can be performed in a unified spatial coordinate system. For example, in industrial scenes, images captured by sensors at different angles may have rotation, translation and other deviations. Registration processing can eliminate these deviations and provide accurate image data for subsequent target object recognition and path planning. Due to the complexity of the industrial field environment, there may also be high-frequency noise in the images in the multi-image data. These noises will make the image blurred and affect the feature extraction and recognition of the target object. By adopting the Gaussian filtering algorithm, the image can be smoothed and the influence of high-frequency noise can be reduced, making the image detail information clearer, which is convenient for the subsequent feature extraction module to accurately extract the gradient information and local binary information of the target object, improve the recognition accuracy of the target object, and improve the accuracy of the industrial robot motion control system.

[0102] Furthermore, the wireless communication module is signal-connected to the computer vision module, feature extraction module, path planning module, motion control module and dynamic adjustment module. The wireless communication module is used to transmit data from each module to the terminal device, and the management personnel remotely monitor the system through the terminal device.

[0103] Specifically, the wireless communication module includes Bluetooth and WiFi. Managers use terminal devices to connect to the system via Bluetooth or WiFi, and transmit the operating data of each module of the system to the terminal device through the communication module so that managers can monitor remotely. At the same time, the wireless communication module can realize dynamic networking between robots and improve the efficiency of collaboration between robots.

[0104] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. Industrial robot motion control system based on computer vision, characterized by ； A computer vision module, comprising a multi-image acquisition unit and an image fusion unit, wherein the multi-image acquisition unit is configured to install sensors at different locations on site and acquire images through the multiple sensors to obtain multi-image data, and the image fusion unit is configured to fuse the multi-image data to obtain fused image data; A feature extraction module is used to analyze the fused image data to obtain gradient information and local binary information, and combine the gradient information and local binary information to obtain extracted features. The feature extraction module includes template features, and the extracted features are combined with the template features to obtain target point data; The path planning module and motion control module analyze the target point data to obtain the robot's planned path information. After obtaining the planned path information, the motion control module can control the robot's movements. The dynamic adjustment module obtains the actual path information by real-time monitoring of the robot's motion data, compares the actual path information with the planned path information to obtain the motion difference information, adjusts the robot's motion data in real time based on the motion difference information, and sets a deviation threshold for the motion difference information. When the motion difference information exceeds the deviation threshold, the path information is replanned through the path planning module.

2. The computer vision-based industrial robot motion control system according to claim 1, characterized in that: The multi-image data includes image one data, image two data and image three data; The image fusion unit first calculates the information entropy data of multiple image data to obtain information entropy one, information entropy two and information entropy three, adds the information entropy one, information entropy two and information entropy three to obtain the total information entropy, then compares the information entropy data of each image with the total information entropy to obtain the weight of each image, and then sums up each image weight to obtain the total image weight, then multiplies each image one data, image two data and image three data with the corresponding image weight respectively and sums them to obtain the total image data, and finally compares the total image data with the total image weight to obtain the fused image data.

3. The computer vision-based industrial robot motion control system according to claim 2, characterized in that: The feature extraction module is used to analyze the fused image data to obtain pixel value information of the coordinate points of the fused image, analyze the pixel value information of the coordinate points of the fused image to obtain gradient information in the x-direction and gradient information in the y-direction of the coordinate points, combine the gradient information in the x-direction and the gradient information in the y-direction of the coordinate points to obtain the gradient amplitude feature of the coordinate points, and then analyze the size relationship between the pixel value information of the coordinate points of the fused image and the neighboring pixels to obtain the LBP feature of the coordinate points, and combine the gradient amplitude feature of the coordinate points with the LBP feature of the coordinate points to obtain the extracted feature; The feature extraction module includes template features, and the extracted features are combined with the template features for analysis to obtain target point data.

4. The computer vision-based industrial robot motion control system according to claim 3, characterized in that: Determine the target object through the target point data, and determine the target point coordinate information through the position of the target object in the fused image data; The path planning module sets the robot's position as the starting point to obtain the starting point coordinate information, and then combines the target point coordinate information with the starting point coordinate information to obtain the planned path information. The motion control module controls the robot's movement according to the planned path information.

5. The computer vision-based industrial robot motion control system according to claim 4, characterized in that: The dynamic adjustment module obtains actual path information by real-time monitoring of the robot's motion data, and compares the actual path information with the planned path information to obtain motion difference information; A deviation threshold of one is set for the motion difference information. When the motion difference information is not less than the deviation threshold of one, the robot is adjusted in real time, and the total motion difference information per unit time is obtained based on the motion difference information and combined with the time factor. A deviation threshold of two is set for the total motion difference information per unit time. When the total motion difference information per unit time is not less than the deviation threshold of two, the path information is replanned through the motion control module.

6. The computer vision-based industrial robot motion control system according to claim 5, characterized in that: The industrial robot motion control system further comprises: The data storage module is used to store multi-image data, fused image data, template features, planned path information, actual path information and motion difference information for subsequent analysis.

7. The computer vision-based industrial robot motion control system according to claim 1, characterized in that: The industrial robot motion control system further comprises: Wireless communication module, used to transmit data from each module to terminal devices, so that managers can remotely monitor the system through terminal devices; The wireless communication module is signal-connected to the computer vision module, the feature extraction module, the path planning module, the motion control module and the dynamic adjustment module.

8. The computer vision-based industrial robot motion control system according to claim 1, characterized in that: The computer vision module further includes an image processing unit, which is used to process the multi-image data to improve image quality and obtain processed multi-image data; The image processing unit includes registration processing, filtering processing and denoising processing.

Images