A robot motion trajectory monitoring method based on machine vision

Abstract

The invention discloses a robot motion trajectory monitoring method based on machine vision, relates to the field of data monitoring, and solves the technical problem of robot motion trajectory monitoring. The method can obtain robot motion trajectory data information data; and transmit the obtained data information through a narrowband Internet of Things module NB-IoT module; set up multiple cameras to cross-collect robot motion image 3D and 2D optical markers by mutual projection, calculate imaging parallax and quickly realize the robot motion image acquisition process; apply the NB-IoT module to obtain the angular displacement, angular velocity, angular acceleration, force and torque data information of the robot joint through the serial port, if the data is abnormal, the local alarm is directly issued, and the robot motion image is recognized through binocular stereo machine vision technology, and the three-dimensional coordinates of the robot optical markers in the field of view of multiple cameras are used to realize the edge monitoring of the motion trajectory; the invention can greatly improve the robot motion trajectory monitoring capability.

Description

Technical Field

[0001] The present invention relates to the field of data monitoring, and more specifically to a robot motion trajectory monitoring method based on machine vision. Background Art

[0002] ROS (Robot Operating System) provides a series of libraries and tools to help software developers create robot application software. It provides hardware abstraction, device drivers, library functions, visualization, message passing, and package management. ROS complies with the BSD open source license agreement. Robot kinematics includes forward kinematics and inverse kinematics. Forward kinematics is to calculate the position and posture of the robot end given the joint variables of the robot; inverse kinematics is to calculate all joint variables of the robot corresponding to the position given the position and posture of the robot end.

[0003] In the field of industrial application technology, all industries are inseparable from robot vision. Robot vision means not only taking visual information as input, but also processing this information, and then extracting useful information to provide to the robot. In robot motion trajectory monitoring, conventional calculations usually use monitoring equipment. This method usually uses a monitoring method. CN202110938442.0 discloses a robot stir friction welding trajectory planning method based on offline programming. This method constructs a three-dimensional model; imports the three-dimensional model into the offline programming software, uses the offline programming software to complete the working environment layout of the three-dimensional model, establishes the tool parameters of the robot according to the actual size of the robot fixture, establishes the user coordinate system posture parameters of the robot according to the position of the workpiece to be welded and the welding tooling, and draws a spline curve at the welding position of the workpiece to be welded as a welding trajectory; imports the welding trajectory into the offline programming software that completes the working environment layout, and generates a reference welding trajectory according to the three-dimensional model; the robot controller controls the robot to perform the welding process on the workpiece to be welded, and generates an actual welding trajectory; obtains the deviation value between the actual welding trajectory and the reference welding trajectory, and uses the deviation value to correct the reference welding trajectory, thereby forming a standard welding trajectory. This method modeling method realizes robot motion trajectory monitoring, but cannot realize robot process tracking and evaluation. CN202110525224.4 discloses a robot trajectory determination method, device, equipment and storage medium, the method includes: if it is determined to control the bipedal robot to move from the current step to the first step, then obtain the target motion trajectory of the center of mass of the bipedal robot, the target motion trajectory of the center of mass is obtained by superimposing the motion trajectory of the center of mass in at least two directions, and the motion trajectory in each direction is obtained by using different trajectory planning algorithms; according to the target motion trajectory of the center of mass, the bipedal robot is controlled to move from the current step to the first step, and the movement trajectory of the bipedal robot climbing stairs is realized. Although this method improves the robot motion trajectory monitoring to a certain extent, it cannot achieve image smoothing, the image calculation scale is relatively large, and the calculation efficiency is relatively low. Summary of the invention

[0004] In view of the deficiencies of the above-mentioned technologies, the present invention provides a robot motion trajectory monitoring method based on machine vision, which can improve the efficiency of robot motion trajectory monitoring.

[0005] In order to achieve the above technical effects, the present invention adopts the following technical solutions:

[0006] A robot motion trajectory monitoring method based on machine vision, comprising:

[0007] (S1) obtaining robot motion trajectory data information; and transmitting the obtained data information through a narrowband Internet of Things module NB-IoT module;

[0008] In this step, multiple cameras are set up to cross-collect the robot motion image 3D and 2D optical markers by mutual projection, and the imaging parallax is calculated to quickly realize the robot motion image collection process;

[0009] (S2) After collecting data, it is directly transmitted to the wireless base station through the cloud server.

[0010] The NB-IoT module is used to obtain the angular displacement, angular velocity, angular acceleration, force and torque data of the robot joints through the serial port, and the data is processed. If the data is abnormal, an alarm is directly issued locally, and the data is packaged;

[0011] In this step, the robot motion image is recognized by binocular stereo machine vision technology, and the three-dimensional coordinates of the robot optical markers in the field of view of multiple cameras are processed by an external data processing unit connected wirelessly or by cables. The three-dimensional coordinates of the robot space points are solved according to the perspective transformation formula when the camera coordinate system coincides with the real coordinate points of the robot in space, thereby realizing the edge monitoring of the motion trajectory;

[0012] (S3) Detailed detection of motion trajectory feature points is performed by sampling robot motion image information, and the image information is input into a computer for image preprocessing after the image information is collected. The machine vision recognition algorithm of the mixed edge detection and the histogram equalization algorithm are used to improve the image clarity and the detection accuracy of the segment trajectory feature points by using the cumulative distribution function through the grayscale distribution of the sampled data; at the same time, the robot motion trajectory detection system is provided with multiple fault reset instructions to ensure that the system can be stopped and reset in time if a fault occurs during the operation of the system;

[0013] In this step, the robot trajectory tracking is realized through the perspective transformation function;

[0014] (S4) Output the monitoring results.

[0015] As a further technical solution of the present invention, the controller hardware of the NB-IoT module adopts the ARM+FPGA control method, and the main controller uses the STC12C5A60S2 microcontroller, which integrates 64K program memory and four sixteen-bit timers.

[0016] As a further technical solution of the present invention, the FPGA module includes an XC7K325T-2FFG900C chip, which has 16 high-speed transceivers, and each digital channel has two high-speed transceivers.

[0017] As a further technical solution of the present invention, the image data processing method is:

[0018] Construct a perspective transformation function to realize the perspective transformation when the camera coordinate system coincides with the real coordinate point of the robot in space. j The X-axis coordinate is expressed as:

[0019]

[0020] In formula (1), X1 and Z1 represent the X-axis and Z-axis coordinates of the three-dimensional space point in the actual space, respectively. On another camera, M j The coordinate of the point on the X-axis is represented by X2; M j represents the 3D coordinates of an optical marker of the robot motion; j represents a non-zero scale factor; i represents the number of cameras; (x Ci ,y Ci , z Ci ) represents the projection center C in the local camera reference frame CCS i The coordinates of (u ij , v ij ) represents the corresponding image point P in the projection plane of the local camera reference frame WCS ij 2D coordinates of f Indicates focal length; a i , b i and c i Camera direction angle;

[0021] Three-dimensional space point M j The coordinate points corresponding to the Z axis on the two cameras are:

[0022] X2=X1+B

[0023] Z2=Z1=Z (2)

[0025] In formula (2), B represents the distance between the optical centers of the two cameras. The simplified function is:

[0026] In formula (3), Z represents the distance between the point in the three-dimensional space and the imaging plane, Z1 and Z2 represent the corresponding coordinate points of other cameras in the three-dimensional space, and l f represents the focal length of the camera, and D represents the parallax.

[0027] As a further technical solution of the present invention, a machine vision recognition algorithm method for hybrid edge detection is:

[0028] Step 1: preprocess the collected images;

[0029] Step 2: Perform edge computing on the collected image data information by improving the Canny algorithm;

[0030] When the Canny algorithm is improved to eliminate edge missed detection and pseudo point phenomenon, the robot motion image gradient matrix in the x, y, 45° and 135° directions is calculated through the improved 3×3 8-domain algorithm. The 8-domain algorithm formula is:

[0031]

[0032] In formula (5), H x , H y , H 45 , H 135 Represent the robot motion image gradient matrices in the x, y, 45° and 135° directions respectively. These matrices are used to determine the position M and direction θ of the robot motion trajectory detection node (x, y):

[0033] M(x, y) = max(|H x |,|H y |,|H 45 |,|H 135 |)

[0034]

[0035] Step 3: Use Hough transform method to process image data information;

[0036] Use the histogram equalization algorithm to adjust the grayscale distribution of the sampled data.

[0037] The cumulative distribution function c of the gray level of the robot motion image is defined as:

[0038]

[0039] In formula (7), k represents the grayscale variable, the maximum is L level, p represents the effective area of ​​robot movement, and i represents the ordinal number;

[0040] Step 4: Determine the direction of motion by locating the trajectory points, and then determine the effective area of ​​motion to identify the motion trajectory;

[0041] By comparing the robot motion trajectory points and motion directions of the two edge detection algorithms, high-precision motion trajectory detection data is obtained; the machine vision recognition algorithm of the hybrid edge detection is returned to the database, and then the histogram equalization algorithm is used to distribute the grayscale of the sampled data, and the cumulative distribution function is used to improve the clarity of the image and determine the effective area of ​​the robot motion.

[0042] As a further technical solution of the present invention, when edge computing is performed on the collected image data information by improving the Canny algorithm, the edge computing result is evaluated by a convolutional neural network. The convolutional neural network calculation method is:

[0043] The weighted softmax loss function is used to deal with the residual imbalance of the resampling segment. The training set of the edge detection results of the Canny operator is recorded as {x”, y”} p , the weight β of each performance parameter type is evaluated through the loss function:

[0044]

[0045] In formula (8), P is the total number of performance parameter categories after calculation of the Canny algorithm, m p Represents the energy consumed after each calculation. The weighted softmax loss function loss calculation function is:

[0046]

[0047] In formula (9), sum' is the sample sum of the results calculated by the Canny algorithm.

[0048] The beneficial positive effects of the present invention are:

[0049] Different from conventional technologies, it can obtain robot motion trajectory data information; and transmit the obtained data information through the narrowband Internet of Things module NB-IoT module; set up multiple cameras to cross-collect the robot's motion image 3D and 2D optical markers through mutual projection, calculate the imaging parallax and quickly realize the robot motion image acquisition process; use the NB-IoT module to obtain the angular displacement, angular velocity, angular acceleration, force and torque data information of the robot's joints through the serial port, process the data, and directly alarm locally if the data is abnormal, and identify the robot's motion image through binocular stereo machine vision technology, through the three-dimensional coordinates of the robot's optical markers in the field of view of multiple cameras, the data processing is done by an external data processing unit connected wirelessly or by cable. The three-dimensional coordinates of the robot's spatial points are solved according to the perspective transformation formula when the camera coordinate system and the robot's real coordinate points in space coincide, thereby realizing edge monitoring of the motion trajectory; the motion trajectory feature points are detected in detail by sampling the robot's motion image information, and the image information is input into the computer for image preprocessing after the image collection process, and the machine vision recognition algorithm of the mixed edge detection and the histogram equalization algorithm are used to improve the image clarity by using the cumulative distribution function through the grayscale distribution of the sampled data, thereby improving the detection accuracy of the segment trajectory feature points; at the same time, the robot motion trajectory detection system is provided with multiple fault reset instructions to ensure that the fault problems generated during the system operation can be stopped in time. The present invention can greatly improve the robot motion trajectory monitoring capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0051] Figure 1 It is a schematic diagram of the process of the present invention;

[0052] Figure 2 It is a schematic diagram of a specific embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the robot motion trajectory image acquisition method in the present invention;

[0054] Figure 4 This is a schematic diagram of the controller principle of the B-IoT module in the present invention;

[0055] Figure 5 Schematic diagram of the controller principle of the B-IoT module in the present invention. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0057] like Figure 1-Figure 3 As shown, a robot motion trajectory monitoring method based on machine vision includes:

[0058] (S1) obtaining robot motion trajectory data information; and transmitting the obtained data information through a narrowband Internet of Things module NB-IoT module;

[0059] In this step, multiple cameras are set up to cross-collect the robot motion image 3D and 2D optical markers by mutual projection, and the imaging parallax is calculated to quickly realize the robot motion image collection process;

[0060] (S2) After collecting data, it is directly transmitted to the wireless base station through the cloud server, which simplifies the on-site deployment and reduces the implementation cost; the NB-IoT module is used to obtain the angular displacement, angular velocity, angular acceleration, force and torque data information of the robot joint through the serial port, and the data is processed. If the data is abnormal, the local alarm is directly issued and the data is packaged;

[0061] In this step, the robot motion image is recognized by binocular stereo machine vision technology, and the three-dimensional coordinates of the robot optical markers in the field of view of multiple cameras are processed by an external data processing unit connected wirelessly or by cables. The three-dimensional coordinates of the robot space points are solved according to the perspective transformation formula when the camera coordinate system coincides with the real coordinate points of the robot in space, thereby realizing the edge monitoring of the motion trajectory;

[0062] In a specific embodiment, the management platform system transmits the packaged data to the NB-IoT Internet of Things communication module through the serial port. The monitoring data network transmission adopts the wireless transmission control protocol, creates a socket communication interface through low-power wide area network technology, establishes a connection with the cloud service, and performs data analysis on the data packets received by the cloud service. The data is transmitted to the corresponding table of the MySQL database, and the historical monitoring data can be queried. The MySQL database is visually managed using the database management software, and the user can analyze and process the data to complete the monitoring of the robot's motion trajectory.

[0063] (S3) Detailed detection of motion trajectory feature points is performed by sampling robot motion image information, and the image information is input into a computer for image preprocessing after the image information is collected. The machine vision recognition algorithm of the mixed edge detection and the histogram equalization algorithm are used to improve the image clarity and the detection accuracy of the segment trajectory feature points by using the cumulative distribution function through the grayscale distribution of the sampled data; at the same time, the robot motion trajectory detection system is provided with multiple fault reset instructions to ensure that the system can be stopped and reset in time if a fault occurs during the operation of the system;

[0064] In this step, the robot trajectory tracking is realized through the perspective transformation function;

[0065] (S4) Output the monitoring results.

[0066] In the above embodiment, during the operation of the narrowband Internet of things (NB-IoT) module,

[0067] In the above embodiment, the controller hardware of the NB-IoT module adopts the control mode of ARM+FPGA, and the main controller uses the STC12C5A60S2 single-chip microcomputer, which integrates 64K program memory and four sixteen-bit timers. During communication control, in a specific embodiment, a custom bus controller, a data flow controller, and an SRIO controller are also set up, and the interaction of different data information is realized through the channel sub-module. The power module provides working voltage to different modules, and the clock module provides clock pulses. The debugging module provides debugging information, and the memory configuration module provides storage information. The module realizes data information conversion through the JTAG interface and the UART bridge, and realizes data information transfer through the bus switch. Switch control and data information storage are realized through the bus switch and linear flash memory, and the connection of different channels is realized through the channel connector.

[0068] In the above embodiment, the FPGA module uses the XC7K325T-2FFG900C chip, which has 16 high-speed transceivers, and each digital channel has two high-speed transceivers.

[0069] In the specific embodiment, an FPGA controls the test of 8 digital channels. The development board has 8 GPIO LED indicators, a CPU reset button, 4 DIP switches, and a drive rotary encoder switch. The communication module uses the CC2530 communication chip, which contains a high-quality RF transceiver and 51 core, and the internal data memory capacity is 8KB

[11] The data channel controller in the communication controller is combined with a custom communication protocol, and the data sent by the data sender includes the data header of the control signal and the protocol data packet.

[0070] In the above embodiment, the image data processing method is:

[0071] like Figure 3 As shown, Figure 3 The robot motion trajectory image acquisition method is demonstrated.

[0072] The image acquisition device uses an organic light-emitting semiconductor display screen. The research uses binocular stereo machine vision technology to identify the robot's motion image. The three-dimensional coordinates of the robot's optical markers in the "field of view" of multiple cameras are processed by an external data processing unit connected wirelessly or by cables. The perspective transformation function is constructed to achieve the perspective transformation when the camera coordinate system coincides with the robot's real coordinate point in space. The three-dimensional space point M in the perspective transformation function j The X-axis coordinate is expressed as:

[0073]

[0074] In formula (1), X1 and Z1 represent the X-axis and Z-axis coordinates of the three-dimensional space point in the actual space, respectively. On another camera, M j The coordinate of the point on the X-axis is represented by X2; M j represents the 3D coordinates of an optical marker of the robot motion; j represents a non-zero scale factor; i represents the number of cameras; (x Ci ,y Ci , z Ci ) represents the projection center C in the local camera reference frame CCS i The coordinates of (u ij , v ij ) represents the corresponding image point P in the projection plane of the local camera reference frame WCS ij 2D coordinates of f Indicates focal length; a i , b i and c i Camera direction angle;

[0075] Three-dimensional space point M j The coordinate points corresponding to the Z axis on the two cameras are:

[0076] X2=X1+B

[0077] Z2=Z1=Z (2)

[0078] In formula (2), B represents the distance between the optical centers of the two cameras. The simplified function is:

[0079]

[0080] In formula (3), Z represents the distance between the point in the three-dimensional space and the imaging plane, Z1 and Z2 represent the corresponding coordinate points of other cameras in the three-dimensional space, and l f represents the focal length of the camera, and D represents the parallax. The size of the parallax can be used to measure the coordinates of an object in three-dimensional space, and can also calculate the distance between the two cameras and the center point of the robot cursor.

[0081] The present application is based on a machine vision recognition algorithm of hybrid edge detection. The edge detection combines the Hough transform with the improved Canny algorithm. The Hough transform defines a straight line in the robot motion image as a set of collinear points and maps all motion trajectory detection nodes to the function space of the sine function.

[0082] The machine vision recognition algorithm method for hybrid edge detection is:

[0083] Step 1: pre-process the collected image; map all motion trajectory detection nodes to the function space of the sine function;

[0084] f: (x, y)→p=xcos(θ)+ysin(θ) (4)

[0085] In formula (4), (x, y) represents the coordinate position of the characteristic point of the robot motion trajectory, p and θ represent the vertical distance between the robot motion line and the coordinate center and the angle between the normal line of the line and the X-axis, respectively, to determine the characteristic point and motion direction of the robot motion trajectory;

[0086] Step 2: Perform edge computing on the collected image data information by improving the Canny algorithm;

[0087] When the Canny algorithm is improved to eliminate edge missed detection and pseudo point phenomenon, the robot motion image gradient matrix in the x, y, 45° and 135° directions is calculated through the improved 3×3 8-domain algorithm. The 8-domain algorithm formula is:

[0088]

[0089] In formula (5), H x , H y , H 45 , H 135 Represent the robot motion image gradient matrices in the x, y, 45° and 135° directions respectively. These matrices are used to determine the position M and direction θ of the robot motion trajectory detection node (x, y):

[0090] M(x, y) = max(|H x |,|H y |,|H 45 |,|H 135 |)

[0091]

[0092] Step 3: Use Hough transform method to process image data information;

[0093] The grayscale distribution of the sampled data is analyzed using the histogram equalization algorithm. The image clarity is improved using the cumulative distribution function to determine the effective area of ​​robot motion. The cumulative distribution function c of the grayscale of the robot motion image is defined as:

[0094]

[0095] In formula (7), k represents the grayscale variable, the maximum is L level, p represents the effective area of ​​robot movement, and i represents the ordinal number;

[0096] The grayscale distribution of the image collected by the dial is accurately controlled through the image grayscale distribution function of formula (4), and the image clarity is improved by using the cumulative distribution function. The enhanced image information is then output to the robot system motion trajectory detection system and displayed in a digital form by the management platform user.

[0097] Step 4: Determine the direction of motion by locating the trajectory points, and then determine the effective area of ​​motion to identify the motion trajectory;

[0098] By comparing the robot motion trajectory points and motion directions of the two edge detection algorithms, high-precision motion trajectory detection data is obtained; the machine vision recognition algorithm of the hybrid edge detection is returned to the database, and then the grayscale distribution of the sampled data is analyzed using the histogram equalization algorithm, and the image clarity is improved using the cumulative distribution function to determine the effective area of ​​the robot motion;

[0099] Step 5: Output motion trajectory data information

[0100] In conventional technology, the traditional Canny operator is prone to losing edge details when performing edge detection. An improved Canny edge detection algorithm is proposed. First, the traditional Canny idea and defects are analyzed from the perspective of mathematical morphology; then, a Gaussian filter with adaptive scale adjustment is proposed to improve the traditional Gaussian smoothing filter; then the maximum inter-class difference method (Otsu) is used to replace the traditional double threshold detection algorithm to find the optimal threshold, effectively balancing the contradiction between denoising ability and retaining edge detail information; in order to solve the shortcomings of Otsu's long traversal time and poor real-time performance, it is proposed to use the Kalman filter minimum error estimation algorithm to pre-estimate the threshold range of subsequent image sequences to speed up the traversal process.

[0101] In a further embodiment, the Canny operator is output by the following code.

[0102]

[0103]

[0104] In the above steps, when edge computing is performed on the collected image data information by improving the Canny algorithm, the edge computing result is evaluated by the convolutional neural network. The convolutional neural network calculation method is:

[0105] The weighted softmax loss function is used to deal with the residual imbalance of the resampling segment. The training set of the edge detection results of the Canny operator is recorded as {x”, y”} p , the weight β of each performance parameter type is evaluated through the loss function:

[0106]

[0107] In formula (8), P is the total number of performance parameter categories after calculation of the Canny algorithm, m p Represents the energy consumed after each calculation. The weighted softmax loss function loss calculation function is:

[0108]

[0109] In formula (9), sum' is the sample sum of the results calculated by the Canny algorithm.

[0110] In a specific embodiment, in order to accurately evaluate the easily confused Canny algorithm performance parameter types, a progressive method with multiple SDCNNs with different input features was developed, and the learning process of the algorithm can be divided into two parts: in the first part, the SDCNN-1 for diagnosing the entire performance parameter type is trained using the training set of the first signal, and then the confused performance parameter types are determined based on the confusion matrix and the sensitivity of each performance parameter type; in the second part, SDCNN-2 is trained using the second signal set of the confused performance parameter types determined in the first part, and then the prediction results in SDCNN-2 are synthesized with the results in SDCNN-1 to make a final evaluation of the overall performance parameters of the acquisition device.

[0111] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these specific embodiments are only illustrative, and those skilled in the art may omit, replace, and change the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, merging the above method steps so as to perform substantially the same functions in substantially the same manner to achieve substantially the same results is within the scope of the present invention. Therefore, the scope of the present invention is limited only by the appended claims.

Claims

1. A robot motion trajectory monitoring method based on machine vision, characterized in that: include: (S1) Obtaining robot motion trajectory data information; The acquired data information is transmitted through the narrowband Internet of Things module NB-IoT module; In this step, multiple cameras are set up to cross-collect the robot motion image 3D and 2D optical markers by mutual projection, and the imaging parallax is calculated to quickly realize the robot motion image collection process; (S2) After collecting data, it is directly transmitted to the wireless base station through the cloud server. The NB-IoT module is used to obtain the angular displacement, angular velocity, angular acceleration, force and torque data of the robot joints through the serial port, and the data is processed. If the data is abnormal, an alarm is directly issued locally, and the data is packaged; In this step, the robot motion image is recognized by binocular stereo machine vision technology, and the three-dimensional coordinates of the robot optical markers in the field of view of multiple cameras are processed by an external data processing unit connected wirelessly or by cables. The three-dimensional coordinates of the robot space points are solved according to the perspective transformation formula when the camera coordinate system coincides with the real coordinate points of the robot in space, thereby realizing the edge monitoring of the motion trajectory; (S3) Detailed detection of motion trajectory feature points is performed by sampling robot motion image information. After the image information is collected, it is input into a computer for image preprocessing. The machine vision recognition algorithm of the mixed edge detection and the histogram equalization algorithm are used to improve the image clarity and the detection accuracy of the segment trajectory feature points by using the cumulative distribution function through the grayscale distribution of the sampled data. At the same time, the robot motion trajectory detection system is equipped with multiple fault reset instructions to ensure that the system can be stopped and reset in time if a fault occurs during system operation. In this step, the robot trajectory tracking is realized through the perspective transformation function; (S4) Output the monitoring results.

2. The method for monitoring robot motion trajectory based on machine vision according to claim 1, characterized in that: The controller hardware of the NB-IoT module adopts the ARM+FPGA control method. The main controller uses the STC12C5A60S2 microcontroller, which integrates 64K program memory and four 16-bit timers.

3. The method for monitoring robot motion trajectory based on machine vision according to claim 2, characterized in that: The FPGA module contains the XC7K325T-2FFG900C chip, which has 16 high-speed transceivers, with two high-speed transceivers for each digital channel.

4. The method for monitoring robot motion trajectory based on machine vision according to claim 1, characterized in that: The image data processing method is: Construct a perspective transformation function to realize the perspective transformation when the camera coordinate system and the robot's real coordinate point in space coincide with each other. M j of X The axis coordinates are expressed as: （1） In formula (1), X 1 and Z 1 respectively represents the three-dimensional space point in the actual space X Axis and Z The coordinates of the axis, on the other camera, M j Point X The axis coordinates are expressed as X 2; M j The 3D coordinates of an optical marker representing the robot's motion; j represents a non-zero scale factor; i Indicates the number of cameras; ( x Ci , y Ci , z Ci ) represents the projection center in the local camera reference frame CCS C i The coordinates of u ij , v ij ) represents the corresponding image point in the WCS projection plane of the local camera reference frame P ij 2D coordinates of l f Indicates focal length; a point in three-dimensional space M j The coordinate points corresponding to the Z axis on the two cameras are: （2） In formula (2), B Represents the distance between the optical centers of the two cameras. The simplified function is: （3） In formula (3), Z Represents the distance between a point in three-dimensional space and the imaging plane, and Indicates the coordinate points corresponding to other cameras in three-dimensional space, Indicates the focal length of the camera. D Indicates parallax.

5. The method for monitoring robot motion trajectory based on machine vision according to claim 1, characterized in that: The machine vision recognition algorithm method for hybrid edge detection is: Step 1: preprocess the collected images; Step 2: Perform edge computing on the collected image data information by improving the Canny algorithm; When improving the Canny algorithm to eliminate edge omissions and pseudo-points, the improved 3×3 8-field algorithm is used to calculate x , y , 45° and 135° directions of the robot motion image gradient matrix, the 8-domain algorithm formula is: （5） In formula (5), H x , H y , H 45 , H 135 Respectively x , y , 45° and 135° directions of the robot motion image gradient matrix, through which the robot motion trajectory detection nodes are determined ( x , y ) M With direction θ : （6） Step 3: Use Hough transform method to process image data information; Use the histogram equalization algorithm to adjust the grayscale distribution of the sampled data. Cumulative distribution function of gray level of robot motion image c Defined as: （7） In formula (7), k Represents the grayscale variable, the maximum is L class, p Indicates the effective area of ​​robot movement. i Indicates ordinal number; Step 4: Determine the direction of motion by locating the trajectory points, and then determine the effective area of ​​motion to identify the motion trajectory; By comparing the robot motion trajectory points and motion directions of the two edge detection algorithms, high-precision motion trajectory detection data is obtained; the machine vision recognition algorithm of the hybrid edge detection is returned to the database, and then the histogram equalization algorithm is used to distribute the grayscale of the sampled data, and the cumulative distribution function is used to improve the clarity of the image and determine the effective area of ​​the robot motion.

6. The method for monitoring robot motion trajectory based on machine vision according to claim 5, characterized in that: When edge computing is performed on the collected image data information by improving the Canny algorithm, the edge computing result is evaluated by the convolutional neural network. The convolutional neural network calculation method is: The weighted softmax loss function is used to deal with the residual imbalance of the resampling segment. The training set of the edge detection results of the Canny operator is denoted as { x '', y ''} p , the weight of each performance parameter type is evaluated through the loss function β : （8） In formula (8), P is the total number of performance parameter categories of the Canny algorithm after calculation, Represents the energy consumed after each calculation, weighted softmax loss function loss The calculation function is: （9） In formula (9), sum 'Calculates the sample sum of the results of the Canny algorithm.

Images