Video intelligent assistance-based rail-lying type robot control method and system

Abstract

The invention relates to the technical field of intelligent control, in particular to a method and a system for controlling a prone track type robot based on video intelligent assistance, which are characterized in that a video stream acquires a track texture feature map, a difference between adjacent windows of the map is called to calculate a transverse gradient screening offset, and a projection deviation compensation parameter is calculated based on an offset registration contour; and calculating a path cost value optimal sequence by combining abnormal region weighting, monitoring a sequence state, calculating a pulse width by combining encoder feedback and a steering speed, and outputting a closed-loop instruction. According to the method, a texture map is obtained through a video, contrast energy parameter windows are extracted through a gray-level co-occurrence matrix, standard deviation is calculated, transverse gradient screening offset based on cosine similarity is obtained through difference of adjacent windows, centroid projection deviation compensation parameters are calculated through seam identification registration, and a path sequence is generated through Euclidean Manhattan weighting of abnormal area coordinate compensation parameters. And the monitoring path state encoder feeds back the steering speed, calculates the pulse width and outputs a closed-loop instruction.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and particularly to a control method and system for a rail-mounted robot based on video intelligent assistance. Background Art

[0002] The field of intelligent control technology includes key technical elements such as the design of automatic control systems, the processing of sensor signals, and the driving of actuators. Its core lies in achieving precise motion regulation of the controlled object through real-time data acquisition, processing, and feedback. It involves industrial automation production lines, intelligent logistics systems, and the control scenarios of rail-mounted mobile devices. Especially in the application scenarios of rail robots, it is necessary to comprehensively handle multi-dimensional technical problems such as environmental perception, path planning, and motion control, and the system integration requirements are relatively high.

[0003] Among them, the control method for a rail-mounted robot based on video intelligent assistance refers to a technical solution that obtains rail contour data through a vision sensor, establishes a robot pose model using feature point matching technology, and generates drive commands in combination with preset control parameters. It specifically includes three technical links: rail feature recognition, pose deviation calculation, and motion parameter correction. By establishing the mapping relationship between the image coordinate system and the motion coordinate system, and using the proportional-integral-derivative algorithm to output the adjustment amount of the motor speed, continuous motion control along the preset track is finally achieved.

[0004] The prior art relies on fixed feature point matching to establish a pose model, without considering the dynamic changes of textures, resulting in matching errors when the surface is worn or the illumination is uneven. The traditional proportional-integral-derivative algorithm uses static control parameters and is difficult to adapt to the adjustment requirements of variable-speed working conditions, leading to motion overshoot or response delay. The preset trajectory reference angle lacks dynamic adaptation to the geometric features of rail joints, and the pose calculation deviation increases when there are construction errors or thermal deformations. The existing path planning does not associate the coordinates of abnormal areas with motion parameters, and when encountering temporary obstacles, it can only perform emergency braking and cannot generate an optimized path. The fusion of encoder feedback and visual data is insufficient, and there is a timing deviation between control commands and execution actions during high-speed motion, affecting the system stability. For example, when the rail is covered with oil stains, the traditional feature recognition fails and causes deviation, and the prior art lacks an abnormal handling mechanism, resulting in potential safety hazards. Summary of the Invention

[0005] In order to solve the technical problems existing in the prior art, an embodiment of the present invention provides a control method and system for a rail-mounted robot based on video intelligent assistance. The technical solution is as follows:

[0006] A control method for a rail-mounted robot based on video intelligent assistance includes the following steps:

[0007] S1: Obtain the texture features of the track surface. Calculate the gray-level co-occurrence matrix by computing the pixel spacing and direction angle, extract the contrast and energy parameters in the matrix, divide the detection window along the track extension direction and calculate the standard deviation, and generate the track texture feature distribution map;

[0008] S2: Call the eigenvalue of adjacent windows in the track texture feature distribution map, obtain the horizontal gradient vector through differential operation, calculate the cosine similarity of the vector direction angle, and screen the cosine values not exceeding the similarity threshold to generate the hub steering angle correction amount;

[0009] S3: Based on the offset parameter in the hub steering angle correction amount, register the contour coordinates of the track joint identifier, calculate the projection component of the centroid offset, and generate the grasping pose compensation parameter;

[0010] S4: Combine the abnormal area coordinates of the track texture feature distribution map and the projection deviation in the grasping pose compensation parameter to construct the weights of the Euclidean distance and Manhattan distance, and calculate the weighted value to generate the multi-modal optimal path sequence;

[0011] S5: Monitor the execution status of the path nodes in the multi-modal optimal path sequence, calculate the steering pulse width through the steering angle and the current speed value, combine the encoder feedback to obtain the closed-loop control signal, and output the drive wheel closed-loop control instruction.

[0012] As a further solution of the present invention, the track texture feature distribution map is specifically the contrast distribution, energy distribution, and window standard deviation range. The hub steering angle correction amount includes the horizontal offset, angle deviation amount, and threshold trigger identifier. The grasping pose compensation parameter is specifically the coordinate offset, projection deviation amount, and pose adjustment vector. The multi-modal optimal path sequence includes the path cost value matrix, weight distribution coefficient, and node priority sequence. The drive wheel closed-loop control instruction is specifically the steering angle setting value, speed adjustment parameter, and PWM duty cycle.

[0013] As a further solution of the present invention, the steps for obtaining the track texture feature distribution map are as follows:

[0014] S101: Obtain the video stream track surface image data, detect the pixel spacing and direction angle distribution features, establish a spatial coordinate system along the track extension direction, map the pixel spacing data at different angles into the coordinate system, and generate the track texture space mapping set;

[0015] S102: Call the track texture space mapping set, extract the contrast and energy parameters of each coordinate point, establish the dynamic correlation relationship between the parameters, divide the continuous detection units along the track extension direction, calculate the parameter fluctuation amplitude within the unit, and generate the texture feature fluctuation sequence;

[0016] S103: Based on the texture feature fluctuation sequence, statistically calculate the parameter change gradient between adjacent units, construct a gradient distribution heat map, mark the boundary coordinates of the gradient mutation region, integrate the corresponding relationship between the spatial coordinates and the gradient values, and generate an orbital texture feature distribution map.

[0017] As a further solution of the present invention, the obtaining step of the hub steering angle correction amount is as follows:

[0018] S201: Invoke the orbital texture feature distribution map, extract the feature values of adjacent windows, and use the formula:

[0019] ;

[0020] Calculate the lateral gradient vector and generate a dynamic gradient vector set;

[0021] Wherein, represents the comprehensive gradient intensity, represents the feature value of the i-th window, is the sliding average of the vector direction angle, is the trajectory reference angle, is the direction angle range;

[0022] S202: Based on the dynamic gradient vector set, calculate the cosine similarity between each vector direction angle and the trajectory reference angle, compare the similarity value with the set threshold, screen out the abnormal vectors and establish a spatial coordinate index to generate an offset abnormal vector set;

[0023] S203: Invoke the offset abnormal vector set, generate a steering angle correction amount according to the product relationship between the lateral gradient vector modulus length and the direction angle deviation value, integrate the correction amount data, and generate a hub steering angle correction amount.

[0024] As a further solution of the present invention, the obtaining step of the grasping pose compensation parameter is as follows:

[0025] S301: Invoke the hub steering angle correction amount, extract the offset parameter, match the corner coordinates of the visual identifier of the track seam in the current video frame, and use the formula:

[0026] ;

[0027] Calculate the feature point registration accuracy and generate a registration feature point set;

[0028] Wherein, is the feature point registration accuracy, is the corner coordinate of the current frame, is the reference identifier coordinate, is the corner neighborhood area, is the ratio of the current contour perimeter to the standard perimeter, is the angular difference between the current frame and the reference frame;

[0029] S302: Based on the set of registered feature points, calculate the centroid coordinate offset of each feature point, establish a projection coordinate system along the track extension direction, decompose the offset into horizontal and vertical components, and generate the centroid offset projection components;

[0030] S303: Invoke the centroid offset projection components, linearly combine the horizontal and vertical components according to the degrees of freedom weight of the grasping mechanism, and superimpose the device base attitude compensation parameters to generate the grasping pose compensation parameters.

[0031] As a further solution of the present invention, the steps for obtaining the multimodal optimal path sequence are as follows:

[0032] S401: Invoke the coordinates of the abnormal regions in the track texture feature distribution map, calculate the Euclidean distance between the center point of each region and the grasping reference point, and use the formula:

[0033] ;

[0034] Construct the Euclidean distance weight to generate a dynamic Euclidean weight set;

[0035] where is the Euclidean distance weight, are the coordinates of the abnormal region, is the grasping reference point, is the magnitude of the projection deviation vector, is the maximum value of the current projection deviation magnitude;

[0036] S402: Invoke the projection deviation components in the grasping pose compensation parameters, decompose the absolute values of the horizontal and vertical deviations along the track coordinate system, and superimpose the component values according to the Manhattan distance calculation rule to generate the Manhattan distance weight coefficient;

[0037] S403: Based on the dynamic Euclidean weight set and the Manhattan distance weight coefficient, calculate the path cost value by weighted proportional summation, and traverse the candidate paths using a path planning algorithm to generate a multimodal optimal path sequence.

[0038] As a further solution of the present invention, the steps for obtaining the drive wheel closed-loop control instruction are as follows:

[0039] S501: Monitor the steering angle and the current speed value of the path nodes in the multimodal optimal path sequence, multiply the angle value by the speed value, calculate the pulse width reference quantity, and superimpose the device vibration compensation coefficient to generate a dynamic pulse width set;

[0040] S502: Call the dynamic pulse width set, calculate the pulse width correction amount in combination with the wheel speed deviation value fed back by the encoder, establish a linear relationship between the pulse width and the wheel speed deviation, and generate closed-loop control signal parameters;

[0041] S503: Based on the closed-loop control signal parameters, encode the steering angle and speed value according to the drive wheel control protocol, and fuse the safety protection threshold to generate a drive wheel closed-loop control instruction.

[0042] A crawler robot control system based on video intelligent assistance, the system includes: a texture feature analysis module, a lateral offset detection module, a pose compensation generation module, a multi-modal path planning module, and a drive closed-loop control module;

[0043] The texture feature analysis module is used to obtain the pixel spacing and direction angle of the track surface through the video stream, input the pixel spacing and direction angle into the gray-level co-occurrence matrix to extract the contrast parameter and energy parameter, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation within the window, generate a texture feature distribution map and transfer it to the lateral offset detection module and the multi-modal path planning module;

[0044] The lateral offset detection module is used to call the feature values of adjacent windows in the texture feature distribution map, perform a difference operation on the adjacent feature values to generate a lateral gradient vector, input the vector direction angle and the preset track reference angle into the cosine similarity calculation, generate a lateral offset when the calculation result is lower than the set threshold, and transfer the lateral offset to the pose compensation generation module;

[0045] The pose compensation generation module is used to match the contour coordinates of the track seam identifier in the current video frame based on the lateral offset, align the geometric features of the identifier using the feature point registration algorithm, calculate the projection component of the centroid offset on the motion plane, generate the grasping pose compensation parameters and transfer them to the multi-modal path planning module;

[0046] The multi-modal path planning module is used to call the abnormal area coordinates in the texture feature distribution map and the projection deviation in the grasping pose compensation parameters, perform Euclidean distance weighting calculation on the abnormal area coordinates, perform Manhattan distance weighting calculation on the projection deviation, input the two weights into the weighted sum to generate the path cost value, output the multi-modal optimal path sequence and transfer it to the drive closed-loop control module;

[0047] The drive closed-loop control module is used to monitor the execution status of the path nodes in the multi-modal optimal path sequence, multiply the steering angle by the current speed value to calculate the pulse width, combine the encoder feedback value and input it into the PID control algorithm to generate a closed-loop control signal, and output a drive wheel closed-loop control instruction including the steering angle and speed value.

[0048] The beneficial effects brought by the technical solution provided by the embodiments of the present invention at least include:

[0049] In the present invention, the texture features of the track surface are used to extract the contrast and energy parameters through the gray-level co-occurrence matrix. The detection window is divided in the extension direction to calculate the standard deviation difference, enhancing the parsing ability of complex textures. The lateral gradient vector difference operation of the feature values of adjacent windows is combined with the cosine similarity threshold determination to improve the lateral offset detection accuracy and reduce the accumulation of pose deviations at the joints. The geometric feature registration of the visual identifier uses the centroid offset projection component calculation to dynamically adapt the grasping pose compensation parameters and solve the feature misalignment caused by changes in illumination. The Euclidean distance and Manhattan distance are weighted to generate the path cost value, and a multi-modal path sequence is constructed to achieve the coordination of abnormal area avoidance and trajectory optimization. The pulse width and encoder feedback are fused to form a closed-loop control signal, and the steering angle and speed values are dynamically coupled and adjusted to improve the stability of the variable-speed motion trajectory tracking. This solution integrates texture analysis, gradient operation, geometric registration, and multi-modal planning, enhancing the environmental adaptability and control accuracy on the basis of real-time response. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is the flowchart of the method of the present invention;

[0051] Figure 2 is the flowchart for obtaining the distribution diagram of the track texture features of the present invention;

[0052] Figure 3 is the flowchart for obtaining the correction amount of the hub steering angle of the present invention;

[0053] Figure 4 is the flowchart for obtaining the grasping pose compensation parameters of the present invention;

[0054] Figure 5 is the flowchart for obtaining the multi-modal optimal path sequence of the present invention;

[0055] Figure 6 is the flowchart for obtaining the closed-loop control instruction of the drive wheel of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The technical solutions in the present invention will be described below with reference to the accompanying drawings.

[0057] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two can be selected.

[0058] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, their intended meanings are the same. "of", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, their intended meanings are the same.

[0059] In the embodiments of the present invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, their intended meanings are the same.

[0060] To make the technical problems, technical solutions, and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0061] Please refer to Figure 1 , the present invention provides a technical solution: a control method for a track-lying robot based on video intelligent assistance, including the following steps:

[0062] S1: Obtain the texture features of the track surface in the video stream, generate a gray-level co-occurrence matrix by calculating the pixel spacing and direction angle, extract the contrast parameter and energy parameter in the matrix, divide the detection window along the track extension direction, calculate the maximum difference in the standard deviation of the texture features within the window, and generate a track texture feature distribution map;

[0063] S2: Call the eigenvalue of the adjacent window in the track texture feature distribution map, obtain the horizontal gradient vector through the differential operation of the eigenvalue of the adjacent window, calculate the cosine similarity between the vector direction angle and the preset trajectory reference angle, and generate a hub steering angle correction amount including the horizontal offset when the cosine value is lower than the set threshold;

[0064] S3: Based on the offset parameter in the hub steering angle correction amount, match the geometric features of the track joint visual identifier in the current video frame, align the contour coordinates of the identifier using the feature point registration algorithm, calculate the projection component of the centroid offset, and generate a grasping pose compensation parameter;

[0065] S4: Combine the abnormal area coordinates in the track texture feature distribution map with the projection deviation in the grasping pose compensation parameter, construct an Euclidean distance weight based on the abnormal area coordinates, combine the projection deviation to generate a Manhattan distance weight, calculate the path cost value through weighted summation, and generate a multi-modal optimal path sequence;

[0066] S5: Monitor the execution status of the path nodes in the multi-modal optimal path sequence, calculate the pulse width by multiplying the steering angle by the current speed value, combine the encoder feedback value to generate a closed-loop control signal, and output a driving wheel closed-loop control instruction including the steering angle and speed value.

[0067] The distribution map of track texture features specifically includes contrast distribution, energy distribution, and window standard deviation range. The hub steering angle correction amount includes lateral offset, angular deviation, and threshold trigger identification. The grasping pose compensation parameters specifically include coordinate offset, projection deviation, and attitude adjustment vector. The multi-modal optimal path sequence includes path cost value matrix, weight distribution coefficient, and node priority sequence. The driving wheel closed-loop control instruction specifically includes steering angle set value, speed adjustment parameter, and PWM duty cycle.

[0068] Please refer to Figure 2 , and the steps for obtaining the distribution map of track texture features are as follows:

[0069] S101: Obtain the video stream track surface image data, detect the pixel spacing and the distribution characteristics of the direction angle, establish a spatial coordinate system along the track extension direction, map the pixel spacing data at different angles into the coordinate system, and generate a track texture space mapping set;

[0070] When obtaining the video stream track surface image data, an industrial camera is used to collect the track surface images at a rate of 30 frames per second. When detecting the pixel spacing, the Euclidean distance between adjacent track texture feature points is selected as the spacing value. For example, when detecting that the distance between adjacent rivets is 15 pixels, when calculating the distribution characteristics of the direction angle, the track extension direction is used as the reference axis, and the angle between the connection line of each texture feature point and the reference axis is measured. When it is detected that the connection line of a certain feature point forms a 45-degree angle with the reference axis, it is classified into the 45-degree direction group. When establishing the spatial coordinate system, the track starting point is used as the origin, the extension direction is the positive direction of the X-axis, and the vertical direction is the Y-axis. The 0-degree direction spacing data is mapped to the X-axis coordinate interval, and the 90-degree direction data is mapped to the Y-axis coordinate interval. For example, the 15-pixel spacing in the 0-degree direction is mapped to 15 unit lengths on the X-axis in the coordinate system, and the 21.21-pixel spacing in the 45-degree direction is decomposed into 15 units on the X-axis and 15 units on the Y-axis. Through coordinate mapping, the texture space data conversion is completed, and a track texture space mapping set is generated.

[0071] S102: Call the track texture space mapping set, extract the contrast and energy parameters of each coordinate point, establish a dynamic correlation relationship between the parameters, divide continuous detection units along the track extension direction, calculate the parameter fluctuation amplitude within the unit, and generate a texture feature fluctuation sequence;

[0072] When calling the track texture space mapping set, extract the texture point contrast parameter with coordinates (120, 80). Calculate that the maximum gray level difference within the 3×3 neighborhood of this point is 85. For the energy parameter calculation, use the gray level variance value of the 5×5 pixel area around this point, and obtain the energy parameter as 0.67. When establishing the dynamic association relationship, linearly combine the contrast parameter and the energy parameter with a weight of 0.6:0.4. For example, when the contrast of a certain point is 0.8 and the energy is 0.5, the combined parameter value is 0.68. When dividing the detection unit, divide the unit every 50 pixels along the X-axis. Use the range method to calculate the parameter fluctuation amplitude within the unit. For example, within the 5th unit, the maximum comprehensive parameter is 0.75, the minimum is 0.62, and the fluctuation amplitude is 0.13, generating a texture feature fluctuation sequence.

[0073] S103: Based on the texture feature fluctuation sequence, statistically analyze the parameter change gradient between adjacent units, construct a gradient distribution heat map, mark the boundary coordinates of the gradient mutation area, integrate the corresponding relationship between the spatial coordinates and the gradient values, and generate a track texture feature distribution map;

[0074] When based on the texture feature fluctuation sequence, statistically analyze the parameter change gradient between adjacent units, and calculate using the backward difference method. For example, the gradient between the parameter of unit 3 (0.68) and the parameter of unit 4 (0.72) is 0.04 / 50 pixels = 0.0008 per pixel. When constructing the gradient distribution heat map, map the gradient value of 0.0008 to a hue of 120 degrees (green) in the HSV color space. When detecting that the gradient mutation from unit 7 to unit 8 is 0.002, the corresponding hue becomes 0 degrees (red). When marking the boundary of the mutation area, determine the starting coordinate of unit 7 (350, 0) and the ending coordinate of unit 8 (400, 120). When integrating the spatial coordinates and the gradient values, establish the coordinate (375, 60) corresponding to the gradient peak value of 0.0025, generating a track texture feature distribution map.

[0075] Please refer to Figure 3 , the steps for obtaining the hub steering angle correction amount are as follows:

[0076] S201: Call the track texture feature distribution map, extract the feature values of adjacent windows, and use the formula:

[0077] ;

[0078] Calculate the horizontal gradient vector to generate a dynamic gradient vector set;

[0079] Among them, represents the comprehensive gradient intensity, represents the feature value of the i-th window, represents the difference between the feature values of adjacent windows, is the total number of windows, is the sliding average of the vector direction angle, is the trajectory reference angle, is the direction angle range;

[0080] When calling the track texture feature distribution map, the detection area is divided with a window width of 50 pixels along the track extension direction. When extracting the window feature value, calculate the arithmetic mean of the gradient values in each window. For example, window 1 contains gradient values 0.15, 0.18, 0.22, and the feature value , window 2 feature value , window 3 feature value , window 4 feature value , and the difference calculation of adjacent window feature values is as follows:

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] The calculation of the sliding average of the direction angle adopts a 3-window slide. The direction angles of windows 1-3 are 45°, 47°, and 49° respectively: ;

[0086] Track reference angle , direction angle range , substitute into the formula for complete calculation:

[0087] ;

[0088] Table 1 Example table for calculating window feature values

[0089] Window number Gradient value set Eigenvalue #timg# Direction angle 1 0.15,0.18,0.22 0.183 45° 2 0.25,0.24,0.26 0.250 47° 3 0.19,0.21,0.23 0.210 49° 4 0.18,0.17,0.19 0.180 48°

[0090] As shown in Table 1, the calculation process of the square root term in the formula is: take the sum of the squares of the differences of adjacent window feature values and divide by the total number of windows 3, and then take the square root to get 0.0465. The calculation of the direction angle deviation term is 2° divided by the range 30° to get 0.0667. The sum of the two is the final comprehensive gradient intensity 0.1132. When this value exceeds the 0.1 threshold, it is determined as a valid gradient vector, and a dynamic gradient vector set is generated.

[0091] S202: Based on the dynamic gradient vector set, calculate the cosine similarity between the direction angle of each vector and the track reference angle, compare the similarity value with the set threshold, filter out abnormal vectors and establish a spatial coordinate index to generate an offset abnormal vector set;

[0092] When based on the dynamic gradient vector set, calculate the cosine similarity between the direction angle of the vector and the track reference angle. For example, the cosine similarity between the direction angle of a certain vector 47° and the reference angle 45°:

[0093] ;

[0094] When the set threshold is 0.98, this vector is retained because the similarity of 0.9994 is higher than the threshold. When a vector with a direction angle of 58° is detected:

[0095] ;

[0096] Since 0.9744 is lower than the threshold, it is marked as an abnormal vector. When establishing the spatial coordinate index, record the center coordinates (120, 80) of the window where the abnormal vector is located, and generate a set of offset abnormal vectors.

[0097] Table 2 Example Table for Screening Abnormal Vectors

[0098] Vector number Direction angle Cosine similarity Judgment result 1 47° 0.9994 Keep 2 58° 0.9744 Abnormal 3 43° 0.9986 Keep

[0099] As shown in Table 2, by comparing the cosine similarity threshold, abnormal vectors with a large deviation in the direction angle are screened out, and a set of offset abnormal vectors is generated.

[0100] S203: Call the set of offset abnormal vectors, and according to the product relationship between the magnitude of the horizontal gradient vector and the deviation value of the direction angle, generate a steering angle correction amount in proportion, integrate the correction amount data, and generate a hub steering angle correction amount;

[0101] When calling the set of offset abnormal vectors, extract the magnitude of the abnormal vector and the deviation value of the direction angle , and calculate the steering angle correction amount according to the product relationship:

[0102] ;

[0103] When multiple abnormal vectors are detected, the correction amounts are weighted and averaged according to the influence range. For example, when the correction amounts of three abnormal vectors are 0.056, 0.048, and 0.062 respectively, and the weight coefficients are 0.5, 0.3, and 0.2:

[0104] ;

[0105] This result indicates that the steering angle needs to be adjusted by 0.0548 radians (about 3.14°). After integrating the correction amount data of all abnormal areas, a hub steering angle correction amount is generated.

[0106] Please refer to Figure 4 , and the steps for obtaining the grasping pose compensation parameters are as follows:

[0107] S301: Call the hub steering angle correction amount, extract the offset parameter, match the corner coordinates of the visual identifier at the track seam of the current video frame, and use the formula:

[0108] ;

[0109] Calculate the registration accuracy of feature points and generate a set of registered feature points;

[0110] Among them, is the registration accuracy of feature points, is the corner coordinate of the current frame, is the coordinate of the reference mark, is the area of the corner neighborhood, is the ratio of the current contour perimeter to the standard perimeter, is the angle difference between the current frame and the reference frame;

[0111] When calling the correction amount of the hub steering angle, extract the offset parameter Δθ = 0.0548 radians, match the corner coordinates of the track seam mark in the current video frame, and collect corner data using an industrial camera at a resolution of 0.1 mm / pixel. For example, detect the corner coordinates of the current frame , the coordinate of the reference mark , calculate the Euclidean distance:

[0112] ;

[0113] The area of the corner neighborhood is calculated as the actual area of a 5×5 pixel region:

[0114] ;

[0115] The ratio of the current contour perimeter is calculated as the ratio of the detected perimeter of 48.2 mm to the standard perimeter of 50.0 mm: ;

[0116] Take the steering angle deviation of 0.0548 radians between the current frame and the reference frame for the angle difference Δθ1, and substitute it into the formula to calculate the registration accuracy:

[0117] ;

[0118] This result indicates that the registration accuracy of the feature points, 0.260, exceeds the threshold, indicating that its spatial position and angle deviation meet the compensation requirements and are included in the set of registered feature points for subsequent calculations.

[0119] S302: Based on the set of registered feature points, calculate the offset of the centroid coordinates of each feature point, establish a projection coordinate system along the track extension direction, decompose the offset into horizontal and vertical components, and generate the centroid offset projection components;

[0120] When based on the set of registered feature points, calculate the centroid coordinates of the feature points. For example, for the three feature point coordinates (152.3, 80.5), (253.6, 81.2), (355.1, 79.8), the centroid coordinates are calculated as:

[0121] ;

[0122] The reference centroid coordinates are (250.0, 80.0), and the offset is calculated as:

[0123] ;

[0124] Establish a projection coordinate system along the track extension direction. Let the track direction be the X'-axis and the vertical direction be the Y'-axis. The offset is decomposed into:

[0125] ;

[0126] ;

[0127] Generate the centroid offset projection components (3.20 mm, -1.32 mm).

[0128] S303: Call the centroid offset projection components, linearly combine the lateral component and the longitudinal component according to the degrees of freedom weights of the grasping mechanism, and superimpose the attitude compensation parameters of the equipment base to generate the grasping pose compensation parameters;

[0129] When calling the centroid offset projection components, set the lateral degree of freedom weight of the grasping mechanism to 0.7 and the longitudinal weight to 0.3, and calculate the linear combination value:

[0130] ;

[0131] Superimpose the attitude compensation parameter of the equipment base 0.15 mm (the compensation amount corresponding to the base tilt of 0.5° measured by the inclination sensor): ;

[0132] This result indicates that the grasping mechanism needs to be compensated 1.99 mm laterally to generate the grasping pose compensation parameters. The formula realizes the coordinated adjustment of multiple degrees of freedom through weight distribution, improving the accuracy of pose compensation.

[0133] Please refer to Figure 5 , and the steps for obtaining the multi-modal optimal path sequence are:

[0134] S401: Call the coordinates of the abnormal area in the track texture feature distribution map, calculate the Euclidean distance between the center point of each area and the grasping reference point, and use the formula:

[0135] ;

[0136] Construct the Euclidean distance weight to generate the dynamic Euclidean weight set;

[0137] Among them, is the Euclidean distance weight, are the coordinates of the abnormal area, is the grasping reference point, is the magnitude of the projection deviation vector, is the maximum value of the current projection deviation magnitude;

[0138] When calling the orbital texture feature distribution map, the central coordinates of the abnormal areas (120, 80), (250, 75), and (380, 85) are detected, and the grasping reference point is set to (200, 100). Calculate the Euclidean distance of the first abnormal area:

[0139] ;

[0140] The magnitude of the projection deviation vector is taken as 3.2 mm from the pose compensation parameters, and the maximum projection deviation of the current frame , substitute it into the formula to calculate the weight:

[0141] ;

[0142] Table 3 Euclidean weight calculation table

[0143] Abnormal area Center coordinates (mm) Euclidean distance (mm) #timg# (mm) #timg# (mm) #timg# 1 (120,80) 82.46 3.2 5.8 0.00668 2 (250,75) 50.99 4.1 5.8 0.01382 3 (380,85) 180.28 2.8 5.8 0.00268

[0144] As shown in Table 3, since Area 2 is closer to the reference point and has a larger projection deviation, it obtains the highest weight of 0.01382. This weight value reflects the combined influence of the regional spatial position and the projection deviation, and generates a dynamic Euclidean weight set.

[0145] S402: Call the projection deviation components in the grasping pose compensation parameters, decompose the absolute values of the lateral and longitudinal deviations along the orbital coordinate system, and superimpose the component values according to the Manhattan distance calculation rule to generate the Manhattan distance weight coefficient;

[0146] Call the projection deviation components (3.2 mm, -1.32 mm) in the grasping pose compensation parameters. Decompose the lateral component along the orbital coordinate system to be 3.2 mm (in the X-axis direction), and the longitudinal component to be 1.32 mm (in the Y-axis direction). Take the absolute value to calculate the Manhattan distance:

[0147] ;

[0148] Set the reference Manhattan distance to 5.0 mm and generate the weight coefficient:

[0149] ;

[0150] This coefficient indicates that the current deviation reaches 90.4% of the reference value, and the generated Manhattan distance weight coefficient 0.904 will be used for path cost value calculation.

[0151] S403: Calculate the path cost value by summing up the weighted ratios based on the dynamic Euclidean weight set and the Manhattan distance weight coefficient, and traverse the candidate paths using the path planning algorithm to generate a multi-modal optimal path sequence;

[0152] Based on the dynamic Euclidean weight set (0.00668, 0.01382, 0.00268) and the Manhattan coefficient 0.904, set the weighted ratio as 6:4, and calculate the cost value of path node A:

[0153] ;

[0154] Compare with the cost value of path node B, which is 0.2815, select the node B with a lower cost value to join the path sequence. After traversing all candidate nodes, generate a multi-modal optimal path sequence including the node sequence B-D-F-H. This cost value indicates that the comprehensive weight of node B is better, guiding the path planning algorithm to generate the most economical moving trajectory.

[0155] Please refer to Figure 6 , and the steps to obtain the drive wheel closed-loop control instruction are as follows:

[0156] S501: Monitor the steering angle and current speed value of the path nodes in the multi-modal optimal path sequence, multiply the angle value by the speed value, calculate the pulse width reference quantity, and add the device vibration compensation coefficient to generate a dynamic pulse width set;

[0157] When monitoring the path nodes in the multi-modal optimal path sequence, obtain the steering angle of node 5 as 30° and the current speed as 2.0 m / s, and convert the angle value to radians , multiply it by the speed value to calculate the pulse width reference quantity , the device vibration compensation coefficient collects the vibration data of the X-axis at 0.2 g, the Y-axis at 0.15 g, and the Z-axis at 0.1 g through a three-axis acceleration sensor, and calculates the comprehensive vibration energy , and normalizes it to the compensation coefficient , and add it to the reference quantity to generate the dynamic pulse width , For example, for node 6 with a steering angle of 45° (0.7854 rad) and a speed of 1.8 m / s, calculate the reference quantity , when the vibration coefficient is 0.33, the dynamic pulse width is , generating a dynamic pulse width set including the adjustment values of each node.

[0158] S502: Call the dynamic pulse width set, combine with the wheel speed deviation value feedback by the encoder, calculate the pulse width correction quantity, establish a linear relationship between the pulse width and the wheel speed deviation, and generate the closed-loop control signal parameters;

[0159] The calling node 5 has a dynamic pulse width of 1.329 rad·m / s, and the encoder feedback shows an actual wheel speed of 1.95 m / s. Calculate the speed deviation. , Set the proportionality coefficient (It is measured through experiments that for every 0.1 m / s of speed deviation, the pulse width needs to increase by 5%). Calculate the correction amount. , For example, the dynamic pulse width of node 6 is 1.879 rad·m / s, and the actual speed of 1.68 m / s results in a deviation of 0.12 m / s. After correction, the pulse width , Establish a linear relationship table to store the corresponding relationship between the proportionality coefficient and the deviation, and generate a closed-loop control signal parameter set.

[0160] S503: Based on the closed-loop control signal parameters, encode the steering angle and speed value according to the drive wheel control protocol, and fuse the safety protection threshold to generate a drive wheel closed-loop control instruction.

[0161] Based on the corrected pulse width of 1.362 rad·m / s of node 5, according to the encoding rules of the drive wheel CAN protocol, map the steering angle of 30° to the protocol field 0x1E (hexadecimal), map the speed value of 2.0 m / s to 0x02, combine the basic instruction code 0x1E02, set the safety protection threshold to the protocol mask 0x0F corresponding to the maximum allowable pulse width of 2.0 rad·m / s, and perform a bitwise AND operation. , For example, the corrected pulse width of node 6 is 3.006 rad·m / s, which exceeds the safety threshold, triggering a clipping process. , The corresponding encoding is 0x0F02, and finally generate a drive wheel closed-loop control instruction set.

[0162] A rail-mounted robot control system based on video intelligent assistance, the system includes: a texture feature analysis module, a lateral offset detection module, a pose compensation generation module, a multi-modal path planning module, and a drive closed-loop control module.

[0163] The texture feature analysis module is used to obtain the pixel spacing and direction angle of the track surface through the video stream, input the pixel spacing and direction angle into the gray-level co-occurrence matrix to extract the contrast parameter and energy parameter, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation within the window, generate a texture feature distribution map and transfer it to the lateral offset detection module and the multi-modal path planning module.

[0164] The lateral offset detection module is used to call the feature values of adjacent windows in the texture feature distribution map, perform a differential operation on the adjacent feature values to generate a lateral gradient vector, input the vector direction angle and the preset track reference angle into the cosine similarity calculation, generate a lateral offset when the calculation result is lower than the set threshold, and transfer the lateral offset to the pose compensation generation module.

[0165] A pose compensation generation module, which is used to match the contour coordinates of the track seam identifier in the current video frame based on the lateral offset, align the geometric features of the identifier using a feature point registration algorithm, calculate the projection component of the centroid offset on the motion plane, generate grasping pose compensation parameters and transfer them to the multi-modal path planning module;

[0166] A multi-modal path planning module, which is used to call the coordinates of the abnormal area in the texture feature distribution map and the projection deviation in the grasping pose compensation parameters, perform Euclidean distance weighting calculation on the coordinates of the abnormal area, perform Manhattan distance weighting calculation on the projection deviation, input the two weights into weighted summation to generate a path cost value, output a multi-modal optimal path sequence and transfer it to the drive closed-loop control module;

[0167] A drive closed-loop control module, which is used to monitor the execution status of the path nodes in the multi-modal optimal path sequence, multiply the steering angle by the current speed value to calculate the pulse width, and input the encoder feedback value into the PID control algorithm to generate a closed-loop control signal, and output a drive wheel closed-loop control instruction including the steering angle and the speed value.

[0168] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A control method for a track-lying robot based on video intelligent assistance, characterized in that, It includes the following steps: S1: Obtain the track surface texture features. Calculate the gray-level co-occurrence matrix by computing the pixel spacing and direction angle, extract the contrast and energy parameters in the matrix, divide the detection window along the track extension direction and calculate the standard deviation, and generate the track texture feature distribution map; S2: Call the feature values of adjacent windows in the track texture feature distribution map, obtain the lateral gradient vector through differential operation, calculate the cosine similarity of the vector direction angle, and screen the cosine values not exceeding the similarity threshold to generate the hub steering angle correction amount; S3: Based on the offset parameter in the hub steering angle correction amount, register the contour coordinates of the track seam identifier, calculate the projection component of the centroid offset, and generate the grasping pose compensation parameter; S4: Combine the abnormal area coordinates in the track texture feature distribution map with the projection deviation in the grasping pose compensation parameter, construct the weights of the Euclidean distance and Manhattan distance, and calculate and generate the multi-modal optimal path sequence through weighted calculation; S5: Monitor the execution status of the path nodes in the multi-modal optimal path sequence, calculate the steering pulse width through the steering angle and the current speed value, combine the encoder feedback to obtain the closed-loop control signal, and output the drive wheel closed-loop control instruction.

2. The control method of the rail-lying robot based on video intelligent assistance according to claim 1, wherein: The track texture feature distribution map is specifically the contrast distribution, energy distribution, and window standard deviation range. The hub steering angle correction amount includes the lateral offset, angle deviation, and threshold trigger identifier. The grasping pose compensation parameter is specifically the coordinate offset, projection deviation, and attitude adjustment vector. The multi-modal optimal path sequence includes the path cost value matrix, weight distribution coefficient, and node priority sequence. The drive wheel closed-loop control instruction is specifically the steering angle set value, speed adjustment parameter, and PWM duty cycle.

3. The control method of the track-lying robot based on video intelligent assistance according to claim 1, characterized in that: The steps for obtaining the track texture feature distribution map are as follows: S101: Obtain the video stream track surface image data, detect the pixel spacing and direction angle distribution features, establish a spatial coordinate system along the track extension direction, map the pixel spacing data at different angles into the coordinate system, and generate the track texture spatial mapping set; S102: Call the track texture spatial mapping set, extract the contrast and energy parameters of each coordinate point, establish the dynamic correlation relationship between the parameters, divide continuous detection units along the track extension direction, calculate the parameter fluctuation amplitude within the unit, and generate the texture feature fluctuation sequence; S103: Based on the texture feature fluctuation sequence, statistically calculate the parameter change gradient between adjacent units, construct the gradient distribution heat map, mark the boundary coordinates of the gradient mutation area, and integrate the corresponding relationship between the spatial coordinates and the gradient values to generate the track texture feature distribution map.

4. The control method of the track-lying robot based on video intelligent assistance according to claim 1, characterized in that: The steps for obtaining the hub steering angle correction amount are as follows: S201: Call the track texture feature distribution map, extract the feature values of adjacent windows, and use the formula: ； Calculate the lateral gradient vector to generate the dynamic gradient vector set; Among them, represents the comprehensive gradient intensity, represents the eigenvalue of the i-th window, is the sliding average of the vector direction angle, is the trajectory reference angle, is the direction angle range; S202: Based on the dynamic gradient vector set, calculate the cosine similarity between the direction angle of each vector and the trajectory reference angle, compare the similarity value with the set threshold, screen the abnormal vectors and establish the spatial coordinate index, and generate the offset abnormal vector set; S203: Call the offset exception vector set, generate a steering angle correction amount according to the product relationship between the lateral gradient vector modulus and the direction angle deviation value, integrate the correction amount data, and generate a hub steering angle correction amount.

5. The method for controlling a track-lying robot based on video intelligent assistance according to claim 1, wherein: The steps for obtaining the grasping pose compensation parameter are as follows: S301: Call the hub steering angle correction amount, extract the offset parameter, match the corner coordinates of the visual identifier of the track seam in the current video frame, and use the formula: ； Calculate the feature point registration degree and generate a set of registered feature points; Among them, is the feature point registration accuracy, is the corner point coordinates of the current frame, is the reference marker coordinates, is the corner point neighborhood area, is the ratio of the current contour perimeter to the standard perimeter, is the angle difference between the current frame and the reference frame; S302: Based on the set of registered feature points, calculate the centroid coordinate offset of each feature point, establish a projection coordinate system along the track extension direction, decompose the offset into horizontal and vertical components, and generate the centroid offset projection component; S303: Call the centroid offset projection component, linearly combine the horizontal component and the vertical component according to the degrees of freedom weight of the grasping mechanism, and superimpose the device base attitude compensation parameter to generate the grasping pose compensation parameter.

6. The control method of the rail-mounted robot based on video intelligent assistance according to claim 1, wherein: The steps for obtaining the multi-modal optimal path sequence are as follows: S401: Call the coordinates of the abnormal area in the track texture feature distribution map, calculate the Euclidean distance between the center point of each area and the grasping reference point, and use the formula: ； Construct the Euclidean distance weight and generate a dynamic Euclidean weight set; Among them, is the Euclidean distance weight, is the coordinate of the abnormal area, is the grasping reference point, is the modulus of the projection deviation vector, is the maximum value of the current projection deviation modulus; S402: Call the projection deviation component in the grasping pose compensation parameter, decompose the absolute values of the horizontal and vertical deviations along the track coordinate system, and superimpose the component values according to the Manhattan distance calculation rule to generate the Manhattan distance weight coefficient; S403: Based on the dynamic Euclidean weight set and the Manhattan distance weight coefficient, calculate the path cost value by weighted proportional summation, traverse the candidate paths using the path planning algorithm, and generate a multi-modal optimal path sequence.

7. The method for controlling a track-lying robot based on video intelligent assistance according to claim 1, wherein: The steps for obtaining the driving wheel closed-loop control instruction are as follows: S501: Monitor the steering angle and the current speed value of the path nodes in the multi-modal optimal path sequence, multiply the angle value by the speed value, calculate the pulse width reference amount, and superimpose the device vibration compensation coefficient to generate a dynamic pulse width set; S502: Call the dynamic pulse width set, combine the wheel speed deviation value fed back by the encoder, calculate the pulse width correction amount, establish a linear relationship between the pulse width and the wheel speed deviation, and generate the closed-loop control signal parameter; S503: Based on the closed-loop control signal parameter, encode the steering angle and the speed value according to the driving wheel control protocol, and fuse the safety protection threshold to generate the driving wheel closed-loop control instruction.

8. A control system for a track-lying robot based on video intelligent assistance, characterized in that, The system is used for the video intelligent-assisted crawling-rail robot control method according to any one of claims 1-7. The system includes: a texture feature analysis module, a lateral offset detection module, a pose compensation generation module, a multi-modal path planning module, and a driving closed-loop control module; The texture feature analysis module is used to obtain the pixel spacing and direction angle of the track surface through the video stream, input the pixel spacing and direction angle into the gray-level co-occurrence matrix to extract the contrast parameter and the energy parameter, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation in the window, generate the texture feature distribution map, and transmit it to the lateral offset detection module and the multi-modal path planning module; The horizontal offset detection module is used to call the eigenvalue of adjacent windows in the texture feature distribution map, perform differential operation on adjacent eigenvalues to generate a horizontal gradient vector, input the vector direction angle and the preset trajectory reference angle into the cosine similarity calculation. When the calculation result is lower than the set threshold, a horizontal offset is generated and passed to the pose compensation generation module; The pose compensation generation module is used to match the contour coordinates of the current video frame track seam identifier based on the horizontal offset, align the geometric features of the identifier using the feature point registration algorithm, calculate the projection component of the centroid offset on the motion plane, generate the grasping pose compensation parameters and pass them to the multi-modal path planning module; The multi-modal path planning module is used to call the abnormal area coordinates in the texture feature distribution map and the projection deviation in the grasping pose compensation parameters, perform Euclidean distance weighting calculation on the abnormal area coordinates, perform Manhattan distance weighting calculation on the projection deviation, input the two weights into the weighted sum to generate the path cost value, output the multi-modal optimal path sequence and pass it to the drive closed-loop control module; The drive closed-loop control module is used to monitor the execution status of the path nodes in the multi-modal optimal path sequence, multiply the steering angle by the current speed value to calculate the pulse width, combine the encoder feedback value and input it into the PID control algorithm to generate a closed-loop control signal, and output the drive wheel closed-loop control instruction including the steering angle and the speed value.

Images