A control method and system for a track-based robot based on video intelligence assistance

Abstract

The present invention relates to the field of intelligent control technology, specifically a control method and system for a track-based robot based on video intelligent assistance. A track texture feature map is acquired from a video stream, and adjacent window differences are used to calculate lateral gradients and filter offsets. Projection deviation compensation parameters are calculated based on the offsets, and the optimal sequence of path cost values ​​is calculated using weighted calculations of abnormal regions. The sequence status is monitored, and pulse widths are calculated using encoder feedback and steering speed to output closed-loop instructions. The present invention acquires a texture map from video, extracts a contrast energy parameter window using a gray-level co-occurrence matrix, and calculates standard deviations. Lateral gradients are obtained using cosine similarity through adjacent window differences, and offsets are filtered. Seam markers are registered to calculate centroid projection deviation compensation parameters. Euclidean Manhattan weighting is used to generate a path sequence using abnormal region coordinate compensation parameters. The path status is monitored, and encoder feedback and steering speed are used to calculate pulse widths and output closed-loop instructions.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a control method and system for a track-based robot based on video intelligent assistance. Background Art

[0002] The field of intelligent control technology encompasses key technical elements such as automatic control system design, sensor signal processing, and actuator actuation. Its core focus is on achieving precise motion control of controlled objects through real-time data acquisition, processing, and feedback. This applies to scenarios such as industrial automated production lines, intelligent logistics systems, and the control of mobile rail equipment. Rail robotics, in particular, require comprehensive handling of multi-dimensional technical issues such as environmental perception, path planning, and motion control, requiring a high level of system integration.

[0003] The video-assisted control method for track-based robots uses visual sensors to acquire track profile data, employs feature point matching technology to establish a robot pose model, and generates drive commands based on preset control parameters. This involves three technical steps: track feature recognition, pose deviation calculation, and motion parameter correction. By establishing a mapping relationship between the image coordinate system and the motion coordinate system, and utilizing a proportional-integral-differential (PID) algorithm to output motor speed adjustments, the method ultimately achieves continuous motion control along the preset track.

[0004] Existing technologies rely on matching fixed feature points to establish pose models, without considering dynamic changes in texture, resulting in matching errors when the surface is worn or the lighting is uneven. Traditional proportional-integral-differential algorithms use static control parameters, which are difficult to adapt to the adjustment requirements of variable speed conditions, resulting in motion overshoot or response delay. The preset trajectory reference angle lacks dynamic adaptation to the geometric characteristics of the track joints, and the deviation in pose calculation increases when there are construction errors or thermal deformation. Existing path planning does not associate the coordinates of abnormal areas with motion parameters. When encountering temporary obstacles, it can only perform emergency braking and cannot generate an optimized path. The encoder feedback and visual data are not sufficiently integrated, and the control instructions and execution actions produce timing deviations during high-speed movement, affecting the stability of the system. For example, when oil stains cover the track, traditional feature recognition fails and causes deviations. The lack of an exception handling mechanism in existing technologies leads to safety hazards. Summary of the Invention

[0005] In order to solve the technical problems existing in the prior art, the embodiments of the present invention provide a method and system for controlling a track-based robot based on video intelligent assistance. The technical solution is as follows:

[0006] A method for controlling a track-based robot based on video intelligent assistance comprises the following steps:

[0007] S1: Acquire the track surface texture features, calculate the pixel spacing and direction angle to obtain the gray level co-occurrence matrix, extract the contrast and energy parameters in the matrix, divide the detection window along the track extension direction and calculate the standard deviation to generate the track texture feature distribution map;

[0008] S2: calling the feature values ​​of adjacent windows in the track texture feature distribution map, obtaining the lateral gradient vector through differential operation, calculating the cosine similarity of the vector direction angles, and screening the cosine values ​​that do not exceed the similarity threshold to generate the wheel steering angle correction value;

[0009] The steps for obtaining the wheel hub steering angle correction amount are as follows:

[0010] S201: Call the track texture feature distribution map to extract adjacent window feature values ​​using the formula:

[0011] ;

[0012] Calculate the transverse gradient vector and generate a dynamic gradient vector set;

[0013] in, represents the comprehensive gradient strength, represents the eigenvalue of the i-th window, is the sliding average of the vector direction angle, is the trajectory reference angle, is the directional angle range;

[0014] S202: Based on the dynamic gradient vector set, the cosine similarity between the direction angle of each vector and the trajectory reference angle is calculated, the similarity value is compared with a set threshold, abnormal vectors are screened and a spatial coordinate index is established to generate an offset abnormal vector set;

[0015] S203: calling the offset abnormality vector set, converting the steering angle correction value proportionally according to the product relationship between the transverse gradient vector modulus and the direction angle deviation value, integrating the correction value data, and generating the wheel steering angle correction value.

[0016] S3: Based on the offset parameter in the wheel hub steering angle correction, align the track joint marker contour coordinates, calculate the projection component of the center of mass offset, and generate the grasping posture compensation parameter;

[0017] The steps for obtaining the grasping posture compensation parameters are as follows:

[0018] S301: Call the wheel steering angle correction value, extract the offset parameter, match the corner coordinates of the track seam visual marker of the current video frame, and use the formula:

[0019] ;

[0020] Calculate the feature point registration degree and generate a registration feature point set;

[0021] in, is the feature point registration degree, is the coordinate of the corner point of the current frame, is the reference coordinate, is the neighborhood area of ​​the corner point, is the ratio of the current contour perimeter to the standard perimeter, is the angle difference between the current frame and the reference frame;

[0022] S302: Based on the registration feature point set, calculate the centroid coordinate offset of each feature point, establish a projection coordinate system along the track extension direction, decompose the offset into horizontal and vertical components, and generate a centroid offset projection component;

[0023] S303: Calling the center of mass offset projection component, linearly combining the transverse component and the longitudinal component according to the degree of freedom weight of the grasping mechanism, superimposing the device base posture compensation parameter, and generating the grasping posture compensation parameter.

[0024] S4: combining the coordinates of the abnormal area of ​​the track texture feature distribution map with the projection deviation in the grasping posture compensation parameter, constructing the Euclidean distance and Manhattan distance weights, and performing weighted calculation to generate a multimodal optimal path sequence;

[0025] S5: Monitor the execution status of the path nodes in the multimodal optimal path sequence, calculate the steering pulse width through the steering angle and the current speed value, obtain the closed-loop control signal in combination with the encoder feedback, and output the drive wheel closed-loop control instruction.

[0026] As a further solution of the present invention, the track texture feature distribution map is specifically contrast distribution, energy distribution, and window standard deviation range; the wheel hub steering angle correction includes lateral offset, angle deviation, and threshold trigger mark; the grasping posture compensation parameters are specifically coordinate offset, projection deviation, and posture adjustment vector; the multimodal optimal path sequence includes path cost value matrix, weight distribution coefficient, and node priority sequence; the drive wheel closed-loop control instruction is specifically steering angle setting value, speed adjustment parameter, and PWM duty cycle.

[0027] As a further solution of the present invention, the steps of obtaining the track texture feature distribution map are:

[0028] S101: Acquire track surface image data from a video stream, detect pixel spacing and angular distribution characteristics, establish a spatial coordinate system according to the track extension direction, map pixel spacing data at different angles to the coordinate system, and generate a track texture spatial mapping set;

[0029] S102: Calling the track texture space mapping set, extracting the contrast and energy parameters of each coordinate point, establishing a dynamic correlation between the parameters, dividing the continuous detection units along the track extension direction, calculating the parameter fluctuation amplitude within the unit, and generating a texture feature fluctuation sequence;

[0030] S103: Based on the texture feature fluctuation sequence, the parameter change gradients between adjacent units are counted, a gradient distribution heat map is constructed, the boundary coordinates of the gradient mutation area are marked, and the correspondence between the spatial coordinates and the gradient values ​​is integrated to generate a track texture feature distribution map.

[0031] As a further solution of the present invention, the step of obtaining the multimodal optimal path sequence is:

[0032] S401: Call the coordinates of the abnormal area of ​​the track texture feature distribution map, calculate the Euclidean distance between the center point of each area and the grasping reference point, and use the formula:

[0033] ;

[0034] Construct Euclidean distance weights and generate dynamic Euclidean weight sets;

[0035] in, is the Euclidean distance weight, is the coordinate of the abnormal area, To grab the reference point, is the modulus of the projection deviation vector, is the maximum value of the current projection deviation modulus;

[0036] S402: calling the projection deviation component in the grasping posture compensation parameter, decomposing the absolute values ​​of the horizontal and vertical deviations along the track coordinate system, superimposing the component values ​​according to the Manhattan distance calculation rule, and generating a Manhattan distance weight coefficient;

[0037] S403: Based on the dynamic Euclidean weight set and the Manhattan distance weight coefficient, the path cost is calculated by summing the weighted proportions, and the candidate paths are traversed using a path planning algorithm to generate a multimodal optimal path sequence.

[0038] As a further solution of the present invention, the step of obtaining the driving wheel closed-loop control instruction is:

[0039] S501: monitoring the steering angle and current speed value of the path node in the multimodal optimal path sequence, multiplying the angle value and the speed value, calculating the pulse width reference value, adding the equipment vibration compensation coefficient, and generating a dynamic pulse width set;

[0040] S502: calling the dynamic pulse width set, combining the wheel speed deviation value fed back by the encoder, calculating the pulse width correction value, establishing a linear relationship between the pulse width and the wheel speed deviation, and generating closed-loop control signal parameters;

[0041] S503: Based on the closed-loop control signal parameters, the steering angle and the speed value are encoded according to the driving wheel control protocol, integrated with the safety protection threshold, and a driving wheel closed-loop control instruction is generated.

[0042] A video-assisted track-based robot control system, comprising: a texture feature analysis module, a lateral offset detection module, a posture compensation generation module, a multimodal path planning module, and a drive closed-loop control module;

[0043] The texture feature analysis module is used to obtain the pixel spacing and azimuth angle of the track surface from the video stream, input the pixel spacing and azimuth angle into the gray-level co-occurrence matrix to extract the contrast and energy parameters, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation within the window, generate a texture feature distribution map, and pass it to the lateral deviation detection module and the multimodal path planning module;

[0044] The lateral offset detection module is used to call the adjacent window feature values ​​in the texture feature distribution map, perform differential operations on the adjacent feature values ​​to generate a lateral gradient vector, input the vector direction angle and the preset trajectory reference angle into the cosine similarity calculation, and generate a lateral offset when the calculation result is lower than the set threshold. The lateral offset is passed to the posture compensation generation module;

[0045] The pose compensation generation module is used to match the contour coordinates of the track seam marker of the current video frame based on the lateral offset, align the geometric features of the marker using a feature point registration algorithm, calculate the projection component of the center of mass offset on the motion plane, generate the grasping pose compensation parameters and pass them to the multimodal path planning module;

[0046] The multimodal path planning module is used to call the abnormal area coordinates in the texture feature distribution map and the projection deviation in the grasping posture compensation parameters, perform Euclidean distance weight calculation on the abnormal area coordinates, perform Manhattan distance weight calculation on the projection deviation, and take the weighted sum of the two weight inputs to generate the path cost value. The module outputs the multimodal optimal path sequence and passes it to the drive closed-loop control module.

[0047] The drive closed-loop control module is used to monitor the execution status of the path nodes in the multi-modal optimal path sequence, multiply the steering angle by the current speed value to calculate the pulse width, combine the encoder feedback value to input the PID control algorithm to generate a closed-loop control signal, and output the drive wheel closed-loop control instruction containing the steering angle and speed values.

[0048] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0049] In the present invention, the track surface texture features are extracted from contrast and energy parameters through the grayscale co-occurrence matrix, and the detection window is divided in the extension direction to calculate the standard deviation difference, thereby enhancing the ability to analyze complex textures. The lateral gradient vector difference operation of the feature values ​​of adjacent windows is combined with the cosine similarity threshold judgment to improve the accuracy of lateral offset detection and reduce the accumulation of posture deviations at the joints. The visual identification geometric feature registration adopts the calculation of the centroid offset projection component, and the dynamic adaptation of the grasping posture compensation parameters to solve the feature inaccuracy caused by illumination changes. The Euclidean distance and Manhattan distance are weighted to generate the path cost value, and a multimodal path sequence is constructed to achieve the coordination of abnormal area avoidance and trajectory optimization. The pulse width and encoder feedback are fused to form a closed-loop control signal, and the steering angle and speed value are dynamically coupled to improve the tracking stability of the variable speed motion trajectory. This solution integrates texture analysis, gradient operation, geometric registration and multimodal planning to enhance environmental adaptability and control accuracy based on real-time response. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of the method of the present invention;

[0051] Figure 2 This is a flow chart for obtaining a track texture feature distribution map of the present invention;

[0052] Figure 3 This is a flow chart for obtaining the wheel hub steering angle correction value of the present invention;

[0053] Figure 4 This is a flow chart for obtaining the grasping posture compensation parameters of the present invention;

[0054] Figure 5 This is a flowchart for obtaining the multimodal optimal path sequence of the present invention;

[0055] Figure 6 This is a flow chart for obtaining closed-loop control instructions for the drive wheel of the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0057] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0058] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0059] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0060] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0061] See also Figure 1 The present invention provides a technical solution: a control method of a track-based robot based on video intelligence assistance, comprising the following steps:

[0062] S1: Acquire the track surface texture features in the video stream, generate a gray-level co-occurrence matrix by calculating the pixel spacing and direction angle, extract the contrast and energy parameters in the matrix, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation within the window, and generate a track texture feature distribution map;

[0063] S2: Call the adjacent window feature values ​​in the track texture feature distribution map, obtain the lateral gradient vector through the difference operation of the adjacent window feature values, calculate the cosine similarity between the vector direction angle and the preset track reference angle, and generate the wheel steering angle correction value including the lateral offset when the cosine value is lower than the set threshold;

[0064] S3: Based on the offset parameter in the wheel steering angle correction, the geometric features of the track joint visual marker in the current video frame are matched, the marker contour coordinates are aligned using a feature point registration algorithm, the projection component of the center of mass offset is calculated, and the grasping posture compensation parameters are generated;

[0065] S4: Combine the abnormal area coordinates of the track texture feature distribution map with the projection deviation in the grasping posture compensation parameters, construct the Euclidean distance weight based on the abnormal area coordinates, and generate the Manhattan distance weight based on the projection deviation. Calculate the path cost value through weighted summation to generate the multimodal optimal path sequence;

[0066] S5: Monitor the execution status of the path nodes in the multimodal optimal path sequence, calculate the pulse width by multiplying the steering angle and the current speed value, generate a closed-loop control signal based on the encoder feedback value, and output the drive wheel closed-loop control instruction containing the steering angle and speed value.

[0067] The track texture feature distribution map specifically includes contrast distribution, energy distribution, and window standard deviation range. The wheel steering angle correction includes lateral offset, angle deviation, and threshold trigger mark. The grasping posture compensation parameters specifically include coordinate offset, projection deviation, and posture adjustment vector. The multimodal optimal path sequence includes path cost value matrix, weight distribution coefficient, and node priority sequence. The drive wheel closed-loop control instructions specifically include steering angle set value, speed adjustment parameter, and PWM duty cycle.

[0068] See also Figure 2 , the steps to obtain the track texture feature distribution map are:

[0069] S101: Acquire track surface image data from a video stream, detect pixel spacing and angular distribution characteristics, establish a spatial coordinate system according to the track extension direction, map pixel spacing data at different angles to the coordinate system, and generate a track texture spatial mapping set;

[0070] When acquiring video stream track surface image data, an industrial camera is used to capture track surface images at a rate of 30 frames per second. When detecting pixel spacing, the Euclidean distance between adjacent track texture feature points is selected as the spacing value. For example, the spacing between adjacent rivets is detected to be 15 pixels. When calculating the directional angle distribution feature, the track extension direction is used as the reference axis, and the angle between the line connecting each texture feature point and the reference axis is measured. When a feature point connection line is detected to form a 45-degree angle with the reference axis, it is classified into the 45-degree direction group. When establishing a spatial coordinate system, the track starting point is used as the origin, the extension direction is the positive direction of the X-axis, and the vertical direction is the Y-axis. The 0-degree direction spacing data is mapped to the X-axis coordinate interval, and the 90-degree direction data is mapped to the Y-axis coordinate interval. For example, the 15-pixel spacing in the 0-degree direction is mapped to a 15-unit length of the X-axis in the coordinate system, and the 21.21-pixel spacing in the 45-degree direction is decomposed into 15 units of the X-axis and 15 units of the Y-axis. The texture space data conversion is completed through coordinate mapping to generate a track texture space mapping set.

[0071] S102: Calling the track texture space mapping set, extracting the contrast and energy parameters of each coordinate point, establishing a dynamic correlation between the parameters, dividing the continuous detection units along the track extension direction, calculating the parameter fluctuation amplitude within the unit, and generating a texture feature fluctuation sequence;

[0072] When calling the track texture space mapping set, the contrast parameter of the texture point with coordinates (120, 80) is extracted, and the maximum grayscale difference in the 3×3 neighborhood of the point is calculated to be 85. The energy parameter is calculated using the grayscale variance value of the 5×5 pixel area around the point, and the energy parameter is 0.67. When establishing a dynamic association relationship, the contrast parameter and the energy parameter are linearly combined with a weight of 0.6:0.4. For example, a point with a contrast of 0.8 and an energy of 0.5 is combined to obtain a comprehensive parameter value of 0.68. When dividing the detection unit, the unit is divided every 50 pixels along the X axis. The range method is used to calculate the parameter fluctuation amplitude within the unit. For example, the maximum comprehensive parameter in the 5th unit is 0.75 and the minimum is 0.62, with a fluctuation amplitude of 0.13, generating a texture feature fluctuation sequence.

[0073] S103: Based on the texture feature fluctuation sequence, the parameter change gradients between adjacent units are counted, a gradient distribution heat map is constructed, the boundary coordinates of the gradient mutation area are marked, and the correspondence between the spatial coordinates and the gradient values ​​is integrated to generate a track texture feature distribution map;

[0074] Based on the texture feature fluctuation sequence, the gradient of the adjacent unit parameter changes is counted and calculated using the post-difference method. For example, the gradient of unit 3 parameter 0.68 and unit 4 parameter 0.72 is 0.04 / 50 pixels = 0.0008 per pixel. When constructing the gradient distribution heat map, the gradient value 0.0008 is mapped to a 120-degree hue (green) in the HSV color space. When the gradient mutation from unit 7 to unit 8 is detected to be 0.002, the corresponding hue is changed to 0 degrees (red). When marking the boundary of the mutation area, the starting coordinates of unit 7 (350, 0) and the ending coordinates of unit 8 (400, 120) are determined. When integrating the spatial coordinates and the gradient values, the coordinates (375, 60) are established to correspond to the gradient peak 0.0025, and the track texture feature distribution map is generated.

[0075] See also Figure 3 , the steps for obtaining the wheel steering angle correction are:

[0076] S201: Call the track texture feature distribution map and extract the adjacent window feature values ​​using the formula:

[0077] ;

[0078] Calculate the transverse gradient vector and generate a dynamic gradient vector set;

[0079] in, represents the comprehensive gradient strength, represents the eigenvalue of the i-th window, represents the difference of eigenvalues ​​of adjacent windows, is the total number of windows, is the sliding average of the vector direction angle, is the trajectory reference angle, is the directional angle range;

[0080] When calling the track texture feature distribution map, the detection area is divided into 50-pixel windows along the track extension direction. When extracting the window feature value, the arithmetic mean of the gradient value in each window is calculated. For example, window 1 contains gradient values ​​0.15, 0.18, and 0.22, and the feature value , window 2 eigenvalue , window 3 eigenvalue , window 4 eigenvalue , the difference of adjacent window eigenvalues ​​is calculated as follows:

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] The sliding average of the azimuth angle is calculated using a 3-window sliding method, with the azimuth angles of windows 1-3 being 45°, 47°, and 49° respectively: ;

[0086] Track reference angle , directional angle range , substitute the formula to complete the calculation:

[0087] ;

[0088] Table 1 Example of window eigenvalue calculation

[0089]

[0090] As shown in Table 1, the calculation process of the square root term in the formula is: take the sum of the squares of the eigenvalue differences of adjacent windows and divide it by the total number of windows 3, and the square root is 0.0465. The directional angle deviation term is calculated as 2° divided by the range 30° to get 0.0667. The two are added together to obtain the final comprehensive gradient strength of 0.1132. When this value exceeds the threshold of 0.1, it is determined to be a valid gradient vector, and a dynamic gradient vector set is generated.

[0091] S202: Based on the dynamic gradient vector set, the cosine similarity of each vector direction angle and the trajectory reference angle is calculated, the similarity value is compared with the set threshold, abnormal vectors are screened and spatial coordinate indexes are established to generate an offset abnormal vector set;

[0092] When based on a dynamic gradient vector set, the cosine similarity between the vector direction angle and the trajectory reference angle is calculated. For example, the cosine similarity between a vector direction angle of 47° and a reference angle of 45° is:

[0093] ;

[0094] When the threshold is set to 0.98, the vector is retained because its similarity is 0.9994, which is higher than the threshold. When a vector direction angle of 58° is detected:

[0095] ;

[0096] Because 0.9744 is lower than the threshold, it is marked as an abnormal vector. When establishing the spatial coordinate index, the center coordinates of the window where the abnormal vector is located (120, 80) are recorded to generate an offset abnormal vector set.

[0097] Table 2 Abnormal vector screening example table

[0098]

[0099] As shown in Table 2, by comparing the cosine similarity threshold, the abnormal vectors with large directional angle deviations are screened out to generate a set of offset abnormal vectors.

[0100] S203: Calling the offset anomaly vector set, converting the product of the transverse gradient vector modulus and the directional angle deviation value proportionally to generate a steering angle correction value, integrating the correction value data, and generating a wheel steering angle correction value;

[0101] When calling the offset exception vector set, extract the exception vector modulus length Deviation from the direction angle , calculate the steering angle correction according to the product relationship:

[0102] ;

[0103] When multiple abnormal vectors are detected, the correction amount is weighted averaged according to the impact range. For example, when the correction amounts of three abnormal vectors are 0.056, 0.048, and 0.062 respectively, and the weight coefficients are 0.5, 0.3, and 0.2:

[0104] ;

[0105] The result shows that the steering angle needs to be adjusted by 0.0548 radians (about 3.14°). After integrating all the correction data of the abnormal area, the wheel steering angle correction is generated.

[0106] See also Figure 4 , the steps to obtain the grasping posture compensation parameters are:

[0107] S301: Call the wheel steering angle correction value, extract the offset parameter, and match the corner coordinates of the track seam visual marker of the current video frame using the formula:

[0108] ;

[0109] Calculate the feature point registration degree and generate a registration feature point set;

[0110] in, is the feature point registration degree, is the coordinate of the corner point of the current frame, is the reference coordinate, is the neighborhood area of ​​the corner point, is the ratio of the current contour perimeter to the standard perimeter, is the angle difference between the current frame and the reference frame;

[0111] When calling the wheel steering angle correction, extract the offset parameter Δθ = 0.0548 radians, match the corner coordinates of the track joint mark in the current video frame, and use an industrial camera to collect corner data with a resolution of 0.1mm / pixel. For example, the corner coordinates of the current frame are detected , benchmark identification coordinates , calculate the Euclidean distance:

[0112] ;

[0113] Corner neighborhood area Calculated as the actual area of ​​a 5×5 pixel region:

[0114] ;

[0115] Current contour perimeter ratio Calculated as the ratio of the test circumference 48.2mm to the standard circumference 50.0mm: ;

[0116] The angle difference Δθ1 is the steering angle deviation of the current frame and the reference frame, which is 0.0548 radians. Substitute it into the formula to calculate the registration degree:

[0117] ;

[0118] The result shows that the registration degree of the feature point 0.260 exceeds the threshold, indicating that its spatial position and angle deviation meet the compensation requirements and are included in the registration feature point set for subsequent calculations.

[0119] S302: Based on the registration feature point set, calculate the centroid coordinate offset of each feature point, establish a projection coordinate system along the track extension direction, decompose the offset into horizontal and vertical components, and generate the centroid offset projection component;

[0120] When the feature point set is registered, the centroid coordinates of the feature points are calculated. For example, the coordinates of the three feature points are (152.3, 80.5), (253.6, 81.2), and (355.1, 79.8). The centroid coordinates are calculated as follows:

[0121] ;

[0122] The base center coordinates are (250.0, 80.0), and the offset is calculated as:

[0123] ;

[0124] A projection coordinate system is established along the extension direction of the track. The track direction is set as the X' axis and the vertical direction is set as the Y' axis. The offset is decomposed into:

[0125] ;

[0126] ;

[0127] Generates the centroid offset projection component (3.20mm, -1.32mm).

[0128] S303: Calling the center of mass offset projection component, linearly combining the transverse component and the longitudinal component according to the gripping mechanism degree of freedom weight, and superimposing the device base posture compensation parameter to generate the gripping posture compensation parameter;

[0129] When calling the center of mass offset projection component, set the horizontal degree of freedom weight of the gripping mechanism to 0.7 and the vertical weight to 0.3, and calculate the linear combination value:

[0130] ;

[0131] The base attitude compensation parameter of the superimposed device is 0.15mm (the compensation amount corresponds to a 0.5° tilt of the base measured by the inclination sensor): ;

[0132] This result indicates that the gripping mechanism requires a lateral compensation of 1.99 mm, generating the gripping posture compensation parameters. The formula achieves coordinated adjustment of multiple degrees of freedom through weight distribution, improving the accuracy of posture compensation.

[0133] See also Figure 5 , the steps to obtain the multimodal optimal path sequence are:

[0134] S401: Call the coordinates of the abnormal area of ​​the track texture feature distribution map and calculate the Euclidean distance between the center point of each area and the grasping reference point using the formula:

[0135] ;

[0136] Construct Euclidean distance weights and generate dynamic Euclidean weight sets;

[0137] in, is the Euclidean distance weight, is the coordinate of the abnormal area, To grab the reference point, is the modulus of the projection deviation vector, is the maximum value of the current projection deviation modulus;

[0138] When calling the track texture feature distribution map, the center coordinates of the abnormal area are detected (120,80), (250,75), and (380,85). The grab reference point is set to (200,100). The Euclidean distance of the first abnormal area is calculated:

[0139] ;

[0140] Projection deviation vector modulus 3.2mm taken from the pose compensation parameters, the maximum projection deviation of the current frame , substitute into the formula to calculate the weight:

[0141] ;

[0142] Table 3 Euclidean weight calculation table

[0143]

[0144] As shown in Table 3, region 2 receives the highest weight of 0.01382 due to its proximity to the benchmark and large projection deviation. This weight value reflects the combined influence of the regional spatial position and projection deviation, generating a dynamic Euclidean weight set.

[0145] S402: Calling the projection deviation component in the grasping posture compensation parameter, decomposing the absolute values ​​of the horizontal and vertical deviations along the track coordinate system, superimposing the component values ​​according to the Manhattan distance calculation rule, and generating a Manhattan distance weight coefficient;

[0146] Call the projection deviation component (3.2mm, -1.32mm) in the grasping posture compensation parameter, decompose the horizontal component into 3.2mm (X-axis direction) and the vertical component into 1.32mm (Y-axis direction) along the track coordinate system, and take the absolute value to calculate the Manhattan distance:

[0147] ;

[0148] Set the base Manhattan distance to 5.0 mm and generate the weight coefficient:

[0149] ;

[0150] This coefficient indicates that the current deviation reaches 90.4% of the baseline value, and the generated Manhattan distance weight coefficient of 0.904 will be used for path cost calculation.

[0151] S403: Based on the dynamic Euclidean weight set and the Manhattan distance weight coefficient, the path cost is calculated by summing the weighted proportions, and the candidate paths are traversed using a path planning algorithm to generate a multimodal optimal path sequence;

[0152] Based on the dynamic Euclidean weight set (0.00668, 0.01382, 0.00268) and the Manhattan coefficient of 0.904, with a weighting ratio of 6:4, calculate the cost of path node A:

[0153] ;

[0154] Comparing the cost of node B (0.2815) on the path, we select the node B with the lower cost and add it to the path sequence. After traversing all candidate nodes, we generate a multimodal optimal path sequence containing the node sequence BDFH. This cost indicates that node B has a better overall weight, guiding the path planning algorithm to generate the most economical trajectory.

[0155] See also Figure 6 , the steps for obtaining the closed-loop control instructions of the driving wheel are:

[0156] S501: monitoring the steering angle and current speed value of the path node in the multimodal optimal path sequence, multiplying the angle value and the speed value, calculating the pulse width reference value, adding the equipment vibration compensation coefficient, and generating a dynamic pulse width set;

[0157] When monitoring the path nodes in the multimodal optimal path sequence, obtain the turning angle 30° and current speed 2.0m / s of node 5, and convert the angle value into radians. , multiplied by the speed value to calculate the pulse width reference The equipment vibration compensation coefficient is calculated by collecting the vibration data of X axis 0.2g, Y axis 0.15g, and Z axis 0.1g through the three-axis acceleration sensor to calculate the comprehensive vibration energy. , normalized to the compensation coefficient , added to the reference quantity to generate dynamic pulse width For example, the node 6 has a steering angle of 45° (0.7854 rad) and a speed of 1.8 m / s. The calculation of the reference quantity is: , when the vibration coefficient is 0.33, the dynamic pulse width is , generating a dynamic pulse width set containing the adjustment value of each node.

[0158] S502: Calling the dynamic pulse width set, combining the wheel speed deviation value fed back by the encoder, calculating the pulse width correction value, establishing a linear relationship between the pulse width and the wheel speed deviation, and generating closed-loop control signal parameters;

[0159] Call node 5 dynamic pulse width 1.329rad·m / s, encoder feedback actual wheel speed 1.95m / s, calculate speed deviation , set the proportional coefficient (According to the experimental results, the pulse width needs to be increased by 5% for every 0.1m / s of speed deviation). Calculate the correction amount. For example, the dynamic pulse width of node 6 is 1.879rad·m / s, and the actual speed is 1.68m / s, resulting in a deviation of 0.12m / s. The corrected pulse width , establish a linear relationship table to store the corresponding relationship between the proportional coefficient and the deviation, and generate a closed-loop control signal parameter set.

[0160] S503: Based on the closed-loop control signal parameters, the steering angle and speed values ​​are encoded according to the drive wheel control protocol, integrated with the safety protection threshold, and a drive wheel closed-loop control instruction is generated;

[0161] Based on the corrected pulse width of 1.362 rad·m / s at node 5, according to the drive wheel CAN protocol coding rules, the steering angle of 30° is mapped to the protocol field 0x1E (hexadecimal), the speed value of 2.0 m / s is mapped to 0x02, and the basic instruction code 0x1E02 is combined. The safety protection threshold is set to the maximum allowable pulse width of 2.0 rad·m / s corresponding to the protocol mask 0x0F, and a bitwise AND operation is performed. For example, the pulse width of node 6 after correction is 3.006rad·m / s, which exceeds the safety threshold and triggers the limit process. , corresponding to the code 0x0F02, and finally generates the drive wheel closed-loop control instruction set.

[0162] A video-assisted track-based robot control system includes a texture feature analysis module, a lateral offset detection module, a posture compensation generation module, a multimodal path planning module, and a drive closed-loop control module.

[0163] The texture feature analysis module is used to obtain the pixel spacing and azimuth angle of the track surface from the video stream, input the pixel spacing and azimuth angle into the gray-level co-occurrence matrix to extract the contrast and energy parameters, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation within the window, generate a texture feature distribution map, and pass it to the lateral deviation detection module and the multimodal path planning module;

[0164] The lateral offset detection module is used to call the adjacent window feature values ​​in the texture feature distribution map, perform differential operations on the adjacent feature values ​​to generate a lateral gradient vector, input the vector direction angle and the preset trajectory reference angle into the cosine similarity calculation, and generate a lateral offset when the calculation result is lower than the set threshold. The lateral offset is passed to the posture compensation generation module;

[0165] The pose compensation generation module is used to match the contour coordinates of the track seam marker of the current video frame based on the lateral offset, align the geometric features of the marker using a feature point registration algorithm, calculate the projection component of the center of mass offset on the motion plane, generate the grasping pose compensation parameters and pass them to the multimodal path planning module;

[0166] The multimodal path planning module is used to call the abnormal area coordinates in the texture feature distribution map and the projection deviation in the grasping posture compensation parameters, perform Euclidean distance weight calculation on the abnormal area coordinates, perform Manhattan distance weight calculation on the projection deviation, and take the weighted sum of the two weight inputs to generate the path cost value. The module outputs the multimodal optimal path sequence and passes it to the drive closed-loop control module.

[0167] The drive closed-loop control module is used to monitor the execution status of the path nodes in the multi-modal optimal path sequence, multiply the steering angle by the current speed value to calculate the pulse width, combine the encoder feedback value to input the PID control algorithm to generate a closed-loop control signal, and output the drive wheel closed-loop control instruction containing the steering angle and speed values.

[0168] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A control method for a track-based robot based on video intelligence assistance, characterized in that: The following steps are involved: S1: Acquire the track surface texture features, calculate the pixel spacing and direction angle to obtain the gray level co-occurrence matrix, extract the contrast and energy parameters in the matrix, divide the detection window along the track extension direction and calculate the standard deviation to generate the track texture feature distribution map; S2: calling the feature values ​​of adjacent windows in the track texture feature distribution map, obtaining the lateral gradient vector through differential operation, calculating the cosine similarity of the vector direction angles, and screening the cosine values ​​that do not exceed the similarity threshold to generate the wheel steering angle correction value; The steps for obtaining the wheel hub steering angle correction amount are as follows: S201: Call the track texture feature distribution map to extract adjacent window feature values ​​using the formula: ； Calculate the transverse gradient vector and generate a dynamic gradient vector set; in, represents the comprehensive gradient strength, represents the eigenvalue of the i-th window, is the sliding average of the vector direction angle, is the trajectory reference angle, is the directional angle range; S202: Based on the dynamic gradient vector set, the cosine similarity between the direction angle of each vector and the trajectory reference angle is calculated, the similarity value is compared with a set threshold, abnormal vectors are screened and a spatial coordinate index is established to generate an offset abnormal vector set; S203: calling the offset abnormality vector set, converting the proportional value to generate a steering angle correction value according to the product relationship between the transverse gradient vector modulus and the direction angle deviation value, integrating the correction value data, and generating a wheel steering angle correction value; S3: Based on the offset parameter in the wheel hub steering angle correction, align the track joint marker contour coordinates, calculate the projection component of the center of mass offset, and generate the grasping posture compensation parameter; The steps for obtaining the grasping posture compensation parameters are as follows: S301: Call the wheel steering angle correction value, extract the offset parameter, match the corner coordinates of the track seam visual marker of the current video frame, and use the formula: ； Calculate the feature point registration degree and generate a registration feature point set; in, is the feature point registration degree, is the coordinate of the corner point of the current frame, is the reference coordinate, is the neighborhood area of ​​the corner point, is the ratio of the current contour perimeter to the standard perimeter, is the angle difference between the current frame and the reference frame; S302: Based on the registration feature point set, calculate the centroid coordinate offset of each feature point, establish a projection coordinate system along the track extension direction, decompose the offset into horizontal and vertical components, and generate a centroid offset projection component; S303: Calling the center of mass offset projection component, linearly combining the transverse component and the longitudinal component according to the gripping mechanism degree of freedom weight, and superimposing the device base posture compensation parameter to generate the gripping posture compensation parameter; S4: combining the coordinates of the abnormal area of ​​the track texture feature distribution map with the projection deviation in the grasping posture compensation parameter, constructing the Euclidean distance and Manhattan distance weights, and performing weighted calculation to generate a multimodal optimal path sequence; S5: Monitor the execution status of the path nodes in the multimodal optimal path sequence, calculate the steering pulse width through the steering angle and the current speed value, obtain the closed-loop control signal in combination with the encoder feedback, and output the drive wheel closed-loop control instruction.

2. The method for controlling a track-based robot based on video intelligence assistance according to claim 1, characterized in that: The track texture feature distribution map specifically includes contrast distribution, energy distribution, and window standard deviation range; the wheel hub steering angle correction includes lateral offset, angle deviation, and threshold trigger flag; the grasping posture compensation parameters specifically include coordinate offset, projection deviation, and posture adjustment vector; the multimodal optimal path sequence includes path cost value matrix, weight distribution coefficient, and node priority sequence; the drive wheel closed-loop control instruction specifically includes steering angle setting value, speed adjustment parameter, and PWM duty cycle.

3. The method for controlling a track-based robot based on video intelligence assistance according to claim 1, characterized in that: The steps for obtaining the track texture feature distribution map are: S101: Acquire track surface image data from a video stream, detect pixel spacing and angular distribution characteristics, establish a spatial coordinate system according to the track extension direction, map pixel spacing data at different angles to the coordinate system, and generate a track texture spatial mapping set; S102: Calling the track texture space mapping set, extracting the contrast and energy parameters of each coordinate point, establishing a dynamic correlation between the parameters, dividing the continuous detection units along the track extension direction, calculating the parameter fluctuation amplitude within the unit, and generating a texture feature fluctuation sequence; S103: Based on the texture feature fluctuation sequence, the parameter change gradients between adjacent units are counted, a gradient distribution heat map is constructed, the boundary coordinates of the gradient mutation area are marked, and the correspondence between the spatial coordinates and the gradient values ​​is integrated to generate a track texture feature distribution map.

4. The method for controlling a track-based robot based on video intelligence assistance according to claim 1, characterized in that: The steps for obtaining the multimodal optimal path sequence are: S401: Call the coordinates of the abnormal area of ​​the track texture feature distribution map, calculate the Euclidean distance between the center point of each area and the grasping reference point, and use the formula: ； Construct Euclidean distance weights and generate dynamic Euclidean weight sets; in, is the Euclidean distance weight, is the coordinate of the abnormal area, To grab the reference point, is the modulus of the projection deviation vector, is the maximum value of the current projection deviation modulus; S402: calling the projection deviation component in the grasping posture compensation parameter, decomposing the absolute values ​​of the horizontal and vertical deviations along the track coordinate system, superimposing the component values ​​according to the Manhattan distance calculation rule, and generating a Manhattan distance weight coefficient; S403: Based on the dynamic Euclidean weight set and the Manhattan distance weight coefficient, the path cost is calculated by summing the weighted proportions, and the candidate paths are traversed using a path planning algorithm to generate a multimodal optimal path sequence.

5. The method for controlling a track-based robot based on video intelligence assistance according to claim 1, characterized in that: The steps for obtaining the driving wheel closed-loop control instruction are as follows: S501: monitoring the steering angle and current speed value of the path node in the multimodal optimal path sequence, multiplying the angle value and the speed value, calculating the pulse width reference value, adding the equipment vibration compensation coefficient, and generating a dynamic pulse width set; S502: calling the dynamic pulse width set, combining the wheel speed deviation value fed back by the encoder, calculating the pulse width correction value, establishing a linear relationship between the pulse width and the wheel speed deviation, and generating closed-loop control signal parameters; S503: Based on the closed-loop control signal parameters, the steering angle and the speed value are encoded according to the driving wheel control protocol, integrated with the safety protection threshold, and a driving wheel closed-loop control instruction is generated.

6. A track-based robot control system based on video intelligence assistance, characterized in that: The system is used for the video intelligent-assisted track-type robot control method according to any one of claims 1 to 5, and the system includes: a texture feature analysis module, a lateral offset detection module, a posture compensation generation module, a multimodal path planning module, and a drive closed-loop control module; The texture feature analysis module is used to obtain the pixel spacing and azimuth angle of the track surface from the video stream, input the pixel spacing and azimuth angle into the gray-level co-occurrence matrix to extract the contrast and energy parameters, divide the detection window along the track extension direction, calculate the maximum difference of the texture feature standard deviation within the window, generate a texture feature distribution map, and pass it to the lateral deviation detection module and the multimodal path planning module; The lateral offset detection module is used to call the adjacent window feature values ​​in the texture feature distribution map, perform differential operations on the adjacent feature values ​​to generate a lateral gradient vector, input the vector direction angle and the preset trajectory reference angle into the cosine similarity calculation, and generate a lateral offset when the calculation result is lower than the set threshold. The lateral offset is passed to the posture compensation generation module; The pose compensation generation module is used to match the contour coordinates of the track seam marker of the current video frame based on the lateral offset, align the geometric features of the marker using a feature point registration algorithm, calculate the projection component of the center of mass offset on the motion plane, generate the grasping pose compensation parameters and pass them to the multimodal path planning module; The multimodal path planning module is used to call the abnormal area coordinates in the texture feature distribution map and the projection deviation in the grasping posture compensation parameters, perform Euclidean distance weight calculation on the abnormal area coordinates, perform Manhattan distance weight calculation on the projection deviation, and take the weighted sum of the two weight inputs to generate the path cost value. The module outputs the multimodal optimal path sequence and passes it to the drive closed-loop control module. The drive closed-loop control module is used to monitor the execution status of the path nodes in the multi-modal optimal path sequence, multiply the steering angle by the current speed value to calculate the pulse width, combine the encoder feedback value to input the PID control algorithm to generate a closed-loop control signal, and output the drive wheel closed-loop control instruction containing the steering angle and speed values.

Images