Inspection point collection completion confirmation system based on task state machine

Abstract

The application relates to a task state machine-based inspection point position collection completion confirmation system and relates to the technical field of industrial fixed equipment inspection robots. The mean value of the size of continuous pixels of a target frame is obtained to obtain the observation width and height, a basic approximation speed is generated in combination with an expected height, and a target frame distortion rate representing a yaw state is calculated by comparing a reference width-length ratio. Based on the distortion rate, shunt control is performed: when the distortion rate is greater than or equal to a preset threshold value, a forward instruction speed is generated based on the reduction of the basic speed, and a normal registration angular velocity and a counter-attack lateral movement speed are calculated in combination with a lateral center deviation; when the distortion rate is less than the threshold value, the forward speed remains the basic speed, the last two are zero, and the above-mentioned three-axis speeds are combined to generate chassis control instructions to drive the inspection robot to complete accurate alignment.

Description

Technical Field

[0001] This application relates to the field of mobile robots for industrial fixed equipment inspection, specifically a system for confirming the completion of inspection point acquisition based on a task state machine. Background Technology

[0002] In industrial fixed equipment inspection scenarios, inspection robots rely on target detection networks to extract the detection frames of instruments, switch boxes, and other equipment in real time. They then generate forward speed based on the difference between the detection frame height and the desired height, and generate turning or lateral movement speed based on the deviation between the detection frame center and the image center, thereby completing the visual alignment of the equipment. This approach generally assumes that changes in the size of the detection frame are only caused by physical distance, and that the robot always faces the equipment during the approach process.

[0003] In actual inspection scenarios, robots often cannot always face the equipment perpendicularly due to obstacle avoidance or uneven ground, but instead approach at a certain angle. Once the lens is tilted, the outline of the equipment captured is no longer the shape it would have when facing it directly; the width and height will shrink proportionally, causing the inspection frame to undergo shape compression, rather than just becoming smaller overall. Current methods only focus on the overall size of the inspection frame and do not separately identify this shape compression, thus treating the shrinkage caused by lens tilt and the shrinkage caused by increased distance together.

[0004] Operating in this manner, the robot interprets the shrinkage caused by lens skew as insufficient distance and continues to output forward commands, easily colliding with equipment while moving forward, or causing equipment to slip out of the frame during turning. Once the target is lost, the system often triggers a reversal to search for the target again, repeatedly getting caught in a cycle of collision, loss, and search, which is a common cause of failure in field operations for pan-tilt-zoom (PTZ) inspection robots. Therefore, how to identify the shape compression caused by lens skew from changes in the detection frame, and correct the robot's approach path accordingly to avoid collisions and target loss, is a technical problem that urgently needs to be solved in the vision alignment stage. Summary of the Invention

[0005] The purpose of this application is to provide a system for confirming the completion of inspection point acquisition based on a task state machine. This system aims to overcome the risks of distance misjudgment and collision caused by the neglect of perspective distortion under non-orthogonal tangential orientation in existing vision alignment schemes, thereby improving the effective acquisition rate and operational safety of inspection robots in confined spaces.

[0006] The objective of this application can be achieved through the following technical solution: Firstly, a system for confirming the completion of inspection point acquisition based on a task state machine includes the following modules:

[0007] The data acquisition module is used to obtain the reference aspect ratio preset by the device to be aligned, as well as the pixel width sequence and pixel height sequence continuously acquired by the target detection box;

[0008] The data processing module is used to perform time-domain averaging operations on the pixel width sequence and pixel height sequence respectively to generate observation width and observation height, and generate a basic approximation speed based on the observation height and the preset expected acquisition height.

[0009] The yaw assessment module is used to obtain the real-time observed width-to-length ratio of the target detection box based on the observed width and observed height, and to obtain the target box distortion rate characterizing the yaw state by combining the reference width-to-length ratio.

[0010] The first generation module is used to perform a reduction operation on the basic approximation velocity based on the target box distortion rate to generate a forward command velocity when the target box distortion rate is greater than or equal to a preset distortion threshold, extract the lateral center deviation of the target detection box and generate the normal registration angular velocity and the offset lateral movement velocity respectively in combination with the target box distortion rate.

[0011] The second generation module is used to take the basic approximation velocity as the corresponding forward command velocity when the distortion rate of the target box is less than the preset distortion threshold, and to set the normal registration angular velocity and the offset lateral movement velocity to zero.

[0012] The command drive module is used to combine the forward command speed, normal registration angular velocity and counter-movement lateral velocity to generate chassis motion control commands, so as to drive the inspection robot to perform pose compensation movement.

[0013] Secondly, the method for confirming the completion of inspection point collection based on the task state machine includes the following steps:

[0014] Obtain the reference aspect ratio preset by the device to be aligned, as well as the pixel width sequence and pixel height sequence continuously acquired by the target detection box;

[0015] The pixel width sequence and pixel height sequence are respectively subjected to temporal averaging to generate observation width and observation height respectively. A basic approximation speed is generated based on the observation height and the preset expected acquisition height.

[0016] The real-time observed width-to-length ratio of the target detection box is obtained based on the observed width and observed height, and the target box distortion rate, which characterizes the yaw state, is obtained by combining the reference width-to-length ratio.

[0017] When the distortion rate of the target box is greater than or equal to the preset distortion threshold, the basic approximation velocity is reduced based on the distortion rate of the target box to generate the forward command velocity. The lateral center deviation of the target detection box is extracted and combined with the distortion rate of the target box to generate the normal registration angular velocity and the counter-movement lateral velocity respectively.

[0018] When the distortion rate of the target box is less than the preset distortion threshold, the basic approximation velocity is used as the corresponding forward command velocity, and the normal registration angular velocity and the offset lateral movement velocity are both set to zero.

[0019] The forward command speed, normal registration angular velocity, and counter-impact lateral movement speed are combined to generate chassis motion control commands to drive the inspection robot to perform pose compensation movement.

[0020] Thirdly, a computer storage medium stores computer-executable instructions, which, when executed, implement the inspection point acquisition completion confirmation system based on the task state machine described in the first aspect.

[0021] Compared with the prior art, the beneficial effects of this application are:

[0022] This application can identify shape compression caused by skewed entry from changes in the detection box, avoiding misjudging it as being too far away; it can achieve multi-axis collaborative posture compensation without adding additional hardware such as a gimbal; it can overcome the distance misjudgment and collision risk caused by the neglect of perspective distortion under non-orthogonal entry posture in existing vision alignment schemes, and significantly improve the effective acquisition reach rate of robots in narrow and confined spaces. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the module of the inspection point collection completion confirmation system based on the task state machine in this application;

[0024] Figure 2 This is a schematic diagram illustrating the steps of the inspection point collection completion confirmation method based on the task state machine in this application. Detailed Implementation

[0025] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but only to illustrate selected embodiments of this application.

[0026] Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item has been defined in one figure, it does not need to be further defined and explained in subsequent figures. The terms "first", "second", etc. are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0027] In industrial fixed equipment inspection scenarios such as waterworks and pipe corridors, robots typically arrive at each inspection point sequentially according to a preset route. They then use a target detection network to extract the detection frames of equipment such as instruments and switch boxes. The robot generates its forward speed based on the difference between the detection frame height and the desired height, and generates its turning or lateral movement speed based on the deviation between the detection frame center and the image center. This completes the visual alignment of the equipment and allows for the collection of the necessary inspection data for that point. This approach assumes that the size change of the detection frame is only determined by the physical distance, and that the robot always faces the equipment surface during the approach process, without involving additional posture calculation steps.

[0028] In real-world inspection scenarios, robots often cannot always face the equipment perpendicularly due to obstacle avoidance or uneven ground, and frequently approach the target point at a certain angle. At this point, an angle appears between the lens's optical axis and the normal to the equipment surface. According to the imaging principle of pinhole cameras, the projection of the target onto the image plane undergoes affine deformation: the shrinkage ratio in the width and height directions is no longer equal, and the detection frame is flattened from its original shape into another shape; for example, a circular dial is compressed into an ellipse, or the side length ratio of a square switch box changes. Current methods only focus on the overall size scaling of the detection frame, failing to extract this shape compression separately. Therefore, the shape compression caused by lens tilt is treated together with the overall shrinkage caused by increased distance.

[0029] When operating under this logic, the robot may misinterpret the dimensional shrinkage caused by skew as insufficient distance and continue to output forward commands, easily colliding with equipment flanges or instrument boxes at the final stage of approach. If the robot then performs a turning maneuver to correct the skew, the optical center swing caused by the turning may cause the equipment to momentarily slip out of the camera's field of view, triggering a stationary turning search, creating a cycle of collision, target loss, and re-search. This is a common reason for the failure of pan-tilt-zoom (PTZ) inspection robots in confined spaces. The above problems are particularly prominent in scenarios with narrow passages and where large-scale turning and re-approaching are not possible. The robot is often forced to switch to manual intervention due to repeated failures, reducing the automation level of the inspection task.

[0030] Existing conventional solutions often start by lowering the scaling factor of the velocity loop, attempting to reduce the consequences of a collision by slowing down the approach speed. However, such adjustments do not extract yaw information from the shape changes of the detection box. Therefore, they cannot distinguish between two fundamentally different situations: the actual distance is far and the visual size is reduced due to yaw. They can only sacrifice global approach efficiency to obtain a limited safety margin, and they cannot actively perform attitude correction when yaw occurs. The root cause of the problem has never been addressed.

[0031] This invention breaks the implicit assumption that the size of the detection frame only changes with distance. It reconstructs the original one-dimensional distance deviation compensation into a three-dimensional spatial surround compensation logic based on deformation rate constraints: by comparing the observed width-to-length ratio with the preset reference width-to-length ratio, the distortion rate of the target frame representing the yaw state is quantified; then, using this distortion rate as the core variable, a nonlinear reduction is performed on the forward approach velocity, and at the same time, the normal registration angular velocity and the counter-movement lateral velocity are jointly generated, driving the robot to actively perform attitude correction while approaching the device, thereby eliminating the risk of distance misjudgment and collision under non-orthogonal approach attitude without adding additional hardware such as a gimbal.

[0032] Therefore, such as Figure 1 As shown, this application provides a system for confirming the completion of inspection point data collection based on a task state machine, including the following modules:

[0033] The data acquisition module is used to obtain the reference aspect ratio preset by the device to be aligned, as well as the pixel width sequence and pixel height sequence continuously acquired by the target detection box;

[0034] The data processing module is used to perform time-domain averaging operations on the pixel width sequence and pixel height sequence respectively to generate observation width and observation height, and generate a basic approximation speed based on the observation height and the preset expected acquisition height.

[0035] The yaw assessment module is used to obtain the real-time observed width-to-length ratio of the target detection box based on the observed width and observed height, and to obtain the target box distortion rate characterizing the yaw state by combining the reference width-to-length ratio.

[0036] The first generation module is used to perform a reduction operation on the basic approximation velocity based on the target box distortion rate to generate a forward command velocity when the target box distortion rate is greater than or equal to a preset distortion threshold, extract the lateral center deviation of the target detection box and generate the normal registration angular velocity and the offset lateral movement velocity respectively in combination with the target box distortion rate.

[0037] The second generation module is used to take the basic approximation velocity as the corresponding forward command velocity when the distortion rate of the target box is less than the preset distortion threshold, and to set the normal registration angular velocity and the offset lateral movement velocity to zero.

[0038] The command drive module is used to combine the forward command speed, normal registration angular velocity and counter-movement lateral velocity to generate chassis motion control commands, so as to drive the inspection robot to perform pose compensation movement.

[0039] In another implementation, such as Figure 2 As shown, this application also provides a method for confirming the completion of inspection point data collection based on a task state machine, including the following steps:

[0040] Obtain the reference aspect ratio preset by the device to be aligned, as well as the pixel width sequence and pixel height sequence continuously acquired by the target detection box;

[0041] The pixel width sequence and pixel height sequence are respectively subjected to temporal averaging to generate observation width and observation height respectively. A basic approximation speed is generated based on the observation height and the preset expected acquisition height.

[0042] The real-time observed width-to-length ratio of the target detection box is obtained based on the observed width and observed height, and the target box distortion rate, which characterizes the yaw state, is obtained by combining the reference width-to-length ratio.

[0043] When the distortion rate of the target box is greater than or equal to the preset distortion threshold, the basic approximation velocity is reduced based on the distortion rate of the target box to generate the forward command velocity. The lateral center deviation of the target detection box is extracted and combined with the distortion rate of the target box to generate the normal registration angular velocity and the counter-movement lateral velocity respectively.

[0044] When the distortion rate of the target box is less than the preset distortion threshold, the basic approximation velocity is used as the corresponding forward command velocity, and the normal registration angular velocity and the offset lateral movement velocity are both set to zero.

[0045] The forward command speed, normal registration angular velocity, and counter-impact lateral movement speed are combined to generate chassis motion control commands to drive the inspection robot to perform pose compensation movement.

[0046] 1. Obtain the reference aspect ratio and the pixel sequence of the target detection box;

[0047] The inputs to this step are the reference aspect ratio preset by the device to be aligned during the point task configuration phase, and the pixel-level size of the target detection box output by the visual detection model continuously running on the edge computing node; the outputs are the reference aspect ratio values ​​for subsequent steps, and the original sequence of pixel width and pixel height after queue maintenance.

[0048] By pre-calibrating the aspect ratio of the physical contour in the orthogonal state for each device to be inspected during the point task binding stage, and continuously collecting the pixel size of the target detection box during the operation stage, this invention provides a unified comparison benchmark and source of original observation data for subsequent quantification of perspective distortion, avoiding the jitter caused by frame-by-frame individual judgment.

[0049] In practice, before performing the task at each location, the inspection robot enters the reference width-to-length ratio for each device to be aligned via a configuration interface on a host computer. For example, the reference width-to-length ratio of a circular dial can be set to 1.0, and the reference width-to-length ratio of a square switch box can be set to 1.5. This parameter is stored numerically in the location task binding table for the robot to read and call when it arrives at the corresponding location. After the robot arrives at a location according to the preset inspection route, the task state machine switches from the idle state to the alignment state, triggering the start of the vision alignment closed loop.

[0050] Subsequently, edge computing nodes (such as onboard computing units equipped with embedded processors) receive images captured by the camera at fixed intervals and call a preset visual detection model (such as a single-stage object detection network based on YOLO or equivalent precision) to perform inference on each frame of the image, outputting the pixel-level width and pixel-level height of the object detection box in the image coordinate system. The system maintains a first-in-first-out data structure of a preset number of frames in memory. Each time a new detection result is received, its pixel width and pixel height are pushed into the corresponding queue and the oldest frame data is removed, thereby dynamically maintaining a fixed-length sequence of pixel width and pixel height according to the acquisition time sequence for subsequent steps.

[0051] If a target detection box is not detected in a certain frame, the task state machine can maintain the current queue and wait for the next frame's valid detection result, avoiding interruption of the entire acquisition and alignment process due to a single frame's missed detection. Since the types of devices to be aligned are different for different inspection points, the reference aspect ratios are also different. Each time a new point task is entered, the system will reread the reference aspect ratio and expected acquisition height corresponding to that point, and clear the pixel width and pixel height sequences left over from the previous point, avoiding mutual interference of historical data across points, and ensuring that each acquisition and alignment is based on the independent reference of the current point device.

[0052] II. Generate observation width, observation height, and basic approximation speed;

[0053] The inputs to this step are the pixel width and pixel height sequences output above, as well as the system's preset desired acquisition height; the outputs are the smoothed observation width and observation height, and the base approximation speed generated accordingly.

[0054] By performing temporal averaging on the pixel size of multiple consecutive frames, this invention eliminates pixel-level high-frequency jumps in the visual detection model output caused by factors such as high-frequency chassis vibration and mechanical transmission. Then, by using the difference between the smoothed observation height and the desired acquisition height, an initial dynamic benchmark for characterizing the physical approximation distance is established.

[0055] Specifically, the processing terminal extracts the pixel width and pixel height of target detection boxes from multiple consecutively acquired frames in chronological order, and constructs pixel width and pixel height sequences with a preset number of frames each. Then, it obtains the arithmetic mean of each value in the pixel width and pixel height sequences to generate smoothed observation width and observation height. The calculation relationship is as follows:

[0056] ;

[0057] ;

[0058] in, Represents the preset number of frames (dimensionless positive integer). and Representing the pixel width sequence and the pixel height sequence respectively, the first... The original pixel width and pixel height of the frame. and The smoothed observation width and observation height are represented respectively. This temporal smoothing process introduces only a slight phase hysteresis equivalent to the preset frame rate at a typical camera acquisition frame rate, which is far lower than the chassis kinematic response period. It can filter out spurious deformation noise caused by mechanical vibration from the data source without affecting the real-time control performance.

[0059] After obtaining the smoothed observation height, the processing terminal subtracts the observation height from the preset desired acquisition height, using the difference as the distance proxy error for mapping physical space depth. This distance proxy error is then multiplied by a preset approximation ratio gain coefficient to generate the base approximation velocity. The calculation relationship is as follows:

[0060] ;

[0061] in, The preset expected acquisition height (unit: pixels) is physically the standard pixel height of the target device in the image when the robot's optical center is at the optimal shooting distance. This value is usually pre-configured during the point-to-point task binding stage. The preset approximation ratio gain coefficient (unit: mm / s / pixel) is pre-calibrated by engineers based on the chassis dynamic response characteristics; Characterizes the basic approximation velocity (unit: mm / s).

[0062] It should be noted that this basic approximation velocity is a longitudinal drive reference value derived under the ideal assumption that the robot's optical axis is strictly perpendicular to the target device surface. In subsequent steps, if perspective distortion is detected in the target box, this basic approximation velocity will no longer be fully distributed, but will instead be used as a base value subject to a nonlinear reduction constraint on the distortion rate. This design decouples the coarse estimation of the distance dimension from the fine compensation of the yaw dimension: the distance proxy error only needs to be converted into a reference velocity according to a linear proportional relationship, without considering the yaw factor in this step, thus keeping the calculation logic simple. The attitude-related corrections are uniformly left to be centrally processed in subsequent steps based on the target box distortion rate.

[0063] 3. Obtain the target bounding box distortion rate;

[0064] The inputs for this step are the observation width and observation height from the above outputs, as well as the preset reference aspect ratio; the output is the quantized target box distortion rate.

[0065] By subtracting the observed width-to-length ratio from the reference width-to-length ratio and taking the absolute value, this invention separates the shape compression caused by lens misalignment from the overall size change of the detection frame, obtaining a scalar index that only reflects the degree of misalignment and does not carry the direction of misalignment, providing a basis for subsequent state determination.

[0066] Specifically, the processing terminal divides the observation width by the observation height to generate the observation width-to-length ratio, and the calculation relationship is as follows:

[0067] ;

[0068] in, The observed aspect ratio of the target detection box is used in real time. Then, the processing terminal takes the absolute value of the difference between the observed aspect ratio and the reference aspect ratio to remove the yaw direction attribute, generating the target box distortion rate representing the yaw state. The calculation relationship is as follows:

[0069] ;

[0070] in, Characterizes the preset reference width-to-length ratio, The target bounding box distortion rate is characterized (dimensionless, always non-negative). Based on the imaging principle of a pinhole camera, when the camera's optical axis tilts horizontally relative to the normal of the target device's surface, the lateral shrinkage rate of the target's projection onto the sensor plane is related to the yaw angle and independent of the target's physical distance, belonging to pure affine distortion. The absolute value processing method ensures that regardless of whether the robot yaws to the left or right, as long as the severity of the yaw is the same, the calculated target bounding box distortion rate will be the same. This decouples the subsequent state determination and three-axis velocity reconstruction logic from the yaw direction, simplifying the complexity of the control strategy.

[0071] For example, when the observed aspect ratio is 0.90 and the reference aspect ratio is 1.0, the resulting target box distortion rate is 0.10. This value will be used as the direct basis for state determination in the next step. It should be noted that the value of the target box distortion rate is only related to the yaw of the lens relative to the surface of the device, and is independent of the absolute physical distance between the robot and the device. That is, regardless of whether the robot is at a far or near physical position, as long as the yaw angle is the same, the calculated target box distortion rate will be basically the same. This is the physical basis for the present invention to use the distortion rate as a state determination basis independent of distance error and to trigger the attitude compensation branch independently.

[0072] IV. State determination and three-axis velocity command generation based on target box distortion rate;

[0073] The inputs for this step are the target bounding box distortion rate, basic approximation speed, preset distortion threshold, and the horizontal center pixel coordinates and image reference width of the target detection box in the current frame, as well as the final forward command speed, normal registration angular velocity, and offset lateral movement speed.

[0074] By introducing the target box distortion rate as the sole trigger condition for state determination, this invention breaks away from the previous processing method of generating the speeds of the forward, turning, and lateral directions independently. Instead, it strongly binds the three into a set of attitude compensation commands uniformly modulated by the distortion rate, enabling the robot to reduce its forward speed, actively turn back to center, and laterally offset the field of view shift caused by the turning when yawing into the device, thus forming a circling approach trajectory around the device.

[0075] The processing terminal compares the distortion rate of the generated target box with a preset distortion threshold: when the distortion rate of the target box is greater than or equal to the preset distortion threshold, it is determined that the current state is yaw cut-in and the attitude compensation branch is triggered; when the distortion rate of the target box is less than the preset distortion threshold, it is determined that the current state is approximately orthogonal approximation and the straight branch is triggered.

[0076] The aforementioned yaw entry state and approximately orthogonal approximation state constitute the two basic states for the task state machine of this invention to cyclically determine the point alignment stage: In each acquisition cycle, the robot switches between the two states according to the latest calculated target box distortion rate, and issues the corresponding three-axis speed command according to the corresponding state. This process is repeated until the basic approximation speed converges to near the preset range and the target box distortion rate is stably lower than the preset distortion threshold. At this time, the task state machine determines that the acquisition and alignment of the point has been completed, and enters the completion state, thereby confirming that the data acquisition of the inspection point has been completed and can wait to execute the next point task.

[0077] In the straight-line branch, the processing terminal directly uses the basic approximation velocity as the corresponding forward command velocity. Issued, and the normal registration angular velocity was set. With the hedging lateral speed All are set to zero, that is:

[0078] ;

[0079] ;

[0080] ;

[0081] This branch corresponds to scenarios where the target detection box does not undergo significant shape compression. In this case, outputting the base approximation speed in full according to the existing orthogonal approximation logic is sufficient to meet the alignment accuracy requirements, without triggering additional attitude compensation calculations. This maintains the smoothness of the approximation trajectory in the orthogonal state and avoids unnecessary attitude disturbances. This branch is also a common termination state at the end of the robot's approximation process: after the robot has basically returned to normal after several previous acquisition cycles of attitude compensation, and the target box distortion rate has dropped below the preset distortion threshold, the control logic naturally retreats to the straight-line branch and smoothly completes the approximation of the remaining distance at the base approximation speed, avoiding unnecessary lateral movement and rotation even when the attitude has converged.

[0082] In the pose compensation branch, perspective compression at tilted angles causes the pixel height of the detection box to be less than its actual height in an orthogonal state at the same physical distance. If it continues to advance at the base approximation speed, the actual approximation distance may be less than the system's expectation, leading to overshoot collisions. Therefore, the processing terminal first obtains the system's preset distortion penalty mapping coefficient and multiplies the target box distortion rate by this coefficient to generate a penalty angle scalar. The calculation relationship is as follows:

[0083] ;

[0084] in, Characterizes the pre-defined distortion penalty mapping coefficients. The generated penalty angle scalar (unit: radians) is then used by the processing terminal to perform a cosine function operation on the penalty angle scalar to extract a scaling factor that limits over-entry. The base approximation velocity is then multiplied by the scaling factor to generate a forward command velocity with attitude damping attenuation characteristics. The calculation relationship is as follows:

[0085] ;

[0086] The engineering basis for choosing the cosine function as the scaling factor is that when the distortion rate is zero, the cosine value is 1, and the forward velocity is not attenuated; as the distortion rate increases, the cosine value exhibits a nonlinear monotonically decreasing characteristic from gentle to steep, which perfectly matches the control requirement that the more severe the yaw, the more the forward velocity needs to be attenuated, thus realizing a deterministic nonlinear reduction of the basic approximation velocity.

[0087] When the penalty angle scalar approaches half of pi, the cosine value approaches zero, and the forward command speed also approaches zero. This is equivalent to automatically triggering almost complete forward braking when the yaw degree reaches the preset limit, thus gaining sufficient time and space for attitude compensation and preventing the robot from continuing to approach the equipment before completing attitude correction.

[0088] This invention further specifies that the distortion penalty mapping coefficient is not fixed, but dynamically adjusted according to the spatial safety depth between the robot and the equipment. The processing terminal first obtains a preset safe approach speed threshold and compares it with the current basic approach speed: when the basic approach speed is greater than or equal to the safe approach speed threshold, it is determined that the robot has sufficient safety depth, and the system-preset constant coefficient is maintained as the distortion penalty mapping coefficient; when the basic approach speed is less than the safe approach speed threshold, it is determined that the robot has approached the equipment and is in a high-risk condition, and the ratio of the safe approach speed threshold to the basic approach speed is used as an amplification factor, the calculation relationship of which is as follows:

[0089] ;

[0090] The product of the constant coefficient and the amplification factor is then used as the replaced distortion penalty mapping coefficient, and the calculation relationship is as follows:

[0091] ;

[0092] in, Characterizes the preset safe cut-in speed threshold (unit: mm / s). Characterizing the system's preset constant coefficients, The amplification factor (dimensionless) is used to characterize the basic approximation velocity. Since the basic approximation velocity is essentially a linear surrogate quantity of the physical approximation distance, the smaller its value, the closer the physical distance between the robot and the device. This invention utilizes this inverse amplification relationship to automatically increase the distortion penalty mapping coefficient as the robot gets closer to the device, thereby enhancing the reduction force of the penalty angle scalar and the forward velocity. This achieves a differentiated control boundary that allows for moderate yaw and smooth entry at long distances, while executing strict braking at close distances, avoiding the problem of insufficient braking force in extremely close-range scenarios with a uniform coefficient.

[0093] For example, if the base approach speed is reduced to half of the safe approach speed threshold, the amplification factor is 2, and the replacement distortion penalty mapping coefficient is increased to twice the constant coefficient. This causes the same size target box distortion rate to trigger a larger penalty angle scalar and generate a lower cosine scaling factor, which further reduces the forward command speed. Thus, when the robot is already very close to the device, it is restricted from moving forward with stronger damping force, reducing the probability of end-effector scraping or collision.

[0094] After generating the reduced forward command velocity, the processing terminal further generates a normal registration angular velocity to correct the robot's yaw attitude, and a counteracting lateral movement velocity to offset the lateral jerking caused by the rotational motion on the camera's optical center. First, the processing terminal extracts the lateral center pixel coordinates of the target detection box and the image reference width. Based on these, it generates a lateral center deviation characterizing the distance the target deviates from the optical center. Then, it obtains a preset angular velocity gain coefficient. This gain coefficient is multiplied by the lateral center deviation and the target box distortion rate to generate the normal registration angular velocity for active yaw. The calculation relationship is as follows:

[0095] ;

[0096] in, The horizontal center pixel coordinates (in pixels) of the target detection bounding box. Characterizes the baseline width of the image (unit: pixels). The preset angular velocity gain coefficient (unit: radians / second / pixel) is used in this formula. This formula incorporates the target bounding box distortion rate as a weighting factor into the rotation control, ensuring that the system outputs a large rotational angular velocity only when the target deviates from the image center and significant perspective distortion exists. If the target deviates from the center but the distortion rate is extremely small, meaning the robot is in a pure lateral slip rather than a yaw state, the system will suppress rotational movements. This distinguishes between lateral slip and yaw, avoiding miscorrection of simple translation scenes.

[0097] Compared to conventional proportional control methods that generate angular velocities based solely on the lateral center deviation, this invention incorporates the target box distortion rate into the multiplication of angular velocities. This ensures that the magnitude of the normal registration angular velocity depends not only on the target's deviation from the image but also on the severity of the current attitude yaw. The combined effect of these two factors prevents unnecessary rotational actions from being output due to occasional center offsets when the distortion rate is low and the robot's attitude is basically facing the device.

[0098] Because cameras are typically mounted outside the rotation center of the chassis and have a fixed physical front arm length, when the robot performs a stationary rotation, the camera's optical center will trace an arc in space, generating additional tangential linear velocity. If this is not counteracted, targets originally located at the edge of the frame can easily slip out of the field of view instantly. Therefore, the processing terminal negates the product of the normal registration angular velocity and the preset physical front arm length parameter to generate a counteracting lateral velocity with the opposite sign. The calculation relationship is as follows:

[0099] ;

[0100] in, The preset physical front arm length parameter (unit: millimeters) represents the longitudinal vertical distance between the camera optical center and the chassis rotation center, which is usually a rigid constant given in the robot structural drawings. The negative velocity represents the offset lateral velocity (unit: mm / s). This negative velocity generates a lateral displacement vector in physical space that is equal in magnitude and opposite in direction to the tangential linear velocity. It utilizes the omnidirectional translational degrees of freedom of the Mecanum wheel chassis, effectively canceling out the absolute horizontal displacement of the camera's optical center during rotation. Macroscopically, the robot forms a circular trajectory around the device, rather than simply rotating in place, thus fundamentally avoiding the loss of field of view caused by rotation.

[0101] At this point, the forward command speed, normal registration angular velocity, and counter-movement lateral velocity have all been generated. These three are coupled with each other and are mutually causal, together forming a complete speed command set that drives the robot to perform three-dimensional posture compensation movement, upgrading the traditional one-dimensional forward approximation to a spatial orbital approximation around the target device.

[0102] 5. Generate chassis motion control commands and drive their execution;

[0103] The inputs for this step are the forward command velocity, normal registration angular velocity, and counter-movement lateral velocity generated above; the output is the motion control command that drives the inspection robot chassis to perform pose compensation movement.

[0104] By converting three floating-point velocity values ​​into fixed-length integer instruction data and adding a verification mechanism, this invention solves the execution deviation problem caused by data format differences and communication interference between the upper-level vision calculation terminal and the lower-level motion controller, ensuring that the three-dimensional attitude calculation results can be converted into the actual motion of the chassis without loss and reliably.

[0105] In a typical inspection robot architecture, visual reasoning and 3D kinematics calculations are usually performed by an edge computing motherboard running a general-purpose operating system, and the results are floating-point data with multiple decimal places. Meanwhile, the motors are often driven by resource-constrained low-level motion control chips. If these floating-point numbers are transmitted directly via a serial link, not only is the data volume excessive, but different chips are also prone to errors in interpreting floating-point memory formats. Furthermore, electromagnetic interference in industrial environments can cause individual data bits to flip during transmission, leading to unexpected chassis movements. This invention eliminates these risks through two stages: de-floating-point conversion and verification encapsulation.

[0106] Specifically, the processing terminal performs rounding operations on the finally generated forward command speed and the counter-movement speed, and performs rounding operations on the finally generated normal registration angular velocity after multiplying it by a preset amplification factor, so as to generate command data that eliminates floating-point precision. Since the original value of the normal registration angular velocity is small, direct rounding can easily lead to the control quantity returning to zero and precision loss. Therefore, it is necessary to amplify it first and then round it, converting the value in radians / second into an integer value in milliradians / second after multiplying it by a preset amplification factor.

[0107] Subsequently, the processing terminal loads the instruction data into the service data segment of the downlink communication frame, usually in the order of high byte first and low byte last, to avoid byte order parsing differences across chip architectures; preset frame header identifiers, frame tail identifiers, and check bits generated based on the service data segment are added to the downlink communication frame to generate chassis motion control instructions with data verification mechanism.

[0108] The check bit can be XOR check, which involves performing an XOR operation on each byte of the business data segment to obtain a single-byte check value; or it can be an algorithm with stronger error correction and recognition capabilities, such as cyclic redundancy check. Both can identify transmission errors with lower computational overhead. The specific choice can be made based on the intensity of electromagnetic interference on site and the computing power margin of the main control chip, ensuring the effectiveness of the check while controlling the parsing overhead of the underlying motion control chip.

[0109] After receiving the instruction, the underlying motion controller (such as a motion control chip based on an embedded real-time system) first compares the frame header and frame tail identifiers and checks the check bit. If the check passes, the instruction data is extracted and restored to the target speed according to the corresponding magnification factor. The data is then substituted into the preset Mecanum wheel inverse kinematics equation matrix to calculate the target speed of each of the four wheels. Then, the pulse width modulation signal is output through closed-loop speed regulation to drive the drive motors corresponding to the four Mecanum wheels, thereby completing the posture compensation movement of the chassis.

[0110] If the verification fails, the command frame is determined to have been interfered with during transmission. The frame is then discarded, and the robot maintains its original motion state or performs a safe stop, avoiding malfunctions caused by communication interference. Since visual calculation and command issuance are executed cyclically in a fixed period, even if individual frames are discarded due to verification failure, the robot only maintains its original speed within that period. The next acquisition cycle can then continue issuing commands based on the calculation results of the new frame, preventing the overall control flow from being interrupted by a single communication anomaly.

[0111] To verify the feasibility and technical effectiveness of this solution, an application example with specific numerical values ​​is provided, covering the complete processing flow from the input of the original pixel sequence to the output of the chassis motion control command.

[0112] Assume a reference width-to-length ratio of a certain device to be aligned. Preset expected acquisition height Pixels, preset frame rate Approximation of proportional gain coefficient mm / s / pixel, preset distortion threshold The constant part of the distortion penalty mapping coefficients Preset safe cut-in speed threshold mm / s, angular velocity gain coefficient Radius / second / pixel, image baseline width Pixels, physical front arm length parameters Millimeters.

[0113] After the robot cuts into the device at an angle, the visual detection model continuously outputs five frames of target detection bounding boxes with pixel widths of 92, 89, 90, 91, and 88, and pixel heights of 101, 98, 102, 99, and 100, respectively, and horizontal center pixel coordinates... .

[0114] First, calculate the smoothed observation width and observation height: Pixels Pixels; based on this, the fundamental approximation velocity is generated. mm / s.

[0115] Then calculate the observed width-to-length ratio Target bounding box distortion rate .

[0116] Perform state determination: because Greater than or equal to the preset distortion threshold This triggers the attitude compensation branch. Because... mm / s greater than or equal to the safe cut-in speed threshold mm / s, distortion penalty mapping coefficients remain constant Based on this, the penalty angle scalar is calculated. Radius, forward command speed mm / s. Normal registration angular velocity Radius per second; counter-current lateral velocity mm / s.

[0117] The three-axis velocities mentioned above are rounded and amplified: the forward command velocity is rounded to 44 mm / s, the counter-movement velocity is rounded to -200 mm / s, and the normal registration angular velocity is multiplied by a preset amplification factor of 1000 and rounded to 500 mradians / s. These three values ​​are loaded into a downlink communication frame with a header, trailer, and checksum appended before being sent to the underlying motion controller. The underlying motion controller parses the command data and reconstructs it into the target velocity. After substituting it into the inverse kinematics matrix of the Mecanum wheel, it drives the robot to move forward while simultaneously moving laterally to the right and rotating counterclockwise, executing an arc-shaped approximation trajectory around the device.

[0118] As the robot's posture gradually returns to normal, the target box distortion rate continues to decrease in subsequent acquisition cycles. The task state machine also dynamically switches between the yaw entry state and the approximately orthogonal approximation state. When the target box distortion rate stably converges to within the preset distortion threshold and the basic approximation velocity converges to near the preset range, the task state machine determines that the acquisition alignment of the point has been completed, confirming that the data acquisition of the inspection point has been completed.

[0119] Through the above technical solutions, the present invention can achieve the following technical effects:

[0120] In response to the shortcomings of existing technologies that attribute the overall reduction in the size of the detection frame entirely to excessive distance and fail to identify the shape compression caused by the oblique cut, this invention generates the target frame distortion rate by comparing the absolute difference between the observed width-to-length ratio and the reference width-to-length ratio. This allows the invention to distinguish between the deformation caused by lens obliqueness and the overall reduction caused by distance changes, thus avoiding misjudging oblique cut as insufficient distance.

[0121] To address the shortcomings of existing technologies that continuously output forward commands due to distance misjudgment, leading to robot collisions with equipment or targets sliding out of the field of view, and thus getting stuck in a collision and re-search cycle, this invention performs nonlinear reduction on the basic approximation velocity based on the target box distortion rate, and couples the generation of normal registration angular velocity and counter-movement lateral velocity. This enables the robot to actively perform attitude correction and field of view latching while reducing its forward speed, binding the actions in the forward, turning and lateral directions into a unified modulated whole, eliminating the risk of collision and target loss under non-orthogonal cutting posture from the root.

[0122] To address the shortcomings of existing technologies where distortion penalty is statically rigid and cannot adapt to different safety requirements at long and short distances, this invention dynamically adjusts the distortion penalty mapping coefficient based on the comparison between the base approach speed and the preset safe entry speed threshold. This enables the robot to automatically increase the descent force when approaching the device, achieving differentiated control between smooth entry at long distances and strong damping braking at short distances, thus compensating for the insufficient safety margin of a uniform coefficient in extremely close-range scenarios.

[0123] To address the shortcomings of existing technologies that rely on additional hardware such as gimbals to achieve posture adjustment, leading to increased costs and structural complexity, this invention utilizes the omnidirectional mobility of the Mecanum wheels and completes posture compensation through the coordinated motion of chassis rotation and lateral movement. It can achieve active adjustment of the line of sight without the need for any rotating parts, and significantly improves the robot's effective acquisition rate in narrow and confined spaces without increasing the cost of additional hardware such as gimbals.

[0124] To address the shortcomings of existing technologies where visual computation results are transmitted in floating-point format and are susceptible to communication interference and chip resolution differences, this invention converts the three-axis velocity data from floating-point to fixed-length integer instruction data and adds frame headers, frame tails, and check bits, thereby improving the reliability of the process of sending the visual algorithm output to the underlying execution mechanism.

[0125] To address the shortcomings of existing technologies, such as the lack of a unified criterion for determining the completion of data acquisition at a location, the tendency for alignment to end prematurely before the attitude has converged, or the inability to converge for an extended period, thus consuming task time, this invention uses the convergence of the target bounding box distortion rate and the basic approximation velocity to drive the task state machine to cyclically determine between the yaw entry state and the approximately orthogonal approximation state. Only when both convergence conditions are met simultaneously is the data acquisition at the inspection location confirmed as complete. This provides a clear and reliable criterion for the automatic flow of multi-point inspection tasks.

[0126] In another embodiment, this application also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned inspection point acquisition completion confirmation system based on a task state machine.

[0127] The above embodiments are only used to illustrate the technical methods of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of this application without departing from the spirit and scope of the technical methods of this application.

Claims

1. A system for confirming the completion of inspection point data collection based on a task state machine, characterized in that, Includes the following modules: The data acquisition module is used to obtain the reference aspect ratio preset by the device to be aligned, as well as the pixel width sequence and pixel height sequence continuously acquired by the target detection box; The data processing module is used to perform time-domain averaging operations on the pixel width sequence and pixel height sequence respectively to generate observation width and observation height, and generate a basic approximation speed based on the observation height and the preset expected acquisition height. The yaw assessment module is used to obtain the real-time observed width-to-length ratio of the target detection box based on the observed width and observed height, and to obtain the target box distortion rate characterizing the yaw state by combining the reference width-to-length ratio. The first generation module is used to perform a reduction operation on the basic approximation velocity based on the target box distortion rate to generate a forward command velocity when the target box distortion rate is greater than or equal to a preset distortion threshold, extract the lateral center deviation of the target detection box and generate the normal registration angular velocity and the offset lateral movement velocity respectively in combination with the target box distortion rate. The second generation module is used to take the basic approximation velocity as the corresponding forward command velocity when the distortion rate of the target box is less than the preset distortion threshold, and to set the normal registration angular velocity and the offset lateral movement velocity to zero. The command drive module is used to combine the forward command speed, normal registration angular velocity and counter-movement lateral velocity to generate chassis motion control commands, so as to drive the inspection robot to perform pose compensation movement.

2. The inspection point acquisition completion confirmation system based on task state machine according to claim 1, characterized in that, The process of generating the observation width and observation height includes: Extract the pixel width and pixel height of target detection boxes from multiple consecutively acquired frames in chronological order, and construct pixel width and pixel height sequences with a preset number of frames each; The arithmetic mean of each value in the pixel width sequence and the pixel height sequence is obtained respectively, to generate the smoothed observation width accordingly. With observation altitude .

3. The inspection point acquisition completion confirmation system based on task state machine according to claim 2, characterized in that, The process of generating the basic approximation velocity includes: Set the desired acquisition height Subtract the observed height The obtained difference is used as the distance proxy error for mapping physical space depth. The distance proxy error is then compared with a preset approximation ratio gain coefficient. Perform multiplication operations to generate the basic approximation speed. ; 。 4. The inspection point acquisition completion confirmation system based on task state machine according to claim 2, characterized in that, The process of obtaining the object bounding box distortion rate includes: The observation width Divide by the observed height To generate the observed aspect ratio The observed aspect ratio is compared with the reference aspect ratio. The absolute value of the difference is taken to remove the yaw direction attribute and generate the target box distortion rate that represents the yaw state. ; ； 。 5. The inspection point acquisition completion confirmation system based on task state machine according to claim 1, characterized in that, The process of generating forward command velocity includes: When the target box distortion rate When the distortion rate of the target bounding box is greater than or equal to a preset distortion threshold, the distortion rate of the target bounding box is mapped to a preset distortion penalty coefficient. Perform a multiplication operation to generate a penalized angle scalar; Perform a cosine function operation on the penalty angle scalar to extract the scaling factor that limits overcutting, and approximate the base approximation velocity. Perform a multiplication operation with the scaling factor to generate a forward command velocity with attitude damping attenuation characteristics. ; 。 6. The inspection point acquisition completion confirmation system based on task state machine according to claim 5, characterized in that, The process of obtaining the distortion penalty mapping coefficients includes: When the basic approximation speed When the safe cut-in speed threshold is greater than or equal to the preset threshold, the system-preset constant coefficient is used as the distortion penalty mapping coefficient. ; When the base approximation velocity is less than the safe cut-in velocity threshold, the ratio of the safe cut-in velocity threshold to the base approximation velocity is used as an amplification factor, and the product of the constant coefficient and the amplification factor is used as the replaced distortion penalty mapping coefficient. .

7. The inspection point acquisition completion confirmation system based on task state machine according to claim 5, characterized in that, The processes of generating the normal registration angular velocity and the offset lateral velocity respectively include: Extract the horizontal center pixel coordinates of the target detection bounding box and image baseline width Based on the horizontal center pixel coordinates and the image reference width, a horizontal center deviation representing the distance of the target from the optical center is generated; Set the preset angular velocity gain coefficient With respect to the lateral center deviation and the target box distortion rate The three components are multiplied together to generate the normal registration angular velocity used for active yaw. ; The normal registration angular velocity is compared with the preset physical front arm length parameter. The product is negative to generate a counter-current lateral velocity with the opposite sign. ; ； 。 8. The inspection point acquisition completion confirmation system based on task state machine according to claim 1, characterized in that, The process of generating chassis motion control commands includes: The forward command velocity and the counter-movement velocity are rounded down respectively, and the normal registration angular velocity is multiplied by a preset magnification factor and then rounded down to generate command data with floating-point precision eliminated. The instruction data is loaded into the service data segment of the downlink communication frame. A preset frame header identifier, frame tail identifier, and a check bit generated based on the service data segment are added to the downlink communication frame to generate a chassis motion control instruction with a data verification mechanism.

9. A method for confirming the completion of inspection point data collection based on a task state machine, characterized in that, Includes the following steps: Obtain the reference aspect ratio preset by the device to be aligned, as well as the pixel width sequence and pixel height sequence continuously acquired by the target detection box; The pixel width sequence and pixel height sequence are respectively subjected to temporal averaging to generate observation width and observation height respectively. A basic approximation speed is generated based on the observation height and the preset expected acquisition height. The real-time observed width-to-length ratio of the target detection box is obtained based on the observed width and observed height, and the target box distortion rate, which characterizes the yaw state, is obtained by combining the reference width-to-length ratio. When the distortion rate of the target box is greater than or equal to the preset distortion threshold, the basic approximation velocity is reduced based on the distortion rate of the target box to generate the forward command velocity. The lateral center deviation of the target detection box is extracted and combined with the distortion rate of the target box to generate the normal registration angular velocity and the counter-movement lateral velocity respectively. When the distortion rate of the target box is less than the preset distortion threshold, the basic approximation velocity is used as the corresponding forward command velocity, and the normal registration angular velocity and the offset lateral movement velocity are both set to zero. The forward command speed, normal registration angular velocity, and counter-impact lateral movement speed are combined to generate chassis motion control commands to drive the inspection robot to perform pose compensation movement.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement the inspection point acquisition completion confirmation system based on the task state machine as described in any one of claims 1-8.

Images