A computer vision-based trolley following control method, device and medium

Abstract

The application discloses a computer vision-based object carrying vehicle following control method and device and a medium, and relates to the technical field of intelligent mobile control, which comprises the following steps: determining a current working mode and loading a current mode parameter set corresponding to the current working mode; collecting image data and driving state data based on the current mode parameter set; performing target locking and scene analysis on the image data and the driving state data to obtain a scene perception result; calculating a target relative pose parameter and a terrain state index, and real-time correcting the current mode parameter set to generate a dynamic following corridor; determining a current control state based on the dynamic following corridor; calculating a control instruction based on the current control state and the dynamic following corridor, and online fine-tuning the current mode parameter set according to operation feedback. The application improves the following stability of the object carrying vehicle in multiple scenes, the target shielding recovery capability, the protection capability of forbidden entry areas and the safety control precision in complex environments.

Description

Technical Field

[0001] This invention relates to the field of intelligent mobile control technology, and in particular to a method, device and medium for controlling the following of a cargo vehicle based on computer vision. Background Technology

[0002] In the field of intelligent mobile control and service robot technology, cargo vehicle following control typically employs cameras to acquire images of the front and combines this with operational information such as wheel speed, drive current, and braking status to identify, locate, and control the motion of the target being followed. Conventional methods generally rely on target detection, relative pose estimation, and differential drive control to generate linear and angular velocity commands, enabling the cargo vehicle to autonomously follow personnel or designated targets. Some solutions also incorporate obstacle recognition, area constraint recognition, and state feedback adjustment to improve the driving safety and operational stability of cargo vehicles in applications such as camping, golf, and supermarket handling.

[0003] However, from the perspective of actual control needs, conventional methods still have room for further optimization, mainly in two aspects: First, in dynamic scenarios, target information, obstacle information, restricted area information, and congestion level are usually dispersed in control, lacking a unified constraint expression mechanism, which leads to the need to improve the consistency of control decisions in complex environments; Second, under conditions of terrain changes, short-term target occlusion, and fluctuations in operating status, control parameters are mostly set statically or modified in a limited way, which has a relatively limited ability to respond to real-time scene changes and operational feedback, making it difficult to continuously obtain stable and precise following control effects. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a computer vision-based vehicle following control method to solve the problems of insufficient coordination between perception constraints and control decisions in complex scenarios and the limited adaptability of control parameters to real-time environmental changes in existing technologies.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a computer vision-based vehicle following control method, comprising: receiving a scene mode selection instruction, determining the current working mode, and loading a current mode parameter set corresponding to the current working mode; acquiring a sequence of images ahead, as well as wheel speed, motor current, and braking status based on the current mode parameter set, to obtain image data and driving status data; performing target locking and scene parsing on the image data and driving status data to obtain scene perception results; calculating target relative pose parameters and terrain status indicators based on the scene perception results and driving status data, and performing real-time correction on the current mode parameter set to generate a dynamic following corridor; determining the current control state based on the dynamic following corridor; calculating control instructions based on the current control state and the dynamic following corridor to control the operation of the vehicle, and performing online fine-tuning of the current mode parameter set based on operation feedback.

[0008] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, the current mode parameter set includes a reference following distance, a steering dead zone threshold, a speed response gain, a maximum speed limit, an allowable curvature limit, an emergency braking distance threshold, a parking distance threshold in a restricted area, and a slope compensation coefficient.

[0009] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, the scene perception results include target foot coordinates, target confidence, obstacle area, restricted area and congestion index.

[0010] The congestion index is the ratio of the projected area of ​​a dynamic obstacle in the preset forward region to the total area of ​​the preset forward region.

[0011] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, the step of calculating the target relative pose parameters and terrain state indicators based on the scene perception results and driving state data includes:

[0012] Based on the target's foot coordinates and camera calibration parameters, the target is mapped to the ground coordinate system in front of the cargo vehicle to obtain the target's ground coordinates;

[0013] Calculate the target relative pose parameters to the cargo vehicle based on the target ground coordinates;

[0014] Terrain condition indicators are calculated based on changes in motor current and wheel speed.

[0015] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, the step of real-time correction of the current mode parameter set to generate a dynamic following corridor includes:

[0016] Based on the target relative pose parameters, terrain state indicators, and congestion indicators, the baseline following distance, steering dead zone threshold, maximum speed limit, and allowable curvature limit in the current mode parameter set are corrected.

[0017] A dynamic following corridor is generated based on the modified parameters, including the expected following distance, allowable lateral deviation range, allowable speed range, allowable curvature range, and parking boundary position.

[0018] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, wherein: determining the current control state based on the dynamic following corridor includes:

[0019] When the target confidence level is greater than the preset confidence threshold and there are no obstacles blocking the dynamic following corridor, it is judged as normal following;

[0020] When an obstacle intrudes into the dynamic following corridor and there is still a passable lateral margin on one side of the dynamic following corridor, it is determined to decelerate and avoid it;

[0021] When the distance to the boundary of the restricted area is less than the parking distance threshold of the restricted area, it is judged as boundary parking;

[0022] When a target is briefly occluded and the duration of occlusion does not exceed a preset occlusion retention threshold, it is determined to be occlusion retention.

[0023] When the target loss time exceeds the preset target loss threshold, or the obstacle distance is less than the emergency braking distance threshold, the vehicle is determined to be safely stopped.

[0024] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, the step of calculating control commands based on the current control state and the dynamic following corridor includes:

[0025] In normal following mode, the linear velocity command is calculated based on the error between the longitudinal distance and the desired following distance, and the angular velocity command is calculated based on the relationship between the relative heading angle and the corrected steering dead zone threshold.

[0026] Based on the linear velocity and angular velocity commands, and combined with the wheel track and wheel radius of the cargo vehicle, the target rotational speeds of the left and right drive wheels are calculated.

[0027] In the boundary parking state and the safe parking state, a braking command is generated to control the cargo vehicle to stop;

[0028] The control commands include linear velocity commands, angular velocity commands, target rotational speeds of the left and right drive wheels, and braking commands.

[0029] As a preferred embodiment of the computer vision-based vehicle following control method of the present invention, wherein: the step of fine-tuning the current mode parameter set online based on operational feedback includes:

[0030] Collect data on the following error, number of emergency stops, and number of target loss during the operation of the cargo vehicle;

[0031] The current mode parameter set is fine-tuned online based on the following error, the number of emergency stops, and the number of target loss.

[0032] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein the computer program, when executed by the processor, implements any step of the computer vision-based vehicle following control method as described in the first aspect of the present invention.

[0033] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the computer vision-based vehicle following control method as described in the first aspect of the present invention.

[0034] The beneficial effects of this invention are as follows: By calculating the target's relative pose parameters and terrain state indicators based on scene perception results and driving state data, and by making real-time corrections to the current mode parameter set, a dynamic following corridor is generated. This unifies the target position, environmental constraints, and vehicle operating state into an executable control boundary, thereby providing a consistent decision-making basis for normal following, deceleration and avoidance, boundary parking, occlusion maintenance, or safe parking, and improving the stability and scene adaptability of the following process. By fine-tuning the current mode parameter set online based on operational feedback, the control parameters are continuously optimized with the operational results, thereby enhancing the matching of linear velocity commands, angular velocity commands, target rotation speeds of the left and right drive wheels, and braking commands, and improving control precision and continuous availability. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart of a computer vision-based vehicle following control method.

[0037] Figure 2 This is a flowchart for control status determination and control execution. Detailed Implementation

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0040] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0041] Reference Figure 1 and Figure 2 As one embodiment of the present invention, this embodiment provides a computer vision-based vehicle following control method, comprising the following steps:

[0042] S1. Receive scene mode selection instruction, determine the current working mode, and load the current mode parameter set corresponding to the current working mode; the current mode parameter set includes the reference following distance, steering dead zone threshold, speed response gain, allowable curvature limit, maximum speed limit, emergency braking distance threshold, no-entry zone parking distance threshold, and slope compensation coefficient.

[0043] Furthermore, upon receiving a scene mode selection command, the vehicle's human-machine interface first outputs the selectable scene information corresponding to camping mode, golf mode, and shopping mode. Then, it records the user's trigger, confirmation, and cancellation operations for camping mode, golf mode, or shopping mode, and determines the validity of the scene mode selection command based on the final retained confirmation operation. After determining the validity of the scene mode selection command, it retrieves the mode identifier corresponding to camping mode, golf mode, or shopping mode from the preset mode mapping relationship according to the scene category pointed to by the scene mode selection command, and determines the current working mode based on the retrieval results. After determining the current working mode, it continues to access the parameter storage area corresponding to the current working mode, sequentially reading the baseline following distance, steering dead zone threshold, speed response gain, maximum speed limit, allowable curvature limit, emergency braking distance threshold, no-entry zone parking distance threshold, and slope compensation coefficient, forming the current mode parameter set corresponding to the current working mode. The current working mode is written to the operating status record area, and the current mode parameter set corresponding to the current working mode is written to the operating parameter call area.

[0044] It should be noted that the specific steps for setting the mode mapping relationship are as follows: first, a unique mode identifier is set for each of the camping mode, golf mode, and shopping mode; then, a corresponding parameter storage area is established for each mode; then, a one-to-one correspondence is established between the scene mode selection command, the mode identifier, and the parameter storage area address to form a preset mode mapping relationship; after receiving the scene mode selection command, the corresponding mode identifier is first searched, and then the corresponding parameter storage area address is searched according to the mode identifier. The baseline following distance, steering dead zone threshold, maximum speed limit, emergency braking distance threshold, no-entry zone parking distance threshold, speed response gain, allowable curvature limit, and slope compensation coefficient are all pre-calibrated parameters; the pre-calibrated parameters are obtained by conducting comfort following tests, ineffective micro-sway filtering tests, stable operation upper limit tests, measured parking distance calibration, boundary safe parking tests, distance error response tests, minimum stable turning radius tests, and slope compensation tests in camping mode, golf mode, and shopping mode, respectively, and are written into the parameter storage area of ​​the corresponding mode.

[0045] S2. Based on the current mode parameter set, acquire the forward image sequence, wheel speed, motor current and braking status to obtain image data and driving status data.

[0046] Specifically, based on the current mode parameter set, the sampling frame rate, exposure time, and sampling period of the front camera are set according to the maximum speed limit, speed response gain, and slope compensation coefficient in the current mode parameter set. In each sampling period, image frames in the front field of view of the cargo vehicle are continuously acquired and cached in chronological order to form a front image sequence. While forming the front image sequence, the wheel speeds of the left and right drive wheels, the motor currents of the drive motors, and the braking states of the brakes are synchronously read according to the same time reference as the front image sequence. The wheel speeds, motor currents, and braking states are timestamped and periodically aligned. The front image sequence arranged in chronological order is output as image data, and the wheel speeds, motor currents, and braking states corresponding to each moment of the image data are output as drive state data, thus obtaining image data and drive state data.

[0047] S3. Perform target locking and scene parsing on image data and drive state data to obtain scene perception results; among which, scene perception results include target foot coordinates, target confidence, obstacle area, restricted area and congestion index.

[0048] Specifically, when performing target locking and scene parsing based on image data and driving state data, continuous image frames are extracted according to the time sequence in the image data, and the timing alignment with the continuous image frames is completed according to the wheel speed, motor current and braking state in the driving state data.

[0049] After completing temporal alignment, candidate regions of human targets are detected from consecutive image frames, and the displacement of the target is estimated based on the wheel speed of the vehicle. The displacement of the candidate target box in consecutive frames is compensated. When the compensated target displacement is consistent with the continuous tracking trajectory of the human target, it is determined to be a following target, and the target locking result is obtained.

[0050] Extract the center position of the lower edge of the target from the image region corresponding to the target lock, determine the coordinates of the target foot point, and calculate the target confidence based on the continuity of the target appearance and the temporal consistency in consecutive image frames;

[0051] Furthermore, in consecutive image frames, the target detection bounding box corresponding to the current frame and the target detection bounding box corresponding to the previous frame are extracted, and the target image region is cropped from each of the two target detection bounding boxes; then, the two target image regions are converted to the same color space and normalized color histograms are calculated. By taking the minimum value of each item and summing them, the continuity of the target appearance is obtained. , represented as:

[0052] ;

[0053] in, Indicates the target image region in the current frame is in the [number]th position. Normalized values ​​for each histogram interval. Indicates the target image region in the previous frame at the [number]th [frame]. Normalized values ​​for each histogram interval, target appearance continuity The value ranges from 0 to 1, with a larger value indicating a more stable target appearance in consecutive image frames. After obtaining the continuity of the target appearance, the target foot coordinates of the current frame, the previous frame, and the two frames prior are read. The displacement relationship between the target foot coordinates of the previous frame and the two frames prior is used to predict the target foot position of the current frame. Then, the deviation distance between the target foot coordinates of the current frame and the predicted position is calculated to obtain the temporal consistency, expressed as:

[0054] ;

[0055] in, =2 - , =2 - , representing the horizontal and vertical coordinates of the target foot in the current frame, predicted based on the target foot coordinates of the previous frame and the target foot coordinates of the previous two frames, respectively. , Indicates the coordinates of the target's foot point in the current frame. , This indicates the coordinates of the target's foot point in the previous frame. , This indicates the coordinates of the target's foot points in the first two frames. , This represents the width and height of the target detection bounding box in the previous frame, ensuring temporal consistency. The value ranges from 0 to 1. The larger the value, the more continuous the target movement. After obtaining the continuity of the target appearance and the consistency of the time sequence, the average of the two is taken as the target confidence level.

[0056] The system segments the ground region, obstacle region, and restricted area in consecutive image frames to determine obstacle regions and restricted areas. Specifically, it performs pixel-level classification frame by frame, dividing each pixel in a consecutive image frame into three categories: ground region, obstacle region, and restricted area. After pixel-level classification, it extracts connected components from the pixel set belonging to the obstacle region, removes discrete noise pixels with an area smaller than a preset pixel threshold, and performs boundary closure processing on the remaining connected components to obtain candidate obstacle regions. Then, using the target's foot coordinates as a reference to the ground below, it filters out suspended connected components located in the distant background that do not contact the ground. Obstacle candidate regions adjacent to or projected onto the ground area are identified as obstacle regions. After obtaining the obstacle regions, the same connected component extraction and boundary closure methods are used on the pixel set belonging to the restricted area to form restricted candidate regions. Based on the boundary relationship between the restricted candidate regions and the ground area, and the spatial position relationship with the direction of travel of the cargo vehicle, the restricted candidate regions located within the reach of the cargo vehicle are retained, and the retained results are identified as restricted areas. Among them, obstacle regions are used to characterize the physical areas that the cargo vehicle needs to bypass or avoid during its travel, and restricted areas are used to characterize restricted areas that can be identified but are not allowed to be entered.

[0057] For each consecutive image frame, obstacle regions are mapped frame by frame, and the change in the center position of the same obstacle region in adjacent image frames is calculated. When the change in the center position exceeds a preset displacement threshold, the obstacle region is determined to be a dynamic obstacle. The area ratio of dynamic obstacles within a preset forward region is statistically analyzed to obtain a congestion index. The target foot coordinates, target confidence, obstacle region, restricted area, and congestion index are correlated and output in a time sequence consistent with the image data and drive state data to obtain the scene perception result.

[0058] It should be noted that the displacement threshold is obtained by collecting static obstacle samples and dynamic obstacle samples in the target application scenario, statistically analyzing the distribution of the position change of the obstacle region center in adjacent image frames, and pre-calibrating based on the boundary value that distinguishes between static and dynamic obstacles.

[0059] S4. Based on the scene perception results and driving state data, calculate the target relative pose parameters and terrain state indicators, and make real-time corrections to the current mode parameter set to generate a dynamic following corridor.

[0060] S4.1 Based on the target foot coordinates and camera calibration parameters, the following target is mapped to the ground coordinate system in front of the cargo vehicle to obtain the target ground coordinates;

[0061] Specifically, the target foot coordinates are read from the scene perception results, and the camera intrinsic parameters, distortion parameters, camera extrinsic parameters, and camera mounting pose parameters relative to the ground coordinate system in front of the vehicle are read from the camera calibration parameters. The distortion parameters are used to correct the distortion of the image points corresponding to the target foot coordinates, and the distortion-corrected image points are converted into normalized imaging coordinates using the camera intrinsic parameters. Based on the normalized imaging coordinates, the spatial projection ray from the camera optical center to the normalized imaging coordinates is determined by combining the camera extrinsic parameters and the mounting pose parameters. The intersection point of the spatial projection ray and the ground plane corresponding to the ground coordinate system in front of the vehicle is then calculated. After the intersection point is calculated, the lateral and longitudinal position components of the intersection point in the ground coordinate system in front of the vehicle are combined and output as the target ground coordinates.

[0062] S4.2 Calculate the target relative pose parameters to the cargo vehicle based on the target ground coordinates.

[0063] Furthermore, the lateral and longitudinal position components are extracted from the target ground coordinates, and the position definition of the center reference point of the cargo vehicle and the angle definition of the longitudinal direction of the cargo vehicle are simultaneously read from the ground coordinate system in front of the cargo vehicle. After completing the correspondence between the position definitions of the target ground coordinates and the center reference point of the cargo vehicle, the lateral deviation of the target relative to the cargo vehicle is determined by the difference between the lateral position component in the target ground coordinates and the lateral reference value of the center reference point of the cargo vehicle, and the longitudinal distance of the target relative to the cargo vehicle is determined by the difference between the longitudinal position component in the target ground coordinates and the longitudinal reference value of the center reference point of the cargo vehicle. Based on the lateral deviation and the longitudinal distance of the target relative to the cargo vehicle, the direction of the target connection is constructed, and the angle between the direction of the target connection and the longitudinal direction of the cargo vehicle is determined as the relative heading angle of the target relative to the cargo vehicle. The direction of the target connection refers to the direction in the ground coordinate system in front of the cargo vehicle, which is formed by connecting the center reference point of the cargo vehicle and the target ground coordinates with the cargo vehicle center reference point as the starting point and the target ground coordinates as the ending point. Finally, the lateral deviation, longitudinal distance, and relative heading angle of the target relative to the cargo vehicle are output in a time sequence consistent with the target's ground coordinates to obtain the target's relative pose parameters.

[0064] S4.3 Calculate terrain status indicators based on changes in motor current and wheel speed.

[0065] Preferably, the motor current and wheel speed corresponding to consecutive sampling periods are extracted from the drive state data in chronological order, and the timing alignment of the motor current and wheel speed is completed according to the same time base; the changes in motor current and wheel speed, as well as the directions of motor current and wheel speed changes, are calculated between adjacent sampling periods respectively.

[0066] Furthermore, the correlation between the trend of motor current change and the trend of wheel speed change within a continuous sampling period is calculated. When the motor current continuously increases and the wheel speed continuously decreases, it is determined that the ground resistance in front of the cargo vehicle increases. When the motor current continuously decreases and the wheel speed continuously increases, it is determined that the ground resistance in front of the cargo vehicle decreases. When both the amplitude of the change in motor current and the amplitude of the change in wheel speed are within a stable range, it is determined that the ground resistance in front of the cargo vehicle remains stable. According to the preset terrain determination rules, the increase in ground resistance in front of the cargo vehicle, the decrease in ground resistance in front of the cargo vehicle, and the stability of ground resistance in front of the cargo vehicle are mapped to the corresponding terrain state variables. The terrain state variables obtained from the continuous sampling period are then subjected to temporal correlation and continuity verification to output terrain state indicators that can characterize uphill, downhill, or high-resistance road surfaces.

[0067] It should be noted that the process of setting the terrain determination rules includes: first, under standard load and fixed speed command conditions, the cargo vehicle is continuously driven on a flat, hard road surface, and the motor current and wheel speed are collected within multiple time windows to obtain the flat ground reference current range and flat ground reference wheel speed range; then, on known types of uphill roads, downhill roads, and flat high-resistance roads, the same load and speed command are maintained, and the motor current and wheel speed are collected within multiple time windows, and labeled according to the terrain category; subsequently, the rate of change of motor current and the rate of change of wheel speed are calculated for each time window, and the stability threshold is determined by the upper boundary of the flat ground sample distribution. The uphill judgment threshold is determined by the concentrated interval of increasing motor current change rate and decreasing wheel speed change rate in the uphill sample; the downhill judgment threshold is determined by the concentrated interval of decreasing motor current change rate and increasing wheel speed change rate in the downhill sample; and the high resistance judgment threshold is determined by the concentrated interval of continuously increasing motor current change rate and continuously decreasing wheel speed change rate in the high resistance road sample. Finally, it is stipulated that when multiple consecutive time windows meet the same threshold condition, the corresponding terrain category is output, and the stable threshold, uphill judgment threshold, downhill judgment threshold, high resistance judgment threshold, and number of consecutive judgment windows are written into the parameter table to form the preset terrain judgment rule.

[0068] S4.4 Based on the target relative pose parameters, terrain state indicators, and congestion indicators, the baseline following distance, steering dead zone threshold, maximum speed limit, and allowable curvature limit in the current model parameter set are corrected.

[0069] Furthermore, lateral deviation, longitudinal distance, and relative heading angle are read from the target's relative pose parameters, congestion index is read from the scene perception results, and the corresponding ground traffic status is read from the terrain status index. Based on the distance relationship between the longitudinal distance and the baseline following distance, and the traffic load relationship corresponding to the terrain status index, the baseline following distance is corrected to ensure consistency between the baseline following distance and the current target proximity and current ground traffic status. The turning dead zone threshold is corrected based on lateral deviation, relative heading angle, and congestion index to ensure consistency between the turning dead zone threshold and the current turning demand and the traffic density ahead. Based on the longitudinal distance and terrain shape... The maximum speed limit is corrected based on the state index and congestion index to ensure it is consistent with the current target following requirements, current ground traffic conditions, and the distribution of dynamic obstacles ahead. The permissible curvature limit is corrected based on the lateral deviation, relative heading angle, terrain state index, and congestion index to ensure it is consistent with the current turning space and current ground traffic conditions. After correcting the baseline following distance, turning dead zone threshold, maximum speed limit, and permissible curvature limit, the corrected baseline following distance, corrected turning dead zone threshold, corrected maximum speed limit, and corrected permissible curvature limit are written back to the current mode parameter set.

[0070] S4.5 Generate a dynamic following corridor based on the corrected parameters, including the desired following distance, allowable lateral deviation range, allowable speed range, allowable curvature range, and parking boundary position.

[0071] Specifically, the corrected baseline following distance, corrected steering dead zone threshold, corrected maximum speed limit, and corrected allowable curvature limit are sequentially read from the current mode parameter set. Simultaneously, the lateral deviation, longitudinal distance, and relative heading angle in the target's relative pose parameters, as well as the obstacle area and restricted area in the scene perception results, are read. The expected following distance is determined by the correspondence between the corrected baseline following distance and longitudinal distance; the allowable lateral deviation range is determined by the correspondence between the corrected steering dead zone threshold and lateral deviation; the allowable speed range is determined by the distribution relationship of the corrected maximum speed limit and the obstacle area in the ground coordinate system in front of the vehicle; and the allowable curvature range is determined by the correspondence between the corrected allowable curvature limit and the relative heading angle. The parking boundary position is determined based on the boundary position of the restricted area in the ground coordinate system in front of the vehicle and the parking distance threshold of the restricted area. The expected following distance, allowable lateral deviation range, allowable speed range, allowable curvature range, and parking boundary position are associated and combined according to a temporal sequence consistent with the target's relative pose parameters and scene perception results to obtain the dynamic following corridor.

[0072] S5. Determine the current control status based on the dynamic following corridor.

[0073] Specifically, it determines whether the target confidence level is greater than the preset confidence threshold and whether there are obstacles blocking the dynamic following corridor. When the obstacle area forms a lateral through blockage in the dynamic following corridor, or the remaining passable width is less than the sum of the body width and the safe clearance, it is determined to be an obstacle blockage. When the target confidence level is greater than the preset confidence threshold and there are no obstacles blocking the dynamic following corridor, it is determined to be normal following.

[0074] Determine whether the obstacle area intrudes into the dynamic following corridor and calculate the remaining passage width on both sides of the dynamic following corridor. When the obstacle intrudes into the dynamic following corridor and there is still a passable lateral margin on one side of the dynamic following corridor, it is determined to decelerate and avoid it.

[0075] Calculate the distance from the projection point of the current position of the cargo vehicle in the ground coordinate system in front of the cargo vehicle to the nearest boundary of the restricted area. This distance is used as the boundary distance of the restricted area. When the boundary distance of the restricted area is less than the parking distance threshold of the restricted area, it is determined to be boundary parking.

[0076] The duration of occlusion is calculated based on the duration of continuous interruption of the target locking result. When the target is briefly occluded and the duration of occlusion does not exceed the preset occlusion retention threshold, it is determined to be occlusion retention.

[0077] The target loss time is calculated based on the continuous failure duration of the target lock result, and the obstacle distance is calculated based on the spatial relationship between the obstacle area and the current position of the cargo vehicle. When the target loss time exceeds the preset target loss threshold, or the obstacle distance is less than the emergency braking distance threshold, it is determined to be a safe stop.

[0078] It should be noted that the confidence threshold was determined by first acquiring continuous image frames in camping, golf, and shopping modes, and then labeling the correctly following targets and non-target samples in the continuous image frames. Following the calculation methods based on the continuity of target appearance and temporal consistency, the target confidence for correctly following targets and the target confidence for non-target samples were calculated. Subsequently, the distribution intervals of target confidence for correctly following targets and non-target samples were statistically analyzed, and a boundary value that could distinguish between correctly following targets and non-target samples was selected. In camping, golf, and shopping modes, continuous image frames were acquired under conditions of short-term occlusion, continuous occlusion, and complete removal from the field of view, and the duration of continuous interruptions in target locking was recorded simultaneously. The distribution of continuous interruption durations where the target could recover lock after occlusion and the distribution of continuous interruption durations where the target could not recover lock after occlusion were then statistically analyzed. The upper boundary of the distribution of continuous interruption durations where lock could be recovered was determined as the occlusion retention threshold, and the lower boundary of the distribution of continuous interruption durations where lock could not be recovered was determined as the target loss threshold.

[0079] S6. Based on the current control state and the dynamic following corridor, calculate the control command, control the operation of the cargo vehicle, and fine-tune the current mode parameter set online according to the operation feedback.

[0080] S6.1 Under normal following conditions, calculate the linear velocity command based on the error between the longitudinal distance and the desired following distance, and calculate the angular velocity command based on the relationship between the relative heading angle and the corrected steering dead zone threshold.

[0081] Furthermore, under normal following conditions, the error between the longitudinal distance and the expected following distance is obtained by subtracting the expected following distance from the longitudinal distance. Based on the magnitude and direction of the error between the longitudinal distance and the expected following distance, as well as the allowable speed range, the speed adjustment trend of the cargo vehicle along the forward direction of the cargo vehicle is determined. The speed adjustment trend is then converted into amplitude and bounded by combining the speed response gain, and a linear speed command corresponding to the error between the longitudinal distance and the expected following distance is output. The relative heading angle is compared with the corrected steering dead zone threshold. When the relative heading angle exceeds the corrected steering dead zone threshold, the steering correction direction and steering correction magnitude are determined. When the relative heading angle does not exceed the corrected steering dead zone threshold, the steering correction magnitude is kept at zero. Finally, an angular velocity command is generated according to the degree of deviation of the relative heading angle from the corrected steering dead zone threshold.

[0082] S6.2. Based on the linear velocity command and angular velocity command, and combined with the wheel track and wheel radius of the cargo vehicle, the target rotation speed of the left and right drive wheels is calculated.

[0083] Furthermore, the steering-related linear velocity component is obtained by multiplying the angular velocity command by half the wheelbase of the load vehicle. This component is then superimposed and subtracted from the steering-related linear velocity component using the linear velocity command to obtain the left-side and right-side drive linear velocities. The target speed of the left drive wheel is obtained by dividing the left-side drive linear velocity by the wheel radius, and the target speed of the right drive wheel is obtained by dividing the right-side drive linear velocity by the wheel radius. The target speeds of the left and right drive wheels are then checked for consistency with the upper speed limits corresponding to the allowable speed range. The target speeds of the left and right drive wheels are output when they meet the speed constraints. If either the left or right drive wheel target speed exceeds the speed constraints, the linear velocity and angular velocity commands are scaled proportionally and recalculated.

[0084] S6.3. In the boundary parking state and the safe parking state, generate braking commands to control the cargo vehicle to stop; wherein, the control commands include linear velocity commands, angular velocity commands, target speeds of the left and right drive wheels and braking commands.

[0085] Specifically, in the boundary parking state and the safe parking state, the parking demand level of the cargo vehicle is determined based on the remaining distance between the parking boundary position corresponding to the boundary parking state and the current position of the cargo vehicle, or based on the remaining distance between the obstacle distance and the emergency braking distance threshold corresponding to the safe parking state.

[0086] Furthermore, the remaining distance is determined based on the current control state. When the current control state is a boundary parking state, the remaining distance between the parking boundary position and the current position of the cargo vehicle is used as the judgment distance corresponding to the boundary parking state. When the current control state is a safe parking state, the difference between the obstacle distance and the emergency braking distance threshold is used as the judgment distance corresponding to the safe parking state. After obtaining the corresponding judgment distance, the judgment distance is compared with the preset first-level parking demand threshold and second-level parking demand threshold respectively. When the judgment distance is greater than the first-level parking demand threshold, the cargo vehicle's parking demand level is determined to be low parking demand level; when the judgment distance is less than or equal to the first-level parking demand threshold, the vehicle's parking demand level is determined to be low parking demand level. When the threshold is greater than the secondary parking demand threshold, the parking demand level of the cargo vehicle is determined to be medium parking demand level. When the determination distance is less than or equal to the secondary parking demand threshold, the parking demand level of the cargo vehicle is determined to be high parking demand level. Among them, high parking demand level corresponds to prioritizing braking and quickly reducing the target speed of the left and right drive wheels to zero, medium parking demand level corresponds to decelerating in advance and gradually applying braking, and low parking demand level corresponds to first reducing the target speed of the left and right drive wheels before applying braking. It should also be noted that the primary parking demand threshold is greater than the secondary parking demand threshold, and the primary and secondary parking demand thresholds can be pre-calibrated through boundary parking tests and emergency braking tests.

[0087] The target speed of the left and right drive wheels is gradually reduced to zero according to the control cycle, and the braking application sequence and braking holding time corresponding to the parking demand level of the cargo vehicle are generated simultaneously to obtain the braking command. The braking command is checked in a closed loop with the wheel speed and braking status. When the wheel speed drops to the preset parking judgment range and the braking status reaches the effective braking state, the braking command output is maintained. When the wheel speed does not drop to the preset parking judgment range, the braking command output is maintained until the wheel speed is zero.

[0088] S6.4. Collect the following error, number of emergency stops, and number of target loss during the operation of the cargo vehicle;

[0089] Specifically, the longitudinal distance, desired following distance, current control status, target lock result, wheel speed, and braking command are synchronously read according to the continuous control cycle corresponding to the operation of the cargo vehicle.

[0090] The following error is calculated cycle by cycle based on the difference between the longitudinal distance and the expected following distance, and the following error corresponding to each control cycle is continuously recorded in the time sequence of the vehicle's operation. After completing the following error recording, it is checked whether the current control state has entered a safe stop and whether the braking command has been triggered. When the current control state changes from normal following, deceleration avoidance, boundary parking or obstruction holding to a safe stop and the braking command is effectively output, one emergency stop count is accumulated. Based on the comparison between the continuous failure duration of the target lock result and the preset target loss threshold, one target loss count is accumulated when the continuous failure duration of the target lock result exceeds the preset target loss threshold. After completing the cycle-by-cycle recording and accumulation of following error, emergency stop count, and target loss count, the following error, emergency stop count, and target loss count are output in a time sequence consistent with the vehicle's operation.

[0091] S6.5. Fine-tune the current mode parameter set online based on the following error, number of emergency stops, and number of target loss.

[0092] Specifically, the adjustment directions for the baseline following distance and speed response gain are determined based on the cumulative deviation direction and trend of the following error; the adjustment directions for the maximum speed limit and emergency braking distance threshold are determined based on the number of emergency stops; and the adjustment directions for the steering dead zone threshold and slope compensation coefficient are determined based on the number of target loss. After determining each adjustment direction, the baseline following distance, steering dead zone threshold, speed response gain, maximum speed limit, emergency braking distance threshold, no-entry zone stopping distance threshold, and slope compensation coefficient are updated item by item according to the preset fine-tuning step size and parameter boundary range. If the indicators do not improve after the parameter update, the parameters are reverted to the previous set. The updated current mode parameter set is written back to the running parameter call area and maintains its correspondence with the current working mode.

[0093] This embodiment also provides a computer device applicable to the case of a computer vision-based vehicle following control method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the computer vision-based vehicle following control method proposed in the above embodiment.

[0094] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0095] This embodiment also provides a storage medium on which a computer program is stored. When executed by a processor, the program implements the computer vision-based vehicle following control method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0096] In summary, this invention calculates the target's relative pose parameters and terrain state indicators based on scene perception results and driving state data, and performs real-time correction on the current mode parameter set to generate a dynamic following corridor. This unifies the target position, environmental constraints, and vehicle operating state into an executable control boundary, thereby providing a consistent decision-making basis for normal following, deceleration and avoidance, boundary parking, occlusion maintenance, or safe parking, improving the stability and scene adaptability of the following process. By fine-tuning the current mode parameter set online based on operational feedback, the control parameters are continuously optimized with the operational results, thereby enhancing the matching of linear velocity commands, angular velocity commands, target rotation speeds of the left and right drive wheels, and braking commands, and improving control precision and continuous availability.

[0097] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A computer vision-based vehicle following control method, characterized in that, include: Receive scene mode selection instructions, determine the current working mode, and load the current mode parameter set corresponding to the current working mode; Based on the current mode parameter set, a sequence of images ahead, as well as wheel speed, motor current, and braking status, are acquired to obtain image data and driving status data. Target locking and scene parsing are performed on the image data and driving state data to obtain scene perception results; Based on the scene perception results and driving state data, the target relative pose parameters and terrain state indicators are calculated, and the current mode parameter set is corrected in real time to generate a dynamic following corridor. The current control status is determined based on the dynamically following corridor. Based on the current control state and the dynamic following corridor calculation control command, the vehicle is controlled to operate, and the current mode parameter set is fine-tuned online according to the operation feedback.

2. The computer vision-based vehicle following control method as described in claim 1, characterized in that, The current mode parameter set includes the baseline following distance, steering dead zone threshold, speed response gain, maximum speed limit, allowable curvature limit, emergency braking distance threshold, no-entry zone parking distance threshold, and slope compensation coefficient.

3. The computer vision-based vehicle following control method as described in claim 1, characterized in that, The scene perception results include target foot coordinates, target confidence, obstacle area, restricted area, and congestion index; The congestion index is the ratio of the projected area of ​​a dynamic obstacle in the preset forward region to the total area of ​​the preset forward region.

4. The computer vision-based vehicle following control method as described in claim 3, characterized in that, The step of calculating the target's relative pose parameters and terrain state indicators based on the scene perception results and driving state data includes: Based on the target's foot coordinates and camera calibration parameters, the target is mapped to the ground coordinate system in front of the cargo vehicle to obtain the target's ground coordinates; Calculate the target relative pose parameters to the cargo vehicle based on the target ground coordinates; Terrain condition indicators are calculated based on changes in motor current and wheel speed.

5. The computer vision-based vehicle following control method as described in claim 4, characterized in that, The step of real-time correction of the current mode parameter set to generate a dynamic following corridor includes: Based on the target relative pose parameters, terrain state indicators, and congestion indicators, the baseline following distance, steering dead zone threshold, maximum speed limit, and allowable curvature limit in the current mode parameter set are corrected. A dynamic following corridor is generated based on the modified parameters, including the expected following distance, allowable lateral deviation range, allowable speed range, allowable curvature range, and parking boundary position.

6. The computer vision-based vehicle following control method as described in claim 5, characterized in that, The determination of the current control state based on the dynamic following corridor includes: When the target confidence level is greater than the preset confidence threshold and there are no obstacles blocking the dynamic following corridor, it is judged as normal following; When an obstacle intrudes into the dynamic following corridor and there is still a passable lateral margin on one side of the dynamic following corridor, it is determined to decelerate and avoid it; When the distance to the boundary of the restricted area is less than the parking distance threshold of the restricted area, it is judged as boundary parking; When a target is briefly occluded and the duration of occlusion does not exceed a preset occlusion retention threshold, it is determined to be occlusion retention. When the target loss time exceeds the preset target loss threshold, or the obstacle distance is less than the emergency braking distance threshold, the vehicle is determined to be safely stopped.

7. The computer vision-based vehicle following control method as described in claim 6, characterized in that, The control commands calculated based on the current control state and the dynamic following corridor include: In normal following mode, the linear velocity command is calculated based on the error between the longitudinal distance and the desired following distance, and the angular velocity command is calculated based on the relationship between the relative heading angle and the corrected steering dead zone threshold. Based on the linear velocity and angular velocity commands, and combined with the wheel track and wheel radius of the cargo vehicle, the target rotational speeds of the left and right drive wheels are calculated. In the boundary parking state and the safe parking state, a braking command is generated to control the cargo vehicle to stop; The control commands include linear velocity commands, angular velocity commands, target rotational speeds of the left and right drive wheels, and braking commands.

8. The computer vision-based vehicle following control method as described in claim 7, characterized in that, The step of fine-tuning the current mode parameter set online based on operational feedback includes: Collect data on the following error, number of emergency stops, and number of target loss during the operation of the cargo vehicle; The current mode parameter set is fine-tuned online based on the following error, the number of emergency stops, and the number of target loss.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the computer vision-based vehicle following control method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the computer vision-based vehicle following control method according to any one of claims 1 to 8.

Images