Following control method and device of mobile equipment, electronic equipment and storage medium

Abstract

The invention provides a following control method and device of mobile equipment, electronic equipment and a storage medium, target stability is maintained by performing consistency judgment on candidate targets in sensing data, and target confirmation and locking are realized when an interaction trigger instruction corresponding to the candidate targets is detected; in a locking state, target area information is continuously output, depth / distance information is fused to calculate a relative pose, then the relative pose is converted into a distance error and an angle error to generate a chassis control instruction, and closed-loop adjustment of the following process is achieved. Therefore, the false following probability in a multi-person scene can be reduced, the stability and controllability of target locking and following are improved, and a smoother and more accurate following effect is kept when the target moves and the environment changes.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision control technology, and more specifically, to a method, apparatus, electronic device, and storage medium for tracking control of a mobile device. Background Technology

[0002] With the increasing application of service robots, companion robots, and mobile smart terminals in scenarios such as homes, commercial displays, and guided tours, the ability of mobile devices to automatically follow people based on their location has become a crucial capability for improving interactive experiences and service efficiency. Existing following solutions typically rely on sensors such as cameras, LiDAR, or ultrasound to acquire environmental information and use target detection / tracking algorithms to estimate the relative position of people, thereby controlling the chassis movement to achieve following.

[0003] However, existing technologies still have the following shortcomings in practical applications: On the one hand, in situations such as crowd interference, occlusion, changes in lighting, or rapid target movement, mobile devices are prone to drifting, mis-following, or losing the target; especially when multiple candidates are present within the sensing range, the system often struggles to maintain a continuous following relationship with the same person. On the other hand, many solutions lack natural, low-cost, and interpretable human-computer interaction triggering methods. Controls such as target confirmation, initiation, and stopping often rely on preset fixed procedures or complex peripheral operations, resulting in unintuitive target locking, a high probability of false triggering, and difficulty in meeting the needs of rapid switching, real-time control, and secure control in real-world scenarios. Summary of the Invention

[0004] This disclosure provides at least one method, apparatus, electronic device, and storage medium for following control of a mobile device. It maintains target stability by performing consistency determination on candidate targets in perceived data and confirms and locks onto the target upon detecting an interactive trigger command corresponding to a candidate target. In the locked state, it continuously outputs target area information and fuses depth / distance information to calculate relative pose, then converts the relative pose into distance and angle errors to generate chassis control commands, achieving closed-loop adjustment of the following process. This reduces the probability of misfollowing in multi-person scenarios, improves the stability and controllability of target locking and following, and maintains a smoother and more accurate following effect when the target moves and the environment changes.

[0005] This disclosure provides a follow control method for a mobile device, including: Acquire personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and determine the consistency of the candidate targets; When an interactive trigger command corresponding to the candidate target is detected, the candidate target is determined to be a follow target and enters a locked state; In the locked state, continuous tracking is performed on the target being followed to obtain the target area information of the target being followed, and the relative pose information of the target being followed relative to the mobile device is calculated based on the target area information; Based on the relative pose information, a chassis motion control command is generated and sent to the mobile chassis to drive the mobile chassis to follow the target.

[0006] In one optional implementation, personnel-related perception data within the sensing range of the mobile device is acquired, personnel-related perception data is used to detect candidate personnel targets, and consistency determination is performed on the candidate targets. Specifically, this includes: Image data is collected using a binocular camera as relevant perception data of the person. Depth and distance information are calculated based on binocular parallax and stored in association with the image data; Perform facial recognition on the aforementioned personnel-related perception data to output the candidate targets; Based on the feature vectors, appearance consistency, and temporal continuity of the candidate targets, the consistency determination is performed to maintain target consistency.

[0007] In one optional implementation, when an interaction trigger command corresponding to the candidate target is detected, the candidate target is determined to be a follower target and enters a locked state, specifically including: Receive at least one of gesture commands, voice commands, touch commands, and selection commands issued by mobile applications as the interaction trigger command; When the interactive trigger command is successfully associated with the candidate target, the candidate target is identified as the following target and enters a locked state; The system calls a pre-built gesture model library to recognize the gestures of candidate targets and outputs the gesture category. When the gesture category belongs to a preset set, the interaction trigger instruction is generated, wherein the preset set includes at least one or more of the following: palm, fist, scissor, and Okey gesture.

[0008] In one optional implementation, in the locked state, continuous tracking is performed on the target to obtain target area information of the target, and the relative pose information of the target relative to the mobile device is calculated based on the target area information, specifically including: In the locked state, a target bounding box or target outline is generated in real time to represent the target being followed. The target bounding box or target outline is updated as the target moves to form the target region information; A filter-based target tracking algorithm is used to continuously track the target and output the target region information. Calculate the pixel coordinates and their changes based on the target area information; The relative pose information is calculated by fusing depth and distance information with the pixel coordinates, wherein the relative pose information includes at least one of the target relative distance and the target relative azimuth angle.

[0009] In one optional implementation, a chassis motion control command is generated based on the relative pose information and sent to the mobile chassis to drive the mobile chassis to follow the target, specifically including: The relative pose information is converted into distance error and angle error; The chassis motion control command is generated based on the distance error and the angle error; The chassis motion control command is issued to control at least one of the following: moving speed, acceleration, steering angular velocity, moving distance and steering angle, so that the error converges to within the threshold range.

[0010] In one optional implementation, the method further includes: Detect start and stop follow interaction commands while the device is locked. When a start follow interaction command is detected, a start intelligent follow command is sent to the mobile chassis; When a stop follow interaction command is detected, a command to disable smart follow is sent to the mobile chassis and the follow function is exited.

[0011] In one optional implementation, the hand gesture recognition result is mapped to a command to initiate a follow-up interaction; The results of fist-clenching gesture recognition are mapped to a stop-follow interaction command.

[0012] This disclosure also provides a follow control device for a mobile device, comprising: The target determination module is used to acquire personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and make a consistency judgment on the candidate targets; The lock trigger module is used to determine that the candidate target is a following target and enter the lock state when an interactive trigger command corresponding to the candidate target is detected; The pose determination module is used to continuously track the target in the locked state, obtain the target area information of the target, and calculate the relative pose information of the target relative to the mobile device based on the target area information. The motion control module is used to generate chassis motion control commands based on the relative pose information and send them to the mobile chassis to drive the mobile chassis to follow the target.

[0013] This disclosure also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the above-described mobile device follow control method, or any possible implementation of the above-described mobile device follow control method, are performed.

[0014] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the aforementioned mobile device follow control method, or any possible implementation of the aforementioned mobile device follow control method.

[0015] This disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the following control method for the mobile device described above, or the steps in any possible implementation of the following control method for the mobile device described above.

[0016] This disclosure provides a following control method, apparatus, electronic device, and storage medium for a mobile device. It maintains target stability by performing consistency judgment on candidate targets in perceived data and confirms and locks onto the target upon detecting an interactive trigger command corresponding to a candidate target. In the locked state, it continuously outputs target area information and fuses depth / distance information to calculate relative pose. The relative pose is then converted into distance and angle errors to generate chassis control commands, achieving closed-loop adjustment of the following process. This reduces the probability of false following in multi-person scenarios, improves the stability and controllability of target locking and following, and maintains a smoother and more accurate following effect when the target moves and the environment changes.

[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0019] Figure 1A flowchart of a follow control method for a mobile device provided in an embodiment of this disclosure is shown; Figure 2 A flowchart illustrating another follow control method for a mobile device provided in an embodiment of this disclosure is shown; Figure 3 A schematic diagram of a follow control device for a mobile device provided in an embodiment of this disclosure is shown; Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0021] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0022] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0023] Research has revealed the following shortcomings of existing technologies in practical applications: Firstly, mobile devices are prone to drifting, mis-following, or losing track of targets under conditions of crowd interference, occlusion, changes in lighting, or rapid target movement. This is especially true when multiple potential individuals are within the sensing range, making it difficult for the system to maintain a consistent following relationship with the same person. Secondly, many solutions lack natural, low-cost, and interpretable human-computer interaction triggering methods. Controls such as target confirmation, activation, and deactivation often rely on pre-set fixed procedures or complex peripheral operations, resulting in unintuitive target locking, a high probability of false triggering, and an inability to meet the demands of rapid switching, real-time control, and secure control in real-world scenarios.

[0024] Based on the above research, this disclosure provides a following control method, device, electronic device, and storage medium for mobile devices. It maintains target stability by performing consistency judgment on candidate targets in the perceived data, and achieves target confirmation and locking when an interactive trigger command corresponding to a candidate target is detected. In the locked state, it continuously outputs target area information and fuses depth / distance information to calculate relative pose, then converts the relative pose into distance and angle errors to generate chassis control commands, achieving closed-loop adjustment of the following process. This reduces the probability of misfollowing in multi-person scenarios, improves the stability and controllability of target locking and following, and maintains a smoother and more accurate following effect when the target moves and the environment changes.

[0025] To facilitate understanding of this embodiment, a detailed description of a mobile device follow control method disclosed in this disclosure is provided first. The execution entity of the mobile device follow control method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the mobile device follow control method can be implemented by the processor calling computer-readable instructions stored in memory.

[0026] See Figure 1 The diagram shows a flowchart of a mobile device follow control method provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein: S101. Obtain personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and make a consistency determination on the candidate targets.

[0027] In specific implementation, the mobile device includes a sensing module and a processor. The sensing module is used to acquire personnel-related sensing data within the sensing range of the mobile device, and the processor is used to perform personnel candidate target detection and consistency determination on the personnel-related sensing data, so as to prepare the basic data for subsequent target selection and follow-up control.

[0028] Specifically, the personnel-related perception data may include at least one of image data, depth data, and point cloud data. In a preferred embodiment, the perception module includes a binocular camera, which acquires a left-eye image and a right-eye image in each sampling period. The processor performs time synchronization and calibration alignment on the left-eye and right-eye images to obtain binocular image pairs corresponding to the same moment. Based on the binocular image pairs, the processor calculates a binocular disparity map and further obtains depth and distance information corresponding to pixel positions.

[0029] Here, to facilitate subsequent fusion operations, the depth information and distance information are stored in a frame-level association manner with the image data to form a unified personnel-related perception data structure, which includes fields such as timestamp, camera intrinsic and extrinsic parameters, image frames, disparity / depth maps, and distance measurement results.

[0030] Furthermore, after obtaining personnel-related perception data, the processor performs personnel detection or facial recognition processing on the image data to output one or more candidate personnel targets in the current image frame. The candidate personnel targets can be represented using target region information, such as target bounding boxes, target contours, or sets of key points to represent the position and shape of the personnel in the image, and a candidate target identifier is generated for each candidate target.

[0031] Here, to improve the distinguishability of candidate targets, the processor can extract feature vectors from the candidate targets. The feature vectors are used to characterize the appearance information and / or identity information of the candidate targets. The feature vectors can be calculated from the image region of the candidate targets by a preset feature extraction network.

[0032] After outputting candidate targets, the processor performs a consistency determination on the candidate targets to maintain target consistency and reduce the risk of false association caused by multi-target interference. The consistency determination may include: matching the candidate targets in the current frame with the target trajectories that already exist in the previous frame, and determining the matching relationship based on feature vector similarity, appearance consistency index, and temporal continuity constraints.

[0033] The appearance consistency index may include at least one of color histogram difference, texture similarity, and key point structure consistency; the temporal continuity constraint may include at least one of the following: the position change amplitude of the candidate target in adjacent frames, the scale change amplitude, and the continuity of motion direction.

[0034] It should be noted that if the matching degree between the current frame candidate target and a certain historical target trajectory meets a preset threshold, the candidate target is merged into the historical target trajectory and the candidate target identifier corresponding to the trajectory is updated; if the matching degree does not meet the preset threshold, a new target trajectory is created for the candidate target and a new candidate target identifier is assigned.

[0035] In some implementations, to further enhance the robustness of consistency determination in the case of occlusion or short-term loss, the processor may also introduce depth and distance information to participate in the consistency determination. For example, the distance change of the candidate target and the distance change output by the predicted motion model are constrained to suppress false matching caused by image appearance similarity alone.

[0036] In this way, even when there are multiple people targets within the perception range of the mobile device or when there are changes in lighting or occlusion interference, the device can maintain continuous identification and stable output of the same person target, providing a reliable foundation for target association, locking and following control of subsequent interactive trigger commands.

[0037] S102. When an interactive trigger command corresponding to the candidate target is detected, the candidate target is determined to be a follow target and enters a locked state.

[0038] In practice, after detecting and determining the consistency of candidate targets, the mobile device enters the target selection stage. This allows the user to explicitly specify the target to be followed in a natural interactive manner when there are multiple candidate targets within the perception range, thereby reducing the risk of misfollowing and improving the controllability of the following behavior.

[0039] Specifically, the interactive trigger command is used to represent the selection intention of the follow target, and its source may include at least one of gesture commands, voice commands, touch / button commands, and selection commands issued by mobile applications. In each control cycle, the mobile device's processor detects the interactive information corresponding to the candidate personnel target, and when it detects an interactive trigger command that meets preset trigger conditions, it establishes an association between the interactive trigger command and the corresponding candidate target, thereby determining the follow target to be locked.

[0040] In a preferred implementation, the interaction trigger command is a gesture command. The processor calls a pre-built gesture model library to analyze the hand region or human key point region of the candidate target and outputs the gesture category; the gesture model library may include at least one or more of the following: palm, fist, scissor, and Okey gesture. To avoid false triggers, the processor may set a gesture verification mechanism: for example, requiring the same gesture to remain stable within N consecutive frames, the gesture confidence to be higher than a threshold, and the gesture occurrence location to fall within the bounding box of the candidate target. Only when the verification passes is a valid interaction trigger command generated.

[0041] Here, when associating interactive trigger commands with candidate targets, the processor can adopt the following association strategy: when the interactive trigger command comes from a visual gesture, the target area where the trigger gesture is located is the primary association basis, that is, the command is preferentially associated with the candidate target containing the gesture area; when the interactive trigger command comes from touch / button or mobile application selection, the candidate target identifier selected by the user or its selected area in the interface is the association basis; when the interactive trigger command comes from a voice command, the voice wake-up direction, the nearest candidate target, and the historical consistency score of the candidate target can be combined for comprehensive association.

[0042] It should be noted that if multiple candidate targets meet the association conditions within the same period, the candidate target with the highest association confidence will be selected as the target to be locked, or the conflict handling process will be triggered (e.g., prompting the user to re-trigger or select).

[0043] Furthermore, when the association between the instruction and the candidate target is completed and the triggering condition is met, the processor determines the candidate target as the follower target and enters a locked state. After entering the locked state, the processor assigns a unique follower target identifier to the follower target and writes this identifier into the follower state machine to drive the subsequent continuous tracking and pose calculation process. Simultaneously, the processor can freeze or reduce resource usage on non-follower targets, that is, in subsequent cycles, priority is given to performing tracking updates on the follower target, while other candidate targets are only detected at low frequency or no longer output, in order to reduce interference caused by target switching.

[0044] Here, in the locked state, the mobile device can further generate locking prompts to provide interpretable feedback, such as drawing the bounding box / outline corresponding to the target on the display screen and attaching a locking icon, or providing voice / light effects to indicate that the target selection has been completed. Through the above-mentioned interactive triggering and locking mechanism, explicit selection and stable locking of the target can be achieved in multi-person scenarios, reducing the probability of misselection and misfollowing, and providing stable target input for subsequent continuous tracking, relative pose estimation, and chassis closed-loop following.

[0045] In some implementations, the locking state is managed by a follower state machine, which includes at least an unlocked state, a pending confirmation state, a locked state, and a follower state, and may optionally include a paused state and a release state.

[0046] Initially, the mobile device is in an unlocked state, where candidate target detection and consistency determination are performed, and a candidate target set is output. When an interaction trigger command is detected and it is initially successfully associated with a candidate target, the state machine switches from the unlocked state to the pending confirmation state. In the pending confirmation state, the interaction trigger command is verified and its consistency is checked. The verification and consistency check includes at least: the gesture / voice confidence level reaches a threshold, the trigger duration meets the requirements, the trigger area is located within the candidate target area, and the consistency score of the candidate target in multiple consecutive frames is not lower than a threshold. When the verification passes in the pending confirmation state, the state machine switches to the locked state and assigns a follow target identifier to the locked target to fix subsequent tracking input. When the verification fails or times out in the pending confirmation state, the state machine returns to the unlocked state and continues candidate target detection.

[0047] In the locked state, the mobile device prioritizes continuous tracking of the target and keeps the target identifier unchanged. When a follow interaction command is detected, the state machine switches from the locked state to the follow state and initiates the chassis motion control closed loop. When a stop follow interaction command is detected, the state machine switches back from the follow state to the locked state or the paused state. In this state, chassis motion control commands are stopped, but tracking and updates of the target are still maintained to facilitate rapid resumption of follow. When a follow interaction command is detected, the target is lost for more than a preset time, or a safety event occurs (such as obstacle risk or abnormal speed), the state machine switches to the release state. In the release state, the target identifier is cleared and follow control is stopped. If necessary, a fallback strategy is triggered (such as returning to a preset endpoint or entering a safe parking mode).

[0048] In this way, by setting the hierarchical states and switching conditions of the state machine, the orderly management of target selection, locking, following start / stop and release can be achieved, reducing the probability of false triggering and false switching, and improving the stability and safety controllability of the following process.

[0049] S103. In the locked state, continuous tracking is performed on the following target to obtain the target area information of the following target, and the relative pose information of the following target relative to the mobile device is calculated based on the target area information.

[0050] In specific implementation, when the mobile device enters the locked state, the processor uses the following target identifier as the only tracking object and performs continuous tracking of the following target within a continuous control cycle to obtain stable target area information. Furthermore, based on the target area information, the processor calculates the relative pose information of the following target relative to the mobile device, thereby providing a closed-loop pose input for subsequent chassis motion control.

[0051] For details, please refer to Figure 2The diagram shows a flowchart of another mobile device follow control method provided in this disclosure embodiment. The method includes steps S1031 to S1035, wherein: S1031. In the locked state, generate a target bounding box or target outline to represent the target being followed.

[0052] S1032. Update the target bounding box or target outline as the target moves to form the target region information.

[0053] S1033. A filter-type target tracking algorithm is used to continuously track the target and output the target area information.

[0054] S1034. Calculate the pixel coordinates and their changes based on the target area information.

[0055] S1035. Fuse depth and distance information and the pixel coordinates to calculate the relative pose information, wherein the relative pose information includes at least one of the target relative distance and the target relative azimuth angle.

[0056] Specifically, the continuous tracking can employ a target tracking algorithm to estimate the position and scale changes of the target in consecutive frames. In the locked state, the processor uses the target region at the moment of locking as the initial template or initial tracking window, and updates the target region information of the target in each subsequent frame. The target region information can be represented in at least one of the following forms: target bounding box, target contour, set of human keypoints, or target segmentation mask; wherein, the target bounding box may include parameters such as center point coordinates, width, and height, and the target contour or segmentation mask can be used to improve the accuracy of target region description in cases of complex pose changes or local occlusion.

[0057] As a preferred embodiment, the target tracking algorithm can be a filtering or correlation filtering tracking algorithm, such as the KCF algorithm based on correlation filtering, to achieve high real-time performance while keeping the computational complexity under control.

[0058] Here, to improve tracking robustness, the processor can introduce a confidence assessment and re-acquisition mechanism during continuous tracking: when the tracking confidence is higher than the threshold, the target region information of the current frame is directly output and the tracking model is updated; when the tracking confidence is lower than the threshold or the target region undergoes abnormal changes, local re-detection is triggered or re-association is performed in conjunction with consistency judgment to avoid target drift caused by short-term occlusion, sudden changes in illumination or rapid movement.

[0059] The processor can also perform time-series filtering on the output target area information, such as moving average or Kalman filtering on the target center point position and target scale parameters, to suppress jitter and improve the smoothness of pose output.

[0060] Furthermore, after obtaining the target area information, the processor calculates the relative pose information of the target relative to the mobile device based on the target area information. The relative pose information may include at least one of the target relative distance and the target relative azimuth angle, and may optionally include auxiliary quantities for control such as the target relative velocity, lateral offset, and longitudinal offset.

[0061] The specific calculation method can be implemented as follows: The processor determines the pixel coordinates (u, v) of the target in the image coordinate system based on the target area information. The pixel coordinates can be the center point of the target bounding box, the center of the human key point, or the centroid of the target contour. If depth / distance information exists, the processor reads the depth or distance value in the pixel coordinates (u, v) or its neighborhood to obtain the relative distance between the target and the camera. Further, the processor combines the camera intrinsic parameters to convert the pixel coordinates (u, v) into a spatial direction vector in the camera coordinate system, and calculates the target's relative azimuth angle from this direction vector. The target's relative azimuth angle can represent the deflection angle of the target relative to the moving direction of the mobile device, which is used for subsequent steering control.

[0062] In some implementations, if the personnel-related perception data is obtained from a binocular camera, the processor can obtain a depth map through binocular parallax and perform statistical fusion of the depth within the target area. For example, it can use the median depth within the area, a weighted average, or the mean after removing outliers to reduce the impact of single-point depth noise on distance estimation. If the target area information uses a set of human key points, it is preferable to select key points on the torso or chest and abdomen for depth sampling to reduce distance jitter caused by posture changes such as arm swings.

[0063] In this way, through the above-mentioned continuous tracking and relative pose calculation process, the mobile device can continuously and stably output the target area information and relative pose information corresponding to the target being followed in the locked state. This allows the subsequent chassis control to be adjusted in real time based on the closed-loop error of distance and orientation, thereby improving the stability, real-time performance and anti-interference capability of the following process.

[0064] In some implementations, the continuous tracking process is further configured with a target loss determination and occlusion recovery mechanism to improve robustness and security in the locked state. Specifically, the processor calculates the tracking confidence score while outputting target region information in each frame. The tracking confidence score can be determined by at least one of the following: correlation response peak, target appearance matching score, key point visibility ratio, or segmentation mask stability.

[0065] Here, when the tracking confidence level is lower than a first threshold, the target being followed is determined to be in a suspected loss state, and a loss counter is activated. In the suspected loss state, the processor maintains the most recent valid relative pose information as a short-term prediction input, and simultaneously performs local re-detection within a preset search window centered on the most recent target region to attempt to recover the target region information. The local re-detection can employ face detection / face recognition or feature vector-based similarity matching, and consistency determination constrains the candidate re-detection results to maintain consistency with the historical target being followed.

[0066] When a redetection is successful within a preset number of consecutive frames (e.g., the loss counter does not exceed the second threshold), the processor uses the target area information obtained from the redetection as the tracking initialization area and clears the loss counter, thereby achieving rapid recovery after occlusion. When the loss counter exceeds the second threshold or a target area that meets the consistency judgment cannot be detected for a continuous period of time, it is determined that the following target is lost, and the loss processing procedure is triggered.

[0067] Optionally, the loss handling process may include: switching the state machine from the follow state to the lock state or the unlock state; stopping or restricting the issuance of chassis motion control commands to enter a safe parking or low-speed tracking mode; and outputting prompt information to prompt the user to re-trigger the interaction command to complete the target selection and locking.

[0068] In this way, through the above-mentioned target loss judgment and occlusion recovery mechanism, the system can reduce target drift and false following in cases of short-term occlusion by personnel, field of view cut-out, or sudden changes in lighting, and can safely exit following when it cannot be recovered, thereby improving the system's stability and controllability.

[0069] S104. Generate chassis motion control commands based on the relative pose information and send them to the mobile chassis to drive the mobile chassis to follow the target.

[0070] In specific implementation, after the processor continuously obtains the relative pose information of the target relative to the mobile device in the locked state, it converts the relative pose information into a control quantity that can be used for chassis control, and generates a chassis motion control command to send to the mobile chassis to drive the mobile chassis to follow the target smoothly and in real time, thereby maintaining the preset following distance and following orientation relationship.

[0071] Specifically, the relative pose information includes at least the target relative distance and / or the target relative azimuth angle. The processor determines the target desired distance and desired azimuth angle according to a preset following strategy. For example, the desired distance is set to Dref, and the desired azimuth angle is 0° (indicating the target is directly in front of the mobile device). The processor then compares the current relative distance D and the current relative azimuth angle θ with the corresponding desired values ​​to obtain the distance error ed = D. Dref and the angle error eθ = θ 0. The distance error is used to characterize the forward and backward distance deviation between the mobile device and the following target, and the angle error is used to characterize the deflection deviation of the following target relative to the travel direction of the mobile device.

[0072] Here, when generating chassis motion control commands, the processor can construct and couple control laws for distance error and angle error respectively, outputting linear velocity control quantity v and angular velocity control quantity ω, thereby forming chassis motion control commands. The control law can be at least one of proportional control, PID control, model predictive control, or state estimation-based control methods.

[0073] As a preferred approach, the processor performs PID calculations on the distance error e_d to obtain the linear velocity v, and performs PID calculations on the angle error e_θ to obtain the angular velocity ω. To improve tracking smoothness, the processor can also set acceleration and angular velocity rate of change constraints on v and ω to avoid jitter or sharp turns caused by abrupt changes in control commands. Furthermore, the processor can set speed saturation and dead-zone strategies: when |e_d| is less than the distance dead-zone threshold, v is made to approach 0 to stabilize the distance; when |e_θ| is less than the angle dead-zone threshold, ω is made to approach 0 to stabilize the orientation, thereby reducing frequent corrections caused by small errors.

[0074] The chassis motion control commands may include at least one of the following: movement speed, acceleration, steering angular velocity, movement distance, and steering angle. For different chassis types such as differential chassis or omnidirectional chassis, the processor can map v and ω to execution quantities such as left and right wheel speeds, wheel group steering angles, or individual wheel vector velocities based on the chassis kinematic model, and send them to the chassis control module through a preset communication interface (e.g., serial port, CAN, Ethernet, or wireless link). After receiving the motion control commands, the chassis control module drives the motor to execute corresponding straight-line, steering, or combined movements, causing the mobile device to move towards the target direction and gradually reduce the distance and angle errors until the errors converge to a preset threshold range, thereby achieving stable following of the target.

[0075] In some implementations, to enhance safety and environmental adaptability, the processor can perform safety constraint judgments before issuing chassis motion control commands. For example, it can tailor or suppress the motion control commands by combining obstacle detection results, maximum speed limits, minimum safe distance limits, and emergency braking conditions. When an obstacle risk is detected or the following target is too close, the processor can reduce the linear speed or trigger a stop. When the following target's deflection angle is too large, the processor can prioritize outputting angular velocity to adjust the orientation before advancing, thereby reducing the risk of collision from lateral pursuit.

[0076] In this way, through the above-mentioned mechanism for generating and issuing control commands from relative pose, real-time closed-loop adjustment of the following process can be achieved, improving the following stability, smoothness, and safety controllability.

[0077] As one possible implementation, in the locked state, the system detects start and stop follow interaction commands; upon detecting a start follow interaction command, it sends a start intelligent follow command to the mobile chassis; upon detecting a stop follow interaction command, it sends a stop intelligent follow command to the mobile chassis and exits follow. The hand gesture recognition result is mapped to the start follow interaction command; the fist gesture recognition result is mapped to the stop follow interaction command.

[0078] In this embodiment, to improve the controllability and safety of the following process, the mobile device does not necessarily drive the chassis to enter the following state immediately after locking onto the target. Instead, it continuously monitors start and stop follow interaction commands while in the locked state to achieve explicit start and stop control of the intelligent follow function. In other words, the locked state is used to maintain continuous tracking and pose calculation of the target, while allowing the user to trigger start or stop at any time, thereby avoiding erroneous actions in scenarios such as multiple people, narrow passages, or when the target is briefly stationary.

[0079] Specifically, in the locked state, the processor periodically checks the interaction information associated with the target being followed and outputs a start-follow interaction command or a stop-follow interaction command based on the type of interaction trigger instruction. When a start-follow interaction command is detected, the processor generates and sends a start-intelligent-follow command to the mobile chassis, causing the chassis control module to switch from standby / locked mode to follow mode. The chassis then begins receiving and executing chassis motion control commands calculated from relative pose information, thereby driving the mobile chassis to follow the target. When a stop-follow interaction command is detected, the processor generates and sends a stop-intelligent-follow command to the mobile chassis, causing the chassis control module to exit follow mode and stop executing follow control. The mobile device then enters a state after exiting follow (e.g., returning to the locked state or paused state). After exiting follow, the processor can continue to maintain tracking updates of the target so that when the user triggers the start command again, follow can be quickly resumed without needing to reselect the target.

[0080] In a preferred embodiment, the start follow interaction command and stop follow interaction command are obtained by mapping the gesture recognition results. The processor calls a preset gesture model library to recognize the gestures of the follow target and outputs the gesture category and confidence level; when a palm gesture is recognized and the confidence level meets a preset threshold, the palm gesture recognition result is mapped to the start follow interaction command; when a fist gesture is recognized and the confidence level meets a preset threshold, the fist gesture recognition result is mapped to the stop follow interaction command.

[0081] Here, to reduce the probability of false triggering, the processor can set gesture validity verification conditions. The verification conditions include at least one of the following: the gesture duration reaches a threshold, the recognition results of multiple consecutive frames are consistent, the gesture occurrence area is within the target area information range of the following target, and the gesture confidence is greater than a threshold. Only when the verification passes will the corresponding start / stop command be confirmed and sent to the chassis.

[0082] In some implementations, to avoid frequent start / stop switching, the processor can set instruction cooldown time or debouncing logic: ignore duplicate start instructions within a preset time window after the start intelligent follow instruction has been issued, and ignore duplicate stop instructions within a preset time window after the stop intelligent follow instruction has been issued; at the same time, priority can be set for start / stop instructions, for example, when conflicting start and stop instructions are detected at the same time, stop is executed first to improve system security.

[0083] In this way, through the above-mentioned start / stop interaction command detection and gesture mapping mechanism, users can control the start and end of the follow function in a natural and intuitive way, improving the controllability, interactivity and operational security of the follow process.

[0084] This disclosure provides a mobile device follow control method that maintains target stability by performing consistency judgment on candidate targets in perceived data, and confirms and locks the target when an interactive trigger command corresponding to the candidate target is detected. In the locked state, target area information is continuously output and depth / distance information is fused to calculate the relative pose. The relative pose is then converted into distance and angle errors to generate chassis control commands, achieving closed-loop adjustment of the follow process. This reduces the probability of misfollowing in multi-person scenarios, improves the stability and controllability of target locking and following, and maintains a smoother and more accurate following effect when the target moves and the environment changes.

[0085] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0086] Based on the same inventive concept, this disclosure also provides a mobile device following control device corresponding to the mobile device following control method. Since the principle of the device in this disclosure for solving the problem is similar to the above-mentioned mobile device following control method in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0087] Please see Figure 3 , Figure 3 This is a schematic diagram of a follow control device for a mobile device provided in an embodiment of this disclosure. Figure 3As shown in the illustration, the mobile device follow control device 300 provided in this embodiment includes: The target determination module 310 is used to acquire personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and make a consistency determination on the candidate targets.

[0088] The lock trigger module 320 is used to determine that the candidate target is a following target and enter the lock state when an interactive trigger command corresponding to the candidate target is detected.

[0089] The pose determination module 330 is used to perform continuous tracking of the following target in the locked state, obtain the target area information of the following target, and calculate the relative pose information of the following target relative to the mobile device based on the target area information.

[0090] The motion control module 340 is used to generate chassis motion control commands based on the relative pose information and send them to the mobile chassis to drive the mobile chassis to follow the target.

[0091] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0092] This disclosure provides a mobile device follow control device that maintains target stability by performing consistency judgment on candidate targets in perceived data, and confirms and locks the target when an interactive trigger command corresponding to the candidate target is detected. In the locked state, it continuously outputs target area information and fuses depth / distance information to calculate the relative pose, then converts the relative pose into distance error and angle error to generate chassis control commands, achieving closed-loop adjustment of the follow process. This reduces the probability of misfollowing in multi-person scenarios, improves the stability and controllability of target locking and following, and maintains a smoother and more accurate following effect when the target moves and the environment changes.

[0093] Corresponding to Figure 1 In addition to the following control method for mobile devices, this disclosure also provides an electronic device 400, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 400 provided in an embodiment of this disclosure, including: Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including main memory 421 and external memory 422; the main memory 421, also called internal memory, is used to temporarily store the computational data in processor 41, as well as the data exchanged with external memory 422 such as hard disk. Processor 41 exchanges data with external memory 422 through main memory 421. When the electronic device 400 is running, processor 41 and memory 42 communicate through bus 43, enabling processor 41 to execute... Figure 1 and Figure 2 The steps of the follow control method for mobile devices.

[0094] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the mobile device follow control method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0095] This disclosure also provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can perform the steps of the mobile device follow control method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0096] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0100] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A follow control method for a mobile device, characterized in that, include: Acquire personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and determine the consistency of the candidate targets; When an interactive trigger command corresponding to the candidate target is detected, the candidate target is determined to be a follow target and enters a locked state; In the locked state, continuous tracking is performed on the target being followed to obtain the target area information of the target being followed, and the relative pose information of the target being followed relative to the mobile device is calculated based on the target area information. Based on the relative pose information, a chassis motion control command is generated and sent to the mobile chassis to drive the mobile chassis to follow the target.

2. The method according to claim 1, characterized in that, Acquire personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and perform consistency determination on the candidate targets, specifically including: Image data is collected using a binocular camera as relevant perception data of the person. Depth and distance information are calculated based on binocular parallax and stored in association with the image data; Perform facial recognition on the aforementioned personnel-related perception data to output the candidate targets; Based on the feature vectors, appearance consistency, and temporal continuity of the candidate targets, the consistency determination is performed to maintain target consistency.

3. The method according to claim 1, characterized in that, When an interactive trigger command corresponding to the candidate target is detected, the candidate target is determined to be a follow target and enters a locked state, specifically including: Receive at least one of gesture commands, voice commands, touch commands, and selection commands issued by mobile applications as the interaction trigger command; When the interactive trigger command is successfully associated with the candidate target, the candidate target is identified as the following target and enters a locked state; The system calls a pre-built gesture model library to recognize the gestures of candidate targets and outputs the gesture category. When the gesture category belongs to a preset set, the interaction trigger instruction is generated, wherein the preset set includes at least one or more of the following: palm, fist, scissor, and Okey gesture.

4. The method according to claim 1, characterized in that, In the locked state, continuous tracking is performed on the target to obtain target area information of the target, and the relative pose information of the target relative to the mobile device is calculated based on the target area information, specifically including: In the locked state, a target bounding box or target outline is generated in real time to represent the target being followed. The target bounding box or target outline is updated as the target moves to form the target region information; A filter-based target tracking algorithm is used to continuously track the target and output the target region information. Calculate the pixel coordinates and their changes based on the target area information; The relative pose information is calculated by fusing depth and distance information with the pixel coordinates, wherein the relative pose information includes at least one of the target relative distance and the target relative azimuth angle.

5. The method according to claim 1, characterized in that, Based on the relative pose information, a chassis motion control command is generated and sent to the mobile chassis to drive the mobile chassis to follow the target. Specifically, this includes: The relative pose information is converted into distance error and angle error; The chassis motion control command is generated based on the distance error and the angle error; The chassis motion control command is issued to control at least one of the following: moving speed, acceleration, steering angular velocity, moving distance and steering angle, so that the error converges to within the threshold range.

6. The method according to claim 1, characterized in that, The method further includes: Detect start and stop follow interaction commands while the device is locked. When a start follow interaction command is detected, a start intelligent follow command is sent to the mobile chassis; When a stop follow interaction command is detected, a command to turn off smart follow is sent to the mobile chassis and the follow is exited.

7. The method according to claim 1, characterized in that: Map the hand gesture recognition results to initiate follow-up interaction commands; The results of fist-clenching gesture recognition are mapped to a stop-follow interaction command.

8. A follow control device for a mobile device, characterized in that, include: The target determination module is used to acquire personnel-related perception data within the perception range of the mobile device, detect candidate personnel targets based on the personnel-related perception data, and make a consistency judgment on the candidate targets; The lock trigger module is used to determine that the candidate target is a following target and enter the lock state when an interactive trigger command corresponding to the candidate target is detected; The pose determination module is used to continuously track the target in the locked state, obtain the target area information of the target, and calculate the relative pose information of the target relative to the mobile device based on the target area information. The motion control module is used to generate chassis motion control commands based on the relative pose information and send them to the mobile chassis to drive the mobile chassis to follow the target.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the follow control method for the mobile device as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the follow control method for a mobile device as described in any one of claims 1 to 7.

Images