Long-distance robot target following method and device, medium and equipment

Abstract

The invention discloses a long-distance robot target following method and device, a medium and equipment. Target recognition is carried out according to RGB image data to obtain the size and position of a target; predicting the size and position of a target in a next moment frame through a Kalman filtering method, and carrying out maximum IOU matching; if matching succeeds, the track is updated; if the matching is unsuccessful, generating a topological graph for the current image through a diffusion model, inputting the topological graph, the current image and previous continuous n frames of observation graphs into a navigation model to obtain a predicted waypoint comprising a sub-node closest to the target, navigating to the sub-node closest to the target according to the predicted waypoint, continuously updating the topological graph and the sub-node, and obtaining a navigation result; and continuing to track the target until the target is detected. The method only depends on visual features, the dependence of a model and the complexity of an algorithm are greatly reduced while long-distance navigation tracking is guaranteed, and meanwhile, the cost of hardware is reduced.

Description

Technical Field

[0001] The invention relates to a long-distance robot target following method, device, medium and equipment, belonging to the technical field of path navigation. Background Art

[0002] The robot visual local navigation problem can be defined as the optimal collision-free trajectory planning scheme to move the robot from the starting point to the target point within the visual range. This allows the robot to achieve efficient local navigation and path planning relying only on the front camera, reducing the dependence on sensors and the complexity of the algorithm. Existing methods have achieved outstanding performance by leveraging the powerful feature association capabilities of Transformer and the visual feature extraction capabilities of neural networks. However, in real environments, robots need to constantly analyze real scenes, understand the behavior (trajectory) of other road users, and predict their next position in order to plan safe movements and reduce their reaction time.

[0003] When the target is lost, existing technologies need to cooperate with GPS positioning or inertial navigation to find the target again, and these methods require increasing the cost of robot hardware. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a long-distance robot target following method, device, medium and equipment.

[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0006] In one aspect, the present invention provides a long-distance robot target following method, comprising: Get the RGB image data in front of the robot; Perform target recognition on RGB image data to obtain the size and position of the target; According to the size and position of the identified target, the size and position of the target in the next frame are predicted by the Kalman filter method, and the size and position of the identified target are matched with the size and position of the target in the predicted next frame for maximum IOU; If the IOU matching value is greater than the threshold, it is considered a successful match and the trajectory is updated; If the IOU matching value is not greater than the threshold, the match is considered unsuccessful. The topology map is generated for the current image through the diffusion model, and the topology map is compared with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target. According to the predicted waypoint, the system navigates to the subnode closest to the target, continuously updates the topology graph and subnodes, and continues to track the target until the target is detected.

[0007] Furthermore, the object recognition on the RGB image data to obtain the size and position of the object includes: Defining the target state ; Get the target status at the last moment ob t-1 , calculate the target state at the current moment ob t A priori estimate of , the corresponding formula is: ; In the formula, F Represents the state change matrix; Use the target status information of the last moment Estimate the target state information at the current moment The prior estimate covariance of , the corresponding formula is: ; In the formula, t >1 o'clock, , exist t =1, , Estimate the covariance matrix for known initial states, T The symbol for the transposed matrix, Q represents the process noise covariance matrix, K is the Kalman gain, H is the state observation matrix, I is the identity matrix; Kalman Gain K The corresponding formula is: ; In the formula, R represents the observation noise covariance matrix; The size and position of the target in the next frame at the next moment are predicted by updating the state. The corresponding formula is: ; in, is the observed value at the current time t, is a prediction of the observed value based on the prior estimate.

[0008] Furthermore, the current image is subjected to a diffusion model to generate a topological map, and the topological map is combined with the current image and the previous continuous n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target, and navigation to the subnode closest to the target is performed according to the predicted waypoint, including: Generate multiple images as candidate sub-nodes through the DDPM diffusion model according to the input image, wherein the candidate sub-nodes represent the possible locations where the robot may go; Through the heuristic function, the candidate child nodes that are closer to the current position are evaluated as child nodes; Estimate the minimum distance from the robot to the child node according to the visual navigation model to generate a connection edge, and update the topology map according to the connection edge; Waypoints are predicted based on the topological map, and the waypoints include the position coordinates of the robot in the physical environment, the relative direction and distance information to the subnodes, and the corresponding navigation instructions. The robot is navigated to the subnode with the smallest distance from the current position based on the waypoints.

[0009] In a second aspect, the present invention further provides a long-distance robot target following device, comprising: An acquisition module is used to acquire RGB image data in front of the robot; The target detection module is used to identify the target based on the RGB image data to obtain the size and position of the target; The target tracking module is used to predict the size and position of the target in the next frame by using the Kalman filter method according to the size and position of the identified target, and to perform maximum IOU matching between the size and position of the identified target and the size and position of the predicted target in the next frame. If the IOU matching value is greater than the threshold, it is considered a successful match and the tracking trajectory is updated; If the IOU matching value is not greater than the threshold, the match is considered unsuccessful. The topology map is generated for the current image through the diffusion model, and the topology map is compared with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target. According to the predicted waypoint, the system navigates to the subnode closest to the target, continuously updates the topology graph and subnodes, and continues to track the target until the target is detected.

[0010] Furthermore, the target detection module is used to extract the size and position Box of the target using a lightweight target detection framework according to the RGB image data, Box = [x, y, w, h], [x, y] represents the center point coordinates of the target, and [w, h] represents the width and height of the target in the image.

[0011] Furthermore, the target tracking module is used to: Defining the target state ; Get the target status at the last moment ob t-1 , calculate the target state at the current moment ob t A priori estimate of , the corresponding formula is: ; In the formula, F Represents the state change matrix; Use the target status information of the last moment Estimate the target state information at the current moment The prior estimate covariance of , the corresponding formula is: ; In the formula, t >1 o'clock, , exist t =1, , Estimate the covariance matrix for known initial states, T The symbol for the transposed matrix, Q represents the process noise covariance matrix, K is the Kalman gain, H is the state observation matrix, I is the identity matrix; Kalman Gain K The corresponding formula is: ; In the formula, R represents the observation noise covariance matrix; The size and position of the target in the next frame at the next moment are predicted by updating the state. The corresponding formula is: ; in, is the observed value at the current time t, is a prediction of the observed value based on the prior estimate.

[0012] Furthermore, the target tracking module is also used for: Generate multiple images as candidate sub-nodes through the DDPM diffusion model according to the input image, wherein the candidate sub-nodes represent the possible locations where the robot may go; Through the heuristic function, the candidate child nodes that are closer to the current position are evaluated as child nodes; Estimate the minimum distance from the robot to the child node according to the visual navigation model to generate a connection edge, and update the topology map according to the connection edge; Waypoints are predicted based on the topological map, and the waypoints include the position coordinates of the robot in the physical environment, the relative direction and distance information to the subnodes, and the corresponding navigation instructions. The robot is navigated to the subnode with the smallest distance from the current position based on the waypoints.

[0013] In a third aspect, the present invention further provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method of the first aspect.

[0014] In a fourth aspect, the present invention also provides a computer device, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method of the first aspect.

[0015] The beneficial effects achieved by the present invention are: The present invention can reduce the impact of detection errors and environmental noise on tracking accuracy, and can predict the target position and adapt to short-term occlusion when the target cannot be detected. In the case of lost tracking target, the visual navigation model can be used to predict waypoints, publish odometer data, and navigate to the child node closest to the target until the target is found, thereby continuing to track the target and completing long-distance tracking. In addition, the present invention only relies on visual features, which greatly reduces the dependence on the model and the complexity of the algorithm while ensuring long-distance navigation tracking, and reduces the cost of hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the complete process of target following; Figure 2 It is a schematic diagram of the target tracking process; Figure 3 It is a schematic diagram of the visual navigation process; Figure 4 It is a flow chart of the VINT model. DETAILED DESCRIPTION

[0017] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0018] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0019] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0020] Embodiment 1, as Figure 1 As shown, this embodiment introduces a long-distance robot target following method, including: Get the RGB image data in front of the robot; Perform target recognition on RGB image data to obtain the size and position of the target; According to the size and position of the identified target, the size and position of the target in the next frame are predicted by the Kalman filter method, and the size and position of the identified target are matched with the size and position of the target in the predicted next frame for maximum IOU; If the IOU matching value is greater than the threshold, it is considered a successful match and the trajectory is updated; If the IOU matching value is not greater than the threshold, the match is considered unsuccessful. The topology map is generated for the current image through the diffusion model, and the topology map is compared with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target. According to the predicted waypoint, the system navigates to the subnode closest to the target, continuously updates the topology graph and subnodes, and continues to track the target until the target is detected.

[0021] The step of performing target recognition on the RGB image data to obtain the size and position of the target includes: According to the RGB image data, a lightweight target detection framework is used to extract the size and position Box of the target, Box = [x, y, w, h], [x, y] represents the center point coordinates of the target, and [w, h] represents the width and height of the target in the image.

[0022] The method of predicting the size and position of the target in the next frame by using the Kalman filter method includes: Defining the target state ; Get the target status at the last moment ob t-1 , calculate the target state at the current moment ob t A priori estimate of , the corresponding formula is: ; In the formula, F Represents the state change matrix; Use the target status information of the last moment Estimate the target state information at the current moment The prior estimate covariance of , the corresponding formula is: ; In the formula, t >1 o'clock, , exist t =1, , Estimate the covariance matrix for known initial states, T The symbol for the transposed matrix, Q represents the process noise covariance matrix, K is the Kalman gain, H is the state observation matrix, I is the identity matrix; Kalman Gain K The corresponding formula is: ; In the formula, R represents the observation noise covariance matrix; The size and position of the target in the next frame at the next moment are predicted by updating the state. The corresponding formula is: ; in, is the observed value at the current time t, is a prediction of the observed value based on the prior estimate.

[0023] The topological map is generated by the diffusion model for the current image, and the topological map is combined with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target, and navigation to the subnode closest to the target is performed according to the predicted waypoint, including: Generate multiple images as candidate sub-nodes through the DDPM diffusion model according to the input image, wherein the candidate sub-nodes represent the possible locations where the robot may go; Through the heuristic function, the candidate child nodes that are closer to the current position are evaluated as child nodes; Estimate the minimum distance from the robot to the child node according to the visual navigation model to generate a connection edge, and update the topology map according to the connection edge; Waypoints are predicted based on the topological map, and the waypoints include the position coordinates of the robot in the physical environment, the relative direction and distance information to the subnodes, and the corresponding navigation instructions. The robot is navigated to the subnode with the smallest distance from the current position based on the waypoints.

[0024] Embodiment 2: This embodiment introduces a long-distance robot target following method, comprising: Step 1: Get the RGB image data published by the camera topic in the intelligent robot Ros system.

[0025] Step 2: Input the RGB image data into the object detection module for object recognition.

[0026] The target detection module is a lightweight target detection framework yolov8. The target detection module outputs the target category and position [x, y, w, h], where [x, y] represents the center point coordinates of the target. [w, h] represents the width and height of the target in the image.

[0027] Step 3: The size and position of the target output by the target detection module are input into the target tracking module.

[0028] 3.1. Get the target size and position identified by the target detection module in each frame, that is, the target's Box[x, y, w, h].

[0029] 3.2. Input Box [x, y, w, h] into the target tracking module, and use the Kalman filter method to predict the target's motion trajectory Box1 [x1, y1, w1, h1] in the next frame. Match Box with Box1 for the maximum iou. If the iou is greater than the threshold of 0.7, it is considered a successful match and the trajectory is updated. Figure 2 shown.

[0030] The information involved in the Kalman filter method is defined as follows: Target state , Indicates the location of the target, Indicates the width of the target, Indicates the height of the target; State transition matrix , the process noise covariance matrix , the state observation matrix , the observation noise covariance matrix , the initial state estimated covariance matrix .

[0031] Kalman filtering is divided into two steps: prediction and update. The prediction includes prior state estimation, and the update includes Kalman gain and posterior state estimation.

[0032] 1: Calculate the prior estimate ( ): ; in, is the state prediction at time t, is the state prediction at time t-1.

[0033] 2: Use the target state information at the previous moment to estimate the prior estimated covariance of the target state information at the current moment, calculated as follows:

[0034] In the formula, t >1 o'clock, ,exist t =1, , K is the Kalman gain, I is the identity matrix.

[0035] 3: Calculate Kalman gain :

[0036] The Kalman gain determines the relative weight of predictions and measurements.

[0037] 4: Calculate the posterior state estimate , combining prediction and measurement information to update the status,

[0038] in, is the observed value at time t, is a prediction of the observed value based on the prior estimate.

[0039] Step 4: When target tracking is lost, the visual navigation module is used to find the target again, and the target tracking module is updated to track the target again. Specifically, Figure 3 As shown in the figure, the current image and the previous four consecutive frames of images are used as observation images; other images are generated from the current image through the diffusion model DDPM, and the generated images are scored through heuristic learning. If the scoring is successful, it is used as a child node, and a topological map is constructed for each node; the observation image and the topological map are input into the navigation model vint, the waypoints are predicted, and navigation is performed to the child node with the smallest distance from the current position.

[0040] 1. The establishment of the topology graph is divided into sub-node generation and sub-node scoring.

[0041] Multiple images are generated through the DDPM diffusion model as candidate child nodes, indicating the possible locations of the robot. Through heuristic functions such as geometric distance, nodes that are closer to the current location are evaluated as child nodes. Connection edges are generated based on the minimum distance estimated by the visual navigation model vint, and the topology graph is updated.

[0042] 2. Waypoint prediction: The predicted waypoints include the robot's position coordinates in the physical environment, the relative direction and distance information to these sub-nodes, and the corresponding navigation instructions (such as forward speed, adjusted posture), and publish topics through ROS, so that it can navigate to the sub-node with the smallest distance from the current position.

[0043] 3. Vint model: like Figure 4 As shown in the figure, the VINT model is based on the transformer architecture. It takes 5 consecutive frames of pictures as observation images, uses the sub-images generated by the diffusion model as input, and inputs both into the EfficientNet module to extract visual features, which are then concat into 512-dimensional tokens. Combined with position encoding, these encodings are processed by a multi-layer multi-head attention module to generate waypoints. The sub-node closest to the current position is selected and navigated until the target is found.

[0044] The images generated by the diffusion model are mainly used to guide the robot to navigate step by step from the starting position without the need for map construction.

[0045] Embodiment 3, based on the same inventive concept as other embodiments, this embodiment introduces a long-distance robot target following device, comprising: An acquisition module is used to acquire RGB image data in front of the robot; The target detection module is used to identify the target based on the RGB image data to obtain the size and position of the target; The target tracking module is used to predict the size and position of the target in the next frame by using the Kalman filter method according to the size and position of the identified target, and to perform maximum IOU matching between the size and position of the identified target and the size and position of the predicted target in the next frame. If the IOU matching value is greater than the threshold, it is considered a successful match and the tracking trajectory is updated; If the IOU matching value is not greater than the threshold, the match is considered unsuccessful. The topology map is generated for the current image through the diffusion model, and the topology map is compared with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target. According to the predicted waypoint, the system navigates to the subnode closest to the target, continuously updates the topology graph and subnodes, and continues to track the target until the target is detected.

[0046] The target detection module is used to extract the size and position Box of the target using a lightweight target detection framework according to the RGB image data, Box = [x, y, w, h], [x, y] represents the center point coordinates of the target, and [w, h] represents the width and height of the target in the image.

[0047] The target tracking module is used to: Defining the target state ; Get the target status at the last moment ob t-1 , calculate the target state at the current moment ob t A priori estimate of , the corresponding formula is: ; In the formula, F Represents the state change matrix; Use the target status information of the last moment Estimate the target state information at the current moment The prior estimate covariance of , the corresponding formula is: ; In the formula, t >1 o'clock, , exist t =1, , Estimate the covariance matrix for known initial states, T The symbol for the transposed matrix, Q represents the process noise covariance matrix, K is the Kalman gain, H is the state observation matrix, Iis the identity matrix; Kalman Gain K The corresponding formula is: ; In the formula, R represents the observation noise covariance matrix; The size and position of the target in the next frame at the next moment are predicted by updating the state. The corresponding formula is: ; in, is the observed value at the current time t, is a prediction of the observed value based on the prior estimate.

[0048] The target tracking module is further used for: Generate multiple images as candidate sub-nodes through the DDPM diffusion model according to the input image, wherein the candidate sub-nodes represent the possible locations where the robot may go; Through the heuristic function, the candidate child nodes that are closer to the current position are evaluated as child nodes; Estimate the minimum distance from the robot to the child node according to the visual navigation model to generate a connection edge, and update the topology map according to the connection edge; Waypoints are predicted based on the topological map, and the waypoints include the position coordinates of the robot in the physical environment, the relative direction and distance information to the subnodes, and the corresponding navigation instructions. The robot is navigated to the subnode with the smallest distance from the current position based on the waypoints.

[0049] Embodiment 4 is based on the same inventive concept as other embodiments. This embodiment introduces a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes the method described in Embodiment 1.

[0050] Example 5 is based on the same inventive concept as other examples. This example introduces a computer device, including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method described in Example 2.

[0051] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0053] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0055] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A long-distance robot target following method, characterized in that: include: Get the RGB image data in front of the robot; Perform target recognition on RGB image data to obtain the size and position of the target; According to the size and position of the identified target, the size and position of the target in the next frame are predicted by the Kalman filter method, and the size and position of the identified target are matched with the size and position of the target in the predicted next frame for maximum IOU; If the IOU matching value is greater than the threshold, it is considered a successful match and the trajectory is updated; If the IOU matching value is not greater than the threshold, the match is considered unsuccessful. The topology map is generated for the current image through the diffusion model, and the topology map is compared with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target. According to the predicted waypoint, the system navigates to the subnode closest to the target, continuously updates the topology graph and subnodes, and continues to track the target until the target is detected.

2. The long-distance robot target following method according to claim 1, characterized in that: The step of performing target recognition on the RGB image data to obtain the size and position of the target includes: According to the RGB image data, a lightweight target detection framework is used to extract the size and position Box of the target, Box = [x, y, w, h], [x, y] represents the center point coordinates of the target, and [w, h] represents the width and height of the target in the image.

3. The long-distance robot target following method according to claim 2, characterized in that: The method of predicting the size and position of the target in the next frame by using the Kalman filter method includes: Defining the target state ; Get the target status at the last moment ob t-1 , calculate the target state at the current moment ob t A priori estimate of , the corresponding formula is: ； In the formula, F Represents the state change matrix; Use the target status information of the last moment Estimate the target state information at the current moment The prior estimate covariance of , the corresponding formula is: ； In the formula, t >1 o'clock, , exist t =1, , Estimate the covariance matrix for known initial states, T The symbol for the transposed matrix, Q represents the process noise covariance matrix, K is the Kalman gain, H is the state observation matrix, I is the identity matrix; Kalman Gain K The corresponding formula is: ； In the formula, R represents the observation noise covariance matrix; The size and position of the target in the next frame at the next moment are predicted by updating the state. The corresponding formula is: ； in, is the observed value at the current time t, is a prediction of the observed value based on the prior estimate.

4. The long-distance robot target following method according to claim 1, characterized in that: The topological map is generated by the diffusion model for the current image, and the topological map is combined with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target, and navigation to the subnode closest to the target is performed according to the predicted waypoint, including: Generate multiple images as candidate sub-nodes through the DDPM diffusion model according to the input image, wherein the candidate sub-nodes represent the possible locations where the robot may go; Through the heuristic function, the candidate child nodes that are closer to the current position are evaluated as child nodes; Estimate the minimum distance from the robot to the child node according to the visual navigation model to generate a connection edge, and update the topology map according to the connection edge; Waypoints are predicted based on the topological map, and the waypoints include the position coordinates of the robot in the physical environment, the relative direction and distance information to the subnodes, and the corresponding navigation instructions. The robot is navigated to the subnode with the smallest distance from the current position based on the waypoints.

5. A long-distance robot target following device, characterized in that: include: An acquisition module is used to acquire RGB image data in front of the robot; The target detection module is used to identify the target based on the RGB image data to obtain the size and position of the target; The target tracking module is used to predict the size and position of the target in the next frame by using the Kalman filter method according to the size and position of the identified target, and to perform maximum IOU matching between the size and position of the identified target and the size and position of the predicted target in the next frame. If the IOU matching value is greater than the threshold, it is considered a successful match and the tracking trajectory is updated; If the IOU matching value is not greater than the threshold, the match is considered unsuccessful. The topology map is generated for the current image through the diffusion model, and the topology map is compared with the current image and the previous continuous image. n The frame observation graph is input into the navigation model to obtain the predicted waypoint including the subnode closest to the target. According to the predicted waypoint, the system navigates to the subnode closest to the target, continuously updates the topology graph and subnodes, and continues to track the target until the target is detected.

6. The long-distance robot target following device according to claim 5, characterized in that: The target detection module is used to extract the size and position Box of the target using a lightweight target detection framework according to the RGB image data, Box = [x, y, w, h], [x, y] represents the center point coordinates of the target, and [w, h] represents the width and height of the target in the image.

7. The long-distance robot target following device according to claim 6, characterized in that: The target tracking module is used to: Defining the target state ; Get the target status at the last moment ob t-1 , calculate the target state at the current moment ob t A priori estimate of , the corresponding formula is: ； In the formula, F Represents the state change matrix; Use the target status information of the last moment Estimate the target state information at the current moment The prior estimate covariance of , the corresponding formula is: ； In the formula, t >1 o'clock, , exist t =1, , Estimate the covariance matrix for known initial states, T The symbol for the transposed matrix, Q represents the process noise covariance matrix, K is the Kalman gain, H is the state observation matrix, I is the identity matrix; Kalman Gain K The corresponding formula is: ； In the formula, R represents the observation noise covariance matrix; The size and position of the target in the next frame at the next moment are predicted by updating the state. The corresponding formula is: ； in, is the observed value at the current time t, is a prediction of the observed value based on the prior estimate.

8. The long-distance robot target following device according to claim 5, characterized in that: The target tracking module is further used for: Generate multiple images as candidate sub-nodes through the DDPM diffusion model according to the input image, wherein the candidate sub-nodes represent the possible locations where the robot may go; Through the heuristic function, the candidate child nodes that are closer to the current position are evaluated as child nodes; Estimate the minimum distance from the robot to the child node according to the visual navigation model to generate a connection edge, and update the topology map according to the connection edge; Waypoints are predicted based on the topological map, and the waypoints include the position coordinates of the robot in the physical environment, the relative direction and distance information to the subnodes, and the corresponding navigation instructions. The robot is navigated to the subnode with the smallest distance from the current position based on the waypoints.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions which, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 4.

10. A computer device, characterized in that: include, One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods described in claims 1 to 4.

Images