Mobile robot-based pose estimation method, device and computer equipment

Abstract

This application provides a method, apparatus, computer device, and storage medium for pose estimation of a mobile robot. The method includes: acquiring a target image through a vision system; estimating the pose information of a target object from the target image using a preset pose estimation algorithm; determining the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object based on the pose information; searching a preset prior knowledge base for the target observation angle of the vision system associated with the pose category; determining the target observation position of the mobile robot based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object; controlling the mobile robot to move to the target observation position, and simultaneously executing the step of acquiring the target image through the vision system. This method can improve the accuracy of pose estimation.

Description

Technical Field

[0001] This application relates to the field of computer vision technology. Specifically, this application relates to a pose estimation method, apparatus, computer device, and storage medium for mobile robots. Background Technology

[0002] Pose estimation is a computer vision technique for detecting objects in images and videos, used to determine the location of key parts of an object within an image. Human pose estimation, as an important branch of pose estimation, is crucial for machines to understand human movement. Through human pose estimation, the relative positions of key nodes throughout the human body in space can be obtained (generally in the form of three-axis coordinates). Currently, this technology is widely used in many fields such as motion detection, virtual reality, human-computer interaction, and video surveillance.

[0003] Traditional pose estimation methods typically utilize visual information acquired from multiple types of cameras, then use this visual information to estimate the pose of the target object. However, with traditional techniques, the accuracy of pose estimation is low when the target object is occluded. Summary of the Invention

[0004] Therefore, it is necessary to provide a pose estimation method, apparatus, computer equipment, and storage medium for mobile robots to address the aforementioned technical problems.

[0005] Firstly, this application provides a pose estimation method based on a mobile robot. The mobile robot is equipped with a vision system; the method includes:

[0006] The vision system acquires a target image; wherein the target image contains a target object;

[0007] Using a preset pose estimation algorithm, the pose information of the target object is estimated from the target image;

[0008] Based on the pose information of the target object, the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object are determined;

[0009] In a pre-defined prior knowledge base, the target observation angle of the visual system associated with the posture category is searched; wherein, the prior knowledge base stores the target observation angles of the visual system associated with different posture categories.

[0010] The target observation position of the mobile robot is determined based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0011] Control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

[0012] In one embodiment, determining the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object based on the pose information of the target object includes:

[0013] The pose information of the target object is input into the feature extractor to obtain the pose features corresponding to the target object;

[0014] The pose features corresponding to the target object are input into the pose classifier to obtain the orientation information and pose category of the target object;

[0015] Based on the pose information of the target object, calculate the proportion of the target object's pose in the target image;

[0016] The distance between the mobile robot and the target object is determined based on the stated proportion.

[0017] In one embodiment, determining the target observation position of the mobile robot based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object includes:

[0018] The current position of the target object is determined based on the current position of the mobile robot, the orientation information of the target object, and the distance between the mobile robot and the target object;

[0019] The target observation position of the mobile robot is determined based on the current position of the target object and the target observation angle of the vision system.

[0020] In one embodiment, determining the target observation position of the mobile robot based on the current position of the target object and the target observation angle of the vision system includes:

[0021] Based on the current observation orientation of the vision system, the target observation angle is rotated clockwise and counterclockwise respectively to determine the candidate observation orientation of the vision system;

[0022] Based on the current position of the target object, a safe observation area corresponding to the target object is determined; wherein, the safe observation area is used to ensure that there is a certain distance between the mobile robot and the target object;

[0023] The location where the candidate observation orientation intersects with the safe observation area is determined as the candidate observation location of the mobile robot;

[0024] The candidate observation location closest to the mobile robot is determined as the target observation location of the mobile robot.

[0025] In one embodiment, controlling the mobile robot to move to the target observation position includes:

[0026] Plan the navigation path and speed for the mobile robot to move from its current position to the target observation position;

[0027] The mobile robot is controlled to move to the target observation position based on the navigation path and at the moving speed.

[0028] In one embodiment, the planning of the movement speed from the mobile robot's current position to the target observation position includes:

[0029] The pose information of the mobile robot is determined based on the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object.

[0030] A kinematic coupling relationship is established based on the posture information of the mobile robot and the posture information of the target object;

[0031] Based on the kinematic coupling relationship, the moving speed of the mobile robot is calculated using a preset control algorithm.

[0032] In one embodiment, the posture categories stored in the prior knowledge base are standing posture, bending posture, and lying posture.

[0033] Secondly, this application provides a pose estimation device based on a mobile robot. The mobile robot is equipped with a vision system; the device includes:

[0034] An image acquisition module is used to acquire a target image through the vision system; wherein the target image contains a target object;

[0035] The pose estimation module is used to estimate the pose information of the target object from the target image using a preset pose estimation algorithm.

[0036] The information determination module is used to determine the orientation information of the target object, the distance between the mobile robot and the target object, and the posture category of the target object based on the posture information of the target object;

[0037] An angle search module is used to search for the target observation angle of the vision system associated with the posture category in a preset prior knowledge base; wherein, the prior knowledge base stores the target observation angles of the vision system associated with different posture categories.

[0038] The position determination module is used to determine the target observation position of the mobile robot based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0039] The mobile control module is used to control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

[0040] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0041] The vision system acquires a target image; wherein the target image contains a target object;

[0042] Using a preset pose estimation algorithm, the pose information of the target object is estimated from the target image;

[0043] Based on the pose information of the target object, the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object are determined;

[0044] In a pre-defined prior knowledge base, the target observation angle of the visual system associated with the posture category is searched; wherein, the prior knowledge base stores the target observation angles of the visual system associated with different posture categories.

[0045] The target observation position of the mobile robot is determined based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0046] Control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

[0047] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0048] The vision system acquires a target image; wherein the target image contains a target object;

[0049] Using a preset pose estimation algorithm, the pose information of the target object is estimated from the target image;

[0050] Based on the pose information of the target object, the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object are determined;

[0051] In a pre-defined prior knowledge base, the target observation angle of the visual system associated with the posture category is searched; wherein, the prior knowledge base stores the target observation angles of the visual system associated with different posture categories.

[0052] The target observation position of the mobile robot is determined based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0053] Control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

[0054] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0055] The vision system acquires a target image; wherein the target image contains a target object;

[0056] Using a preset pose estimation algorithm, the pose information of the target object is estimated from the target image;

[0057] Based on the pose information of the target object, the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object are determined;

[0058] In a pre-defined prior knowledge base, the target observation angle of the visual system associated with the posture category is searched; wherein, the prior knowledge base stores the target observation angles of the visual system associated with different posture categories.

[0059] The target observation position of the mobile robot is determined based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0060] Control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

[0061] The aforementioned mobile robot-based pose estimation method, apparatus, computer device, computer-readable storage medium, and computer program product utilize a vision system mounted on the mobile robot to acquire target images containing the target object. Then, a preset pose estimation algorithm is used to estimate the target object's pose information from the target image. Based on the target object's pose information, the orientation information of the target object, the distance between the mobile robot and the target object, and the target object's pose category are determined. Next, a prior knowledge base is used to search for the target observation angle of the vision system associated with the pose category. Based on the mobile robot's current position, the target observation angle, the target object's orientation information, and the distance between the robot and the target object, the target observation position of the mobile robot is determined. Finally, the mobile robot is controlled to move to the target observation position, and the target image is acquired synchronously. It can be understood that this application utilizes a mobile robot to dynamically track the movement of the target object in an open environment and adjusts the mobile robot's observation angle and position in a timely manner according to the target object's movement to obtain the best pose estimation quality. Especially when the target object is occluded or its pose is difficult to estimate accurately, the automatic adjustment of the observation angle and position by the mobile robot can obtain more comprehensive and accurate visual information about the target object, thereby improving the accuracy of pose estimation. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a pose estimation method based on a mobile robot in one embodiment;

[0063] Figure 2 This is a flowchart of a pose estimation method based on a mobile robot in one embodiment;

[0064] Figure 3 This is a structural block diagram of a pose estimation device based on a mobile robot in one embodiment;

[0065] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] In one embodiment, such as Figure 1 As shown, a pose estimation method based on a mobile robot is provided. The mobile robot is equipped with a vision system. The method includes the following steps S101-S106:

[0068] Step S101: Acquire the target image through the vision system.

[0069] The target image contains the target object. The target object can be an object such as a person, animal, or vehicle.

[0070] In the attitude estimation method provided in this application embodiment, the target image can be acquired in real time through the vision system of the mobile robot until the method stops running.

[0071] Step S102: Using a preset pose estimation algorithm, the pose information of the target object is estimated from the target image.

[0072] Specifically, in step S102, the pose estimation algorithm in the pose estimation module is used to estimate and extract the pose of the target object from the target image. The pose information can be three-dimensional pose data.

[0073] Step S103: Based on the pose information of the target object, determine the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object.

[0074] Specifically, step S103 can be achieved through the following steps S1031-S1034:

[0075] Step S1031: Input the pose information of the target object into the feature extractor to obtain the pose features corresponding to the target object.

[0076] Please refer to step S1031 as well. Figure 2 The pose information of the target object is input into the feature extractor in the pose recognition module to obtain the pose features corresponding to the target object.

[0077] Step S1032: Input the pose features corresponding to the target object into the pose classifier to obtain the orientation information and pose category of the target object.

[0078] In step S1032, the posture features corresponding to the target object are input into the posture classifier in the posture recognition module to obtain the orientation information and posture category of the target object. The posture category of the target object can be one of standing posture, bending posture, and lying posture.

[0079] It should be noted that the pose recognition module has pre-learned the mapping relationship from pose information to orientation information and pose category.

[0080] Step S1033: Calculate the proportion of the target object's pose in the target image based on the target object's pose information.

[0081] In step S1033, as an example, based on the pose information of the target object, the number of pixels occupied by the pose of the target object in the target image is determined, and the ratio of the number of pixels occupied by the pose of the target object to the total number of pixels in the target image is calculated to obtain the proportion of the pose of the target object in the target image.

[0082] Step S1034: Determine the distance between the mobile robot and the target object based on the proportion.

[0083] In step S1034, as an example, a relational model mapping proportions to distances is pre-established. This relational model can be a formula or a lookup table. Thus, by inputting the proportion of the target object's pose in the target image into the established relational model, the distance between the mobile robot and the target object is obtained.

[0084] Step S104: Search for the target observation angle of the visual system associated with the pose category in the preset prior knowledge base.

[0085] The prior knowledge base stores the target observation angles associated with different posture categories in the visual system. These different posture categories are standing, bending, and lying down. The target observation angle is the optimal observation angle.

[0086] Specifically, the prior knowledge base was built using a fixed-view camera, which mainly collected the best observation angles of the vision system under different postures.

[0087] Step S105: Determine the target observation position of the mobile robot based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0088] Specifically, step S105 can be achieved through the following steps S1051-S1052:

[0089] Step S1051: Determine the current position of the target object based on the current position of the mobile robot, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0090] In step S1051, based on the coordinate system established by the mobile robot to describe the current physical world, the distance vector between the mobile robot and the target object is determined according to the orientation information of the target object and the distance between the mobile robot and the target object. Then, the current position of the target object is determined according to the current position of the mobile robot and the distance vector between the mobile robot and the target object.

[0091] Step S1052: Determine the target observation position of the mobile robot based on the current position of the target object and the target observation angle of the vision system.

[0092] In step S1051, firstly, based on the current observation orientation of the vision system, the target observation orientation is rotated by a clockwise and counterclockwise angle, for example, 45°, respectively, to obtain two candidate observation orientations for the vision system. Then, based on the current position of the target object, a safe observation area corresponding to the target object is determined. Taking a person as an example, a circular safe observation area is constructed around the person, with the person's current position as the center and the person's height as the diameter, to ensure that the mobile robot will not enter this area. Next, the intersection of the two candidate observation orientations with the safe observation area is determined as the candidate observation position for the mobile robot. Finally, among all candidate observation positions, the candidate observation position closest to the mobile robot is determined as the target observation position for the mobile robot. The target observation position is the optimal observation position.

[0093] Step S106: Control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring target images through the vision system.

[0094] Specifically, step S106 can be achieved through the following steps S1061-S1062:

[0095] Step S1061: Plan the navigation path and speed for the mobile robot to move from its current position to the target observation position.

[0096] In step S1061, firstly, based on the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object, the pose information of the mobile robot is estimated using a preset pose back-estimation model. Then, based on the pose information of the mobile robot and the target object, a kinematic coupling relationship is established so that the mobile robot adjusts its pose to maintain consistency with the target object. Finally, based on the kinematic coupling relationship, the mobile robot's movement speed is calculated using a preset control algorithm.

[0097] Step S1062: Control the mobile robot to move to the target observation position based on the navigation path and according to the moving speed.

[0098] In the aforementioned mobile robot-based pose estimation method, a vision system mounted on the mobile robot is used to acquire target images containing the target object. Then, a pre-defined pose estimation algorithm is used to estimate the target object's pose information from the target image. Based on the target object's pose information, the orientation information of the target object, the distance between the mobile robot and the target object, and the target object's pose category are determined. Next, a prior knowledge base is used to search for the target observation angle of the vision system associated with the pose category. Based on the mobile robot's current position, the target observation angle, the target object's orientation information, and the distance between the robot and the target object, the target observation position of the mobile robot is determined. Finally, the mobile robot is controlled to move to the target observation position and simultaneously acquire the target image. This method dynamically tracks the movement of the target object in an open environment and adjusts the mobile robot's observation angle and position in a timely manner according to the target object's movement to obtain optimal pose estimation quality. Especially when the target object is occluded or its pose is difficult to estimate accurately, the automatic adjustment of the observation angle and position by the mobile robot can obtain more comprehensive and accurate visual information about the target object, thereby improving the accuracy of pose estimation.

[0099] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0100] Based on the same inventive concept, this application also provides a mobile robot-based attitude estimation device for implementing the aforementioned mobile robot-based attitude estimation method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the mobile robot-based attitude estimation device provided below can be found in the limitations of the mobile robot-based attitude estimation method described above, and will not be repeated here.

[0101] In one embodiment, such as Figure 3 As shown, a pose estimation device based on a mobile robot is provided. The mobile robot is equipped with a vision system. The device includes:

[0102] Image acquisition module 201 is used to acquire target images through a vision system; wherein the target image contains a target object;

[0103] The pose estimation module 202 is used to estimate the pose information of the target object from the target image using a preset pose estimation algorithm;

[0104] The information determination module 203 is used to determine the orientation information of the target object, the distance between the mobile robot and the target object, and the posture category of the target object based on the posture information of the target object;

[0105] Angle search module 204 is used to search for target observation angles of visual systems associated with pose categories in a preset prior knowledge base; wherein, the prior knowledge base stores target observation angles of visual systems associated with different pose categories.

[0106] The position determination module 205 is used to determine the target observation position of the mobile robot based on the current position of the mobile robot, the target observation angle of the vision system, the orientation information of the target object, and the distance between the mobile robot and the target object.

[0107] The mobile control module 206 is used to control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring target images through the vision system.

[0108] In the aforementioned mobile robot-based pose estimation device, a vision system mounted on the mobile robot is used to acquire target images containing the target object. Then, a preset pose estimation algorithm is used to estimate the target object's pose information from the target image. Based on the target object's pose information, the orientation information of the target object, the distance between the mobile robot and the target object, and the target object's pose category are determined. Next, a prior knowledge base is used to search for the target observation angle of the vision system associated with the pose category. Based on the mobile robot's current position, the target observation angle, the target object's orientation information, and the distance between the robot and the target object, the target observation position of the mobile robot is determined. Finally, the mobile robot is controlled to move to the target observation position and simultaneously acquire the target image. It can be understood that this device uses a mobile robot to dynamically track the movement of the target object in an open environment and adjusts the mobile robot's observation angle and position in a timely manner according to the target object's movement to obtain the best pose estimation quality. Especially when the target object is occluded or its pose is difficult to estimate accurately, the automatic adjustment of the observation angle and position by the mobile robot can obtain more comprehensive and accurate visual information about the target object, thereby improving the accuracy of pose estimation.

[0109] It should be noted that the above embodiments of the mobile robot-based attitude estimation device are only illustrated by the division of the above functional modules when implementing the corresponding functions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the mobile robot-based attitude estimation device and the mobile robot-based attitude estimation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0110] According to one aspect of this application, embodiments of the present invention also provide a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component. When the computer program is executed by a processor, the pose estimation method based on a mobile robot provided in embodiments of this application is performed.

[0111] Furthermore, embodiments of the present invention also provide a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor is capable of executing the computer program stored in the memory. When the computer program is executed by the processor, it can implement the pose estimation method based on a mobile robot provided in any of the above embodiments.

[0112] For example, Figure 4An embodiment of the present invention provides a computer device, which includes a bus 1110, a processor 1120, a transceiver 1130, a bus interface 1140, a memory 1150, and a user interface 1160.

[0113] In this embodiment of the invention, the device further includes a computer program stored in a memory 1150 and executable on a processor 1120, which, when executed by the processor 1120, implements the various processes of the above-described embodiments of the pose estimation method based on a mobile robot.

[0114] Transceiver 1130 is used to receive and send data under the control of processor 1120.

[0115] In this embodiment of the invention, a bus architecture (represented by bus 1110) is used. Bus 1110 may include any number of interconnected buses and bridges. Bus 1110 connects various circuits, including one or more processors represented by processor 1120 and memory represented by memory 1150.

[0116] Bus 1110 represents one or more of several types of bus architectures, including memory buses and memory controllers, peripheral buses, Accelerated Graphics Port (AGP), processors, or local buses using any bus architecture from various bus architectures. As an example and not a limitation, such architectures include: Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) buses, and Peripheral Component Interconnect (PCI) buses.

[0117] The processor 1120 can be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The processors mentioned above include: general-purpose processors, central processing units (CPUs), network processors (NPs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), programmable logic arrays (PLAs), microcontroller units (MCUs) or other programmable logic devices, discrete gates, transistor logic devices, and discrete hardware components. They can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. For example, the processor can be a single-core processor or a multi-core processor, and the processor can be integrated on a single chip or located on multiple different chips.

[0118] Processor 1120 can be a microprocessor or any conventional processor. The method steps disclosed in the embodiments of the present invention can be directly executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in readable storage media known in the art, such as Random Access Memory (RAM), Flash Memory, Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), registers, etc. The readable storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0119] Bus 1110 can also connect various other circuits, such as peripheral devices, voltage regulators, or power management circuits. Bus interface 1140 provides an interface between bus 1110 and transceiver 1130, all of which are well known in the art. Therefore, embodiments of the present invention will not be described further.

[0120] Transceiver 1130 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. For example, transceiver 1130 receives external data from other devices, and transceiver 1130 is used to send data processed by processor 1120 to other devices. Depending on the nature of the computer system, a user interface 1160 may also be provided, such as a touchscreen, physical keyboard, monitor, mouse, speaker, microphone, trackball, joystick, or stylus.

[0121] It should be understood that, in embodiments of the present invention, memory 1150 may further include memory remotely configured relative to processor 1120, and such remotely configured memory can be connected to a server via a network. One or more portions of the aforementioned network may be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), wide area network (WAN), wireless wide area network (WWAN), metropolitan area network (MAN), Internet, public switched telephone network (PSTN), ordinary old-style telephone service (POTS), cellular telephone network, wireless network, Wi-Fi network, and combinations of two or more of the aforementioned networks. For example, cellular telephone networks and wireless networks can be Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), WiMAX, General Packet Radio Service (GPRS), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), Advanced Long Term Evolution (LTE-A), Universal Mobile Telecommunications System (UMTS), Enhanced Mobile Broadband (eMBB), Massive Machine Type Communication (mMTC), Ultra Reliable Low Latency Communications (uRLLC), etc.

[0122] It should be understood that the memory 1150 in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory. Non-volatile memory includes: read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0123] Volatile memory includes random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1150 described in this embodiment includes, but is not limited to, the above and any other suitable types of memory.

[0124] In this embodiment of the invention, the memory 1150 stores the following elements of the operating system 1151 and the application 1152: executable modules, data structures, or subsets thereof, or extended sets thereof.

[0125] Specifically, the operating system 1151 includes various system programs, such as a framework layer, a core library layer, and a driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 1152 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of this embodiment of the invention can be included in the application program 1152. The application program 1152 includes applets, objects, components, logic, data structures, and other computer system executable instructions that perform specific tasks or implement specific abstract data types.

[0126] In addition, this embodiment of the invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the various processes of the above-described embodiments of the pose estimation method based on mobile robots and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0127] Computer-readable storage media include: permanent and non-permanent, removable and non-removable media, which are tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media include: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination thereof. Computer-readable storage media include: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape storage, magnetic disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (e.g., punched cards or raised structures in grooves on which instructions are recorded), or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in the embodiments of the present invention, computer-readable storage media do not include temporary signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, devices, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to solve the problems addressed by the embodiments of the present invention, depending on actual needs.

[0130] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0131] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (including: a personal computer, a server, a data center, or other network device) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media listed above that can store program code.

[0132] In the description of the embodiments of the present invention, those skilled in the art should understand that the embodiments of the present invention can be implemented as methods, apparatuses, devices, and storage media. Therefore, the embodiments of the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Furthermore, in some embodiments, the embodiments of the present invention can also be implemented as a computer program product in one or more computer-readable storage media, the computer-readable storage media containing computer program code.

[0133] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination thereof. In embodiments of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0134] The computer program code contained in the aforementioned computer-readable storage medium may be transmitted using any suitable medium, including wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0135] The embodiments of the present invention describe the provided methods, apparatus, and devices through flowcharts and / or block diagrams.

[0136] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a computer or other programmable data processing apparatus, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0137] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable data processing apparatus provide a process for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0138] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. A pose estimation method for mobile robots, characterized in that, The mobile robot is equipped with a vision system; the method includes: The vision system acquires a target image; wherein the target image contains a target object; Using a preset pose estimation algorithm, the pose information of the target object is estimated from the target image; Based on the pose information of the target object, the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object are determined; In a pre-defined prior knowledge base, the target observation angle of the visual system associated with the posture category is searched; wherein, the prior knowledge base stores the target observation angles of the visual system associated with different posture categories. The current position of the target object is determined based on the current position of the mobile robot, the orientation information of the target object, and the distance between the mobile robot and the target object; Based on the current observation orientation of the vision system, the target observation angle is rotated clockwise and counterclockwise respectively to determine the candidate observation orientation of the vision system; Based on the current position of the target object, a safe observation area corresponding to the target object is determined; wherein, the safe observation area is used to ensure that there is a certain distance between the mobile robot and the target object; The location where the candidate observation orientation intersects with the safe observation area is determined as the candidate observation location of the mobile robot; The candidate observation location closest to the mobile robot is determined as the target observation location of the mobile robot; Control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

2. The method according to claim 1, characterized in that, The step of determining the orientation information of the target object, the distance between the mobile robot and the target object, and the posture category of the target object based on the posture information of the target object includes: The pose information of the target object is input into the feature extractor to obtain the pose features corresponding to the target object; The pose features corresponding to the target object are input into the pose classifier to obtain the orientation information and pose category of the target object; Based on the pose information of the target object, calculate the proportion of the target object's pose in the target image; The distance between the mobile robot and the target object is determined based on the stated proportion.

3. The method according to claim 1, characterized in that, Controlling the mobile robot to move to the target observation position includes: Plan the navigation path and speed for the mobile robot to move from its current position to the target observation position; The mobile robot is controlled to move to the target observation position based on the navigation path and at the moving speed.

4. The method according to claim 3, characterized in that, The planned movement speed from the current position of the mobile robot to the target observation position includes: The pose information of the mobile robot is determined based on the orientation information of the target object, the distance between the mobile robot and the target object, and the pose category of the target object. A kinematic coupling relationship is established based on the posture information of the mobile robot and the posture information of the target object; Based on the kinematic coupling relationship, the moving speed of the mobile robot is calculated using a preset control algorithm.

5. The method according to any one of claims 1-4, characterized in that, The prior knowledge base stores posture categories such as standing posture, bending posture, and lying posture.

6. A pose estimation device based on a mobile robot, characterized in that, The mobile robot is equipped with a vision system; the device includes: An image acquisition module is used to acquire a target image through the vision system; wherein the target image contains a target object; The pose estimation module is used to estimate the pose information of the target object from the target image using a preset pose estimation algorithm. The information determination module is used to determine the orientation information of the target object, the distance between the mobile robot and the target object, and the posture category of the target object based on the posture information of the target object; An angle search module is used to search for the target observation angle of the vision system associated with the posture category in a preset prior knowledge base; wherein, the prior knowledge base stores the target observation angles of the vision system associated with different posture categories. A position determination module is used to determine the current position of the target object based on the current position of the mobile robot, the orientation information of the target object, and the distance between the mobile robot and the target object; to determine the candidate observation orientation of the vision system by rotating the target observation angle clockwise and counterclockwise based on the current observation orientation of the vision system; to determine the safe observation area corresponding to the target object based on the current position of the target object; to determine the position where the candidate observation orientation intersects with the safe observation area as the candidate observation position of the mobile robot; and to determine the candidate observation position closest to the mobile robot as the target observation position of the mobile robot; wherein, the safe observation area is used to ensure that there is a certain distance between the mobile robot and the target object; The mobile control module is used to control the mobile robot to move to the target observation position and simultaneously execute the step of acquiring the target image through the vision system.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images