Movement control method, movement control system, multi-mode cashier robot and medium

Abstract

The invention provides a movement control method, a movement control system, a multi-mode cashier robot and a medium, and relates to the technical field of robots, and the method comprises the steps: obtaining and fusing laser point cloud data and visual feature data of a target region, and obtaining a composite semantic map of the target region; in response to the acquired cashier task, performing path planning based on a composite semantic map to obtain a planned path from the initial position to the target position; and obtaining current environment information of the multi-mode mobile cashier robot, controlling the multi-mode mobile cashier robot to switch from the first moving form to the second moving form according to the current environment information, and moving to the target position along the planned path in the second moving form. According to the invention, the intelligent switching of the wheel and foot forms of the multi-mode mobile cashier service robot is realized, and the environmental adaptability of the multi-mode mobile cashier service robot is improved.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more specifically, to a motion control method, a motion control system, a multimodal cash register robot, and a medium. Background Technology

[0002] With the rapid development of unmanned retail and smart stores, mobile checkout robots have been gradually applied in commercial scenarios such as shopping malls and supermarkets, providing customers with flexible payment services. Existing mobile checkout robots are either single-wheeled or legged structures. Among them, the wheeled structure has poor adaptability to the environment, while the legged structure has a slow movement speed, neither of which can meet the actual use needs of commercial scenarios. Summary of the Invention

[0003] In view of this, the purpose of this application is to overcome the shortcomings of the prior art and provide a motion control method, a motion control system, a multimodal cash register robot, and a medium. This application provides the following technical solution: In a first aspect, this application provides a motion control method applied to a multimodal mobile checkout robot, the multimodal mobile checkout robot comprising: a legged upper body and a wheeled chassis, the legged upper body being detachably connected to the wheeled chassis, the method comprising: Acquire laser point cloud data and visual feature data of the target area, and fuse the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area; In response to the acquired checkout task, path planning is performed based on the composite semantic map to obtain a planned path from the initial position to the target position; The current environmental information of the multimodal mobile checkout robot is obtained, and the multimodal mobile checkout robot is controlled to switch from a first movement mode to a second movement mode according to the current environmental information. The robot then moves towards the target location along the planned path in the second movement mode. The environmental information includes obstacle information, ground information, and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode.

[0004] In one embodiment, the method further includes: obtaining the current position of the multimodal mobile checkout robot; determining whether the distance between the current position and the target position is less than a preset distance threshold; if so, obtaining the interaction requirements of the target user; determining the target interaction posture based on the interaction requirements; and controlling the multimodal mobile checkout robot to adjust to the target interaction posture.

[0005] In one embodiment, the interaction requirements include: user height, ground slope, and / or product height. Obtaining the interaction requirements of the target user includes: obtaining a local area image of the target location, and determining the user height, the ground slope, and / or the product height based on the local area image.

[0006] In one embodiment, after controlling the multimodal mobile POS robot to adjust to the target interactive posture, the method further includes: issuing preset guidance information to prompt the target user to make a payment according to the preset guidance information; determining whether payment information has been obtained, and if so, controlling the multimodal mobile POS robot to return to the initial position.

[0007] In one embodiment, the obstacle information includes: obstacle distribution and obstacle size; the ground information includes: ground type; the passable space information includes: passable width; and controlling the multimodal mobile checkout robot to switch from a first movement mode to a second movement mode based on the current environment information includes: If the obstacle distribution is continuous, or the obstacle size is smaller than a preset size threshold, or the ground type is a preset ground type, or the passable width is smaller than a preset width threshold, then the multimodal mobile cash register robot is controlled to switch from the wheeled form to the legged form. If the obstacle distribution is obstacle-free, the ground type is not the preset ground type, and the passable width is greater than or equal to the preset width threshold, then the multimodal mobile POS robot is controlled to switch from the legged form to the wheeled form.

[0008] In one embodiment, controlling the multimodal mobile checkout robot to move along the planned path towards the target location in the second movement pattern includes: If the second movement mode is a foot-based mode, then after the foot-based upper body and the wheeled chassis are separated, the foot-based upper body is controlled to move along the planned path to the target position. If the second movement mode is a wheeled mode, then after the footed upper body and the wheeled chassis are connected, the wheeled chassis and the footed upper body are controlled to move together along the planned path to the target position.

[0009] In one embodiment, after the control of the separation of the legged upper body and the wheeled chassis, the method further includes: determining an obstacle passage path based on the composite semantic map and the current environmental information of the multimodal mobile POS robot; controlling the wheeled chassis to follow the obstacle passage path, and after passing through the obstacle, controlling the wheeled chassis to connect with the legged upper body.

[0010] Secondly, this application provides a mobile control system, the system comprising: The map building module is used to acquire laser point cloud data and visual feature data of the target area, and fuse the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area. The path planning module is used to respond to the obtained checkout task and perform path planning based on the composite semantic map to obtain the planned path from the initial position to the target position. The control module is used to acquire the current environmental information of the multimodal mobile checkout robot, and control the multimodal mobile checkout robot to switch from a first movement mode to a second movement mode according to the current environmental information, and move towards the target location along the planned path in the second movement mode. The environmental information includes: obstacle information, ground information and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode.

[0011] Thirdly, this application provides a multimodal mobile checkout robot, including: a legged upper body, a wheeled chassis, and the mobile control system described in the second aspect. The legged upper body is detachably connected to the wheeled chassis, and the mobile control system is communicatively connected to both the legged upper body and the wheeled chassis.

[0012] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the motion control method described in the first aspect.

[0013] The mobile control method, mobile control system, multimodal POS robot, and medium provided in this application embodiment include: acquiring laser point cloud data and visual feature data of a target area, fusing the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area; responding to an acquired POS task, performing path planning based on the composite semantic map to obtain a planned path from an initial position to a target position; acquiring the current environmental information of the multimodal mobile POS robot, controlling the multimodal mobile POS robot to switch from a first movement mode to a second movement mode according to the current environmental information, and moving towards the target position along the planned path in the second movement mode. The environmental information includes: obstacle information, ground information, and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode. This application realizes intelligent switching between the wheeled and legged modes of the multimodal mobile POS service robot, improves the environmental adaptability of the multimodal mobile POS service robot, and effectively improves the operational efficiency in commercial service scenarios.

[0014] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart of a motion control method provided in an embodiment of this application is shown; Figure 2 A schematic diagram of the structure of the multimodal mobile checkout robot provided in an embodiment of this application is shown; Figure 3 A schematic diagram of the structure of a mobile control system provided in an embodiment of this application is shown.

[0017] Explanation of key component symbols: 200 - Multimodal mobile POS robot; 210 - Legged upper body; 220 - Wheeled chassis; 300 - Mobile control system; 310 - Map building module; 320 - Path planning module; 330 - Control module. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0019] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the template description is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] Example 1 With the rapid development of unmanned retail and smart stores, mobile checkout robots are beginning to be used in shopping malls, supermarkets, and other scenarios to provide flexible payment services and alleviate checkout pressure during peak hours. Existing mobile checkout robots mostly use a single wheeled chassis or legged structure, which cannot adaptively adjust their movement according to the actual environment and task requirements to adapt to complex offline commercial scenarios. For more information, please refer to [link / reference needed]. Figure 1 This application provides a motion control method, applied to, for example... Figure 2 The multimodal mobile checkout robot 200 shown includes a legged upper body 210 and a wheeled chassis 220. The legged upper body 210 and the wheeled chassis 220 are detachably connected. The method includes steps S110 to S130.

[0022] Step S110: Obtain laser point cloud data and visual feature data of the target area, and fuse the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area.

[0023] In this embodiment, the multimodal mobile checkout robot 200 is equipped with a LiDAR and a depth camera. The LiDAR performs a comprehensive scan of the target area (such as a supermarket or shopping mall) to collect high-precision 2D or 3D point cloud data, outlining the contours and spatial distances of objects such as walls, shelves, and pillars within the target area, ensuring the accuracy of the environmental geometry. At the same time, the depth camera acquires visual features (such as shelf edge textures, product labels, and aisle signs) and color information within the target area, enriching the visual dimension data of the environment.

[0024] Subsequently, the laser point cloud data and visual feature data are correlated and fused. For example, the irregular contours identified by the LiDAR are associated with the texture and markings of the stair steps captured by the depth camera, giving the geometric contours the semantic attribute of "stairs," transforming the data from a simple set of spatial coordinates into information with scene meaning. Finally, based on the fused semantic data, an occupancy grid map with spatial accessibility attributes (each cell is marked as "free," "occupied," or "unknown") and a semantic map annotating key scenes (such as "fresh produce aisle," "cashier counter," "obstacle zone," etc.) are constructed, and the two types of maps are further merged to form a composite semantic map.

[0025] It is understandable that this composite semantic map not only retains high-precision spatial positioning capabilities, but also achieves an understanding of the environment and scene through visual semantic supplementation. This avoids the limitations of single sensor data, enabling multimodal mobile robots to not only perceive space, but also understand the scene. This provides data support for avoiding obstacles and prioritizing reasonable channels during subsequent path planning, while also providing semantic basis for morphological switching decisions during movement.

[0026] Step S120: In response to the obtained cashier task, perform path planning based on the composite semantic map to obtain a planned path from the initial position to the target position.

[0027] In this embodiment, in response to a checkout task issued from a cloud management platform or a local system, the target location, such as 3 meters from aisle A in shelf area B2, is parsed from the checkout task. Based on a composite semantic map, a path planning algorithm is invoked to determine the planned path from the current initial location to the target location. It should be noted that the planned path needs to consider the spatial accessibility attributes of the occupied grid map, automatically avoiding occupied obstacle areas and unknown areas, and prioritizing vacant aisles. Simultaneously, it references key scene information marked on the semantic map to avoid non-priority access areas such as densely populated areas and temporary construction zones.

[0028] Understandably, path planning based on composite semantic maps ensures both spatial feasibility and scenario rationality, improves mobility, reduces unnecessary detours and midway stops, and provides a guarantee for rapid response to checkout tasks and shortens customer waiting time.

[0029] Step S130: Obtain the current environmental information of the multimodal mobile checkout robot 200, and control the multimodal mobile checkout robot 200 to switch from a first movement mode to a second movement mode according to the current environmental information, and move towards the target location along the planned path in the second movement mode. The environmental information includes: obstacle information, ground information and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode.

[0030] In this embodiment, during the movement along the planned path towards the target location, multimodal perception components such as LiDAR, depth camera, ultrasonic sensor, and anti-collision contact edge continuously collect information on obstacles in the current environment (including the distribution and size parameters of obstacles), ground information (including ground types such as flat ground, steps, slopes, uneven ground, etc.), and passable space information (including passable width and height, etc.). The current environmental information is evaluated according to preset judgment rules: if the current environment is a flat ground with no continuous obstacles and the passable width is greater than a preset width threshold, it is determined that wheeled movement is suitable. If the multimodal mobile POS robot 200 is currently in the first movement mode (legged mode), the docking mechanism is locked, the wheeled movement module is activated, and the robot switches to the second movement mode (wheeled mode) to move in a high-speed, low-power mode.

[0031] If continuously distributed obstacles are detected, the obstacle size is smaller than a preset size threshold, the ground type is steps, slope, or uneven terrain, or the passable width is smaller than a preset width threshold, then it is determined that the legged mode is suitable for movement. If the multimodal mobile POS robot 200 is currently in the first movement mode (wheeled mode), then the wheeled module is stopped and the legged movement module is started. Through the coordinated movement of the multi-degree-of-freedom leg joints, it can lift its legs to overcome obstacles, climb slopes, descend, or pass through narrow passages by turning sideways. After switching to the second movement mode (legged mode), it continues to move along the planned path.

[0032] It is understandable that by dynamically perceiving and adaptively switching forms based on real-time environmental information, the advantages of wheeled form in high speed and low energy consumption in flat areas are leveraged, while the strong terrain adaptability of legged form overcomes obstacles that traditional wheeled robots cannot handle, thus avoiding the mobility limitations of single-form robots. At the same time, form switching is based on environmental data decisions, ensuring the rationality of the switching timing and the stability of the movement process, effectively reducing the ineffective consumption of robot stopping and detouring midway, and greatly improving movement efficiency and task completion rate.

[0033] In one embodiment, the obstacle information includes: obstacle distribution and obstacle size; the ground information includes: ground type; the passable space information includes: passable width; and controlling the multimodal mobile checkout robot 200 to switch from a first movement mode to a second movement mode based on the current environment information includes: If the obstacle distribution is continuous, or the obstacle size is smaller than a preset size threshold, or the ground type is a preset ground type, or the passable width is smaller than a preset width threshold, then the multimodal mobile cash register robot 200 is controlled to switch from the wheeled form to the legged form. If the obstacle distribution is obstacle-free, the ground type is not the preset ground type, and the passable width is greater than or equal to the preset width threshold, then the multimodal mobile POS robot 200 is controlled to switch from the legged form to the wheeled form.

[0034] In this embodiment, four key parameters—obstacle distribution, obstacle size, ground type, and passable width—are extracted from the real-time collected environmental information and used as the basis for determining the mode switching. The preset size threshold defines low obstacles that require leg movement to overcome. The preset ground type includes non-flat terrain such as steps, slopes, and uneven surfaces. The preset width threshold distinguishes between wide and narrow passages. When any one of the following four conditions is met—continuous obstacle distribution, obstacle size smaller than the preset size threshold, ground type being a preset ground type, or passable width being less than the preset width threshold—it indicates that the current environment is not suitable for efficient wheeled movement. If the multimodal mobile POS robot 200 is currently in wheeled mode, it is controlled to switch to legged mode, coordinating the multi-degree-of-freedom leg joints to enter the working state. When all three conditions are met—no obstacles, ground type not being a preset ground type, and passable width greater than or equal to the preset width threshold—it indicates that the current environment is a flat and open area, suitable for wheeled movement. If the multimodal mobile POS robot 200 is currently in legged mode, it is controlled to switch back to wheeled mode.

[0035] Understandably, the targeted application of wheeled and legged forms maximizes the advantages of high speed and low energy consumption in flat areas, while the legged form effectively overcomes the limitations of wheeled forms in scenarios such as low, continuous obstacles, complex terrain, and narrow passages, which wheeled forms cannot handle. This solves the environmental adaptability deficiencies of single-form robots. It ensures the smoothness and continuity of the robot's movement and improves path travel efficiency.

[0036] In one embodiment, controlling the multimodal mobile checkout robot 200 to move along the planned path towards the target location in the second movement pattern includes: If the second movement mode is a foot-type mode, then after the foot-type upper body 210 and the wheel-type chassis 220 are separated, the foot-type upper body 210 is controlled to move towards the target position along the planned path; If the second movement mode is a wheeled mode, then after the footed upper body 210 and the wheeled chassis 220 are connected, the wheeled chassis 220 and the footed upper body 210 are controlled to move together along the planned path to the target position.

[0037] In this embodiment, when it is determined to switch from wheeled mode to footed mode, the footed upper body 210 is controlled to separate from the wheeled chassis 220. During the separation process, the footed upper body 210 is kept horizontally stable by fine adjustment of the foot joints. After the separation is completed, the movement trajectory of the multi-degree-of-freedom leg and foot joints is coordinated according to the planned path and the current environmental information (such as the position of obstacles and the width of the passage), and the footed upper body 210 is controlled to move independently along the planned path to the target position. During the movement, the gait is continuously adjusted through sensor feedback to ensure accurate movement direction.

[0038] When switching from legged to wheeled mode, the relative positions of the legged upper body 210 and the wheeled chassis 220 are first confirmed. After controlling the movement of both to the preset docking position, a locking command is sent to the docking mechanism to complete the stable connection between the legged upper body 210 and the wheeled chassis 220. After the connection is completed, by controlling the rotation speed and steering of the wheel set, the wheeled chassis 220 and the legged upper body 210 are driven to move together along the planned path to the target position at high speed and smoothly. At the same time, the movement posture is monitored in real time by sensors, and the deviation is corrected in time.

[0039] It is understandable that by separating or connecting the legged upper body 210 and the wheeled chassis 220, the upper body in legged form can flexibly cross obstacles that the wheeled chassis 220 cannot pass through (such as stairs and narrow passages) without being restricted by the chassis. In wheeled form, the integrated connection ensures the overall stability during high-speed movement and avoids shaking or deviation caused by the separation of parts. This fully leverages the core advantages of both forms and solves the motion coordination problem after switching between the two forms through precise control of the connection state. It ensures that the robot can move efficiently along the planned path in different environments, improving the mobility reliability and task completion rate in complex commercial scenarios.

[0040] In one embodiment, after the control of the foot-type upper body 210 and the wheel-type chassis 220 is separated, the method further includes: The obstacle passage path is determined based on the composite semantic map and the current environmental information of the multimodal mobile checkout robot 200; The wheeled chassis 220 is controlled to travel along the obstacle path, and after passing the obstacle, the wheeled chassis 220 is controlled to connect with the footed upper body 210.

[0041] After the control foot-mounted upper body 210 is separated from the wheeled chassis 220, it is necessary to further call the constructed composite semantic map and combine it with the real-time collected current environmental information (including the type, size, distribution of obstacles and the surrounding passable space) to carry out special planning of the passage path of the wheeled chassis 220: First, extract the semantic attributes of obstacles (such as "unavoidable stairs" and "scattered goods that can be bypassed") and the distribution of surrounding empty areas from the composite semantic map. Then, combine the passable width, ground type and other parameters in the current environmental information to eliminate areas that the wheeled chassis 220 cannot pass through and determine an obstacle passage path that avoids obstacles and has the shortest distance (such as a side passage to bypass obstacles or a passage route using a preset ramp). Subsequently, a path command is sent to the wheeled chassis 220 to control the wheeled chassis 220 to move independently along the obstacle passage path. During the movement, the chassis's built-in sensors provide real-time feedback of position information. At the same time, the movement status of the legged upper body 210 is continuously tracked. When it is detected that the wheeled chassis 220 has passed through the obstacle and arrived at the preset rendezvous area (a safe docking point marked based on a composite semantic map), and the legged upper body 210 has also crossed the obstacle and arrived at the area, the wheeled chassis 220 is controlled to adjust its posture to the docking ready state. Then, the docking mechanism is activated to complete the stable connection between the wheeled chassis 220 and the legged upper body 210, restoring the integrated movement mode.

[0042] Understandably, this approach solves the problem of the chassis failing to pass synchronously after the legged upper body 210 crosses an obstacle, ensuring that robot components do not separate and functions are not lost, and preventing subsequent tasks from being unable to proceed normally due to chassis stagnation. Through specialized path planning based on composite semantic maps, the wheeled chassis 220's passage is made more precise and efficient, reducing unnecessary movement. The mode of separate passage and precise docking after passage not only retains the flexibility of the legged form in overcoming obstacles, but also leverages the mobility advantage of the wheeled chassis 220 in flat areas, further improving the robot's environmental adaptability and the continuity of task completion in complex obstacle scenarios.

[0043] In one embodiment, the method further includes: obtaining the current position of the multimodal mobile POS robot 200; determining whether the distance between the current position and the target position is less than a preset distance threshold; if so, obtaining the interaction requirements of the target user; determining the target interaction posture based on the interaction requirements; and controlling the multimodal mobile POS robot 200 to adjust to the target interaction posture.

[0044] In this embodiment, during the movement along the planned path towards the target location, the multimodal mobile checkout robot 200 continuously acquires its current position and calculates the distance between the current position and the target location in real time. The calculated distance is then compared with a preset distance threshold to determine whether the multimodal mobile checkout robot 200 has reached the target location. When the distance between the current position and the target location is less than the preset distance threshold, the multimodal mobile checkout robot 200 activates its depth camera and microphone array. The depth camera captures information such as the target user's height, ground slope, and / or product height, while the microphone array receives the user's voice requests (e.g., "lower screen height"). The robot integrates visual and voice information to extract the target user's interaction requests (including height adaptation requests, product scanning position adaptation requests, etc.). Next, based on these interaction requests and preset posture adaptation rules, the corresponding target interaction posture is determined (e.g., interaction height for children, interaction height for wheelchair users, forward-leaning posture for high-positioned products, etc.). Finally, through the fine adjustment of the multi-degree-of-freedom leg joints, the multimodal mobile POS robot 200 is adjusted to the target interactive posture. During the adjustment process, the posture data is fed back in real time by sensors to ensure accurate adjustment and stable body.

[0045] Understandably, this approach enables the multimodal mobile checkout robot 200 to anticipate interaction scenarios and proactively adapt to user needs, avoiding the inconvenience of interaction inherent in traditional fixed-posture robots and significantly improving the comfort and convenience of the user experience.

[0046] In one embodiment, the interaction requirements include: user height, ground slope, and / or product height. Obtaining the interaction requirements of the target user includes: obtaining a local area image of the target location, and determining the user height, the ground slope, and / or the product height based on the local area image.

[0047] In this embodiment, when the multimodal mobile checkout robot 200 determines that the distance between its current position and the target position is less than a preset distance threshold, it acquires an image of the local area where the target position is located using a depth camera, obtaining a clear local area image containing the target user, the surrounding ground, and related goods. Subsequently, the robot's image processing unit performs multi-dimensional analysis on the local area image: by extracting the human contour features of the target user in the image, combined with the distance data from the depth camera, the user's height is calculated using a human proportion algorithm; by analyzing the pixel grayscale changes and spatial coordinate distribution of the ground area in the image, the ground slope is derived using a slope calculation model; by identifying the shelf labels of the goods in the image or the positional relationship of the goods relative to reference objects (such as the user's hand or the edge of the shelf), combined with a preset shelf size parameter library, the height of the goods is calculated; if the image simultaneously contains information about the user, the ground, and the goods, the comprehensive determination of the user's height, the ground slope, and the height of the goods is completed simultaneously, ultimately obtaining the target user's interaction requirements.

[0048] It is understandable that using image analysis to obtain interaction needs avoids errors caused by manual input or voice misrecognition, improving the efficiency and accuracy of need acquisition. At the same time, it covers key interaction dimensions such as user height, ground slope, and product height, ensuring that the posture adjustment of the multimodal mobile POS robot 200 can take into account user operation convenience, machine stability, and product interaction adaptability. This solves the problem of interaction limitations caused by traditional robots relying on adjustment of only a single parameter, and provides personalized and humanized POS interaction services.

[0049] In one embodiment, after controlling the multimodal mobile POS robot 200 to adjust to the target interactive posture, the method further includes: issuing preset guidance information to prompt the target user to make payment according to the preset guidance information; determining whether payment information has been obtained, and if so, controlling the multimodal mobile POS robot 200 to return to the initial position.

[0050] In this embodiment, after the multimodal mobile checkout robot 200 is adjusted to the target interaction posture, it calls the preset guidance information database and matches the corresponding voice guidance content and screen display information (e.g., lively voice and large icon text prompts for children, and concise voice and standard operation instructions for ordinary users) according to the previously obtained target user interaction needs (such as user type corresponding to user height, product height, etc.) through the speaker. Then, the voice guidance information is emitted through the speaker, and at the same time, visual guidance information such as the scanning area and payment method selection interface are displayed on the screen to jointly prompt the target user to complete the payment operation according to the instructions.

[0051] During this process, the system continuously monitors whether it has acquired the user's payment information (such as QR code verification information for QR code payments, financial data from card swipes, and facial biometric verification information). Once valid payment information is detected and secure encryption verification is completed, the payment is deemed complete. Subsequently, based on a composite semantic map, a return path is planned from the current location back to the initial location. The multimodal mobile POS robot 200 is controlled to adaptively switch its movement mode according to environmental information along the return path, autonomously returning to the initial location (such as a charging station or waiting area) along the planned return path, awaiting the next POS task.

[0052] Understandably, the matching of preset guidance information with user needs improves the convenience of payment operations, lowers the user's operating threshold, and reduces payment time; the automatic return scheduling after payment is completed allows the robot to be on standby in a loop without human intervention, improving efficiency and scheduling flexibility, and ensuring the continuity and efficiency of the cashier service.

[0053] The mobile control method provided in this application embodiment is applied to a multimodal mobile checkout robot 200. The multimodal mobile checkout robot 200 includes a legged upper body 210 and a wheeled chassis 220. The legged upper body 210 and the wheeled chassis 220 are detachably connected. By acquiring laser point cloud data and visual feature data of a target area, and fusing the laser point cloud data and the visual feature data, a composite semantic map of the target area is obtained. In response to the acquired checkout task, path planning is performed based on the composite semantic map to obtain a planned path from the initial position to the target position. The current environmental information of the multimodal mobile checkout robot 200 is acquired. Based on the current environmental information, the multimodal mobile POS robot 200 is controlled to switch from a first movement mode to a second movement mode, and moves towards the target location along the planned path in the second movement mode. The environmental information includes obstacle information, ground information, and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode. This realizes the intelligent switching between the wheeled and legged modes of the multimodal mobile POS service robot, improves the environmental adaptability of the multimodal mobile POS service robot, and effectively improves the operational efficiency in commercial service scenarios.

[0054] Example 2 In addition, please see Figure 3 This application also provides a mobile control system 300, including: The map building module 310 is used to acquire laser point cloud data and visual feature data of the target area, and fuse the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area. The path planning module 320 is used to respond to the obtained cashier task and perform path planning based on the composite semantic map to obtain a planned path from the initial position to the target position. The control module 330 is used to acquire the current environmental information of the multimodal mobile checkout robot 200, and control the multimodal mobile checkout robot 200 to switch from a first movement mode to a second movement mode according to the current environmental information, and move towards the target location along the planned path in the second movement mode. The environmental information includes: obstacle information, ground information and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode.

[0055] The mobile control system 300 provided in this application embodiment can execute the mobile control method provided in the above-described method embodiment 1. To avoid repetition, it will not be described again here.

[0056] Example 3 In addition, please see again Figure 2 This application provides a multimodal mobile checkout robot 200, including: a legged upper body 210, a wheeled chassis 220, and a mobile control system 300 as described in Embodiment 2. The legged upper body 210 is detachably connected to the wheeled chassis 220, and the mobile control system 300 is communicatively connected to the legged upper body 210 and the wheeled chassis 220 respectively.

[0057] The multimodal mobile POS robot 200 provided in this application embodiment can perform the functions of the mobile control system 300 provided in embodiment 2 above. To avoid repetition, it will not be described again here.

[0058] Example 4 Furthermore, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the motion control method provided in Embodiment 1.

[0059] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0060] The computer-readable storage medium provided in this embodiment can implement the motion control method provided in Embodiment 1. To avoid repetition, it will not be described again here.

[0061] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0062] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0063] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A motion control method, characterized in that, An application is made to a multimodal mobile checkout robot, the multimodal mobile checkout robot comprising: a legged upper body and a wheeled chassis, the legged upper body being detachably connected to the wheeled chassis, the method comprising: Acquire laser point cloud data and visual feature data of the target area, and fuse the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area; In response to the acquired checkout task, path planning is performed based on the composite semantic map to obtain a planned path from the initial position to the target position; The current environmental information of the multimodal mobile checkout robot is obtained, and the multimodal mobile checkout robot is controlled to switch from a first movement mode to a second movement mode according to the current environmental information. The robot then moves towards the target location along the planned path in the second movement mode. The environmental information includes obstacle information, ground information, and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode.

2. The motion control method according to claim 1, characterized in that, The method further includes: Obtain the current position of the multimodal mobile checkout robot; Determine whether the distance between the current position and the target position is less than a preset distance threshold. If so, obtain the interaction requirements of the target user and determine the target interaction posture based on the interaction requirements. Control the multimodal mobile checkout robot to adjust to the target interactive posture.

3. The motion control method according to claim 2, characterized in that, The interaction requirements include: user height, ground slope, and / or product height. Obtaining the target user's interaction requirements includes: Obtain a local area image of the target location, and determine the user's height, the ground slope, and / or the product height based on the local area image.

4. The motion control method according to claim 3, characterized in that, After controlling the multimodal mobile checkout robot to adjust to the target interactive posture, the method further includes: Send a preset guidance message to prompt the target user to make a payment based on the preset guidance message; If payment information has been obtained, the multimodal mobile POS robot is controlled to return to the initial position.

5. The motion control method according to claim 1, characterized in that, The obstacle information includes: obstacle distribution and obstacle size; the ground information includes: ground type; the passable space information includes: passable width; and controlling the multimodal mobile checkout robot to switch from a first movement mode to a second movement mode based on the current environment information includes: If the obstacle distribution is continuous, or the obstacle size is smaller than a preset size threshold, or the ground type is a preset ground type, or the passable width is smaller than a preset width threshold, then the multimodal mobile cash register robot is controlled to switch from the wheeled form to the legged form. If the obstacle distribution is obstacle-free, the ground type is not the preset ground type, and the passable width is greater than or equal to the preset width threshold, then the multimodal mobile POS robot is controlled to switch from the legged form to the wheeled form.

6. The motion control method according to claim 5, characterized in that, Controlling the multimodal mobile checkout robot to move along the planned path to the target location in the second movement pattern includes: If the second movement mode is a foot-based mode, then after the foot-based upper body and the wheeled chassis are separated, the foot-based upper body is controlled to move along the planned path to the target position. If the second movement mode is a wheeled mode, then after the footed upper body and the wheeled chassis are connected, the wheeled chassis and the footed upper body are controlled to move together along the planned path to the target position.

7. The motion control method according to claim 6, characterized in that, After the control of the separation of the foot-type upper body and the wheel-type chassis is completed, it also includes: The obstacle passage path is determined based on the composite semantic map and the current environmental information of the multimodal mobile checkout robot; Control the wheeled chassis to travel along the obstacle path, and after passing the obstacle, control the wheeled chassis to connect with the footed upper body.

8. A mobile control system, characterized in that, The system includes: The map building module is used to acquire laser point cloud data and visual feature data of the target area, and fuse the laser point cloud data and the visual feature data to obtain a composite semantic map of the target area. The path planning module is used to respond to the obtained checkout task and perform path planning based on the composite semantic map to obtain the planned path from the initial position to the target position. The control module is used to acquire the current environmental information of the multimodal mobile checkout robot, and control the multimodal mobile checkout robot to switch from a first movement mode to a second movement mode according to the current environmental information, and move towards the target location along the planned path in the second movement mode. The environmental information includes: obstacle information, ground information and passable space information. The first movement mode is a legged mode and the second movement mode is a wheeled mode, or the first movement mode is a wheeled mode and the second movement mode is a legged mode.

9. A multimodal mobile checkout robot, characterized in that, include: The system comprises a legged upper body, a wheeled chassis, and a mobility control system as described in claim 8, wherein the legged upper body is detachably connected to the wheeled chassis, and the mobility control system is communicatively connected to both the legged upper body and the wheeled chassis.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the motion control method according to any one of claims 1-7.

Images