Control method and apparatus, information display method and apparatus, and mobile device

Abstract

The present application discloses a control method and apparatus, an information display method and apparatus, and a mobile device. The control method comprises: determining, in a global grid map, a global feasible path from a current global position of a mobile device to a target global position, the global feasible path comprising a plurality of initial path points; determining a current local position of the mobile device in a real-time local grid map in which the mobile device is located; determining, from the plurality of initial path points, unreached path points in the real-time local grid map, the unreached path points being path points not reached by the mobile device; according to the order from far to near of the distances between each unreached path point and the current local position, sequentially searching for a local feasible path from the current local position to the corresponding unreached path point; and when the local feasible path from the current local position to the corresponding unreached path point is found for the first time, controlling, on the basis of the local feasible path, the mobile device to move, and returning to executing the step of determining the current local position of the mobile device in the real-time local grid map in which the mobile device is located.

Description

Control methods, information display methods, devices and mobile devices Technical Field

[0001] This application belongs to the field of mobile device technology, and particularly relates to a control method, information display method, device and mobile device. Background Technology

[0002] Path planning refers to the process of determining a feasible path from one location to another in a specific environment to achieve a specific goal. Controlling the movement of mobile devices based on feasible paths can improve their efficiency and enhance their security. Therefore, to improve the efficiency and security of mobile devices, path planning is necessary to determine feasible paths for controlling their movement.

[0003] Summary of the Invention

[0004] This application provides a control method, information display method, device, and mobile device that can determine feasible paths to control the movement of the mobile device, thereby improving the movement efficiency and enhancing security of the mobile device.

[0005] In a first aspect, embodiments of this application provide a control method, including:

[0006] In a global grid map, a globally feasible path is determined from the current global location of the mobile device to the target global location, the globally feasible path including multiple initial path points;

[0007] Determine the current local position of the mobile device in the real-time local grid map where the mobile device is located;

[0008] Among the multiple initial waypoints, an unreachable waypoint in the real-time local grid map is determined, and the unreachable waypoint is a waypoint that the mobile device has not reached.

[0009] According to the order of distance from each unreachable path point to the current local position from farthest to closest, search for local feasible paths from the current local position to the corresponding unreachable path point.

[0010] When a locally feasible path from the current local location to the corresponding unreachable path point is found for the first time, the mobile device is controlled to move based on the locally feasible path, and the process returns to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0011] Secondly, embodiments of this application provide an information display method, including:

[0012] Identify positive and negative obstacles in the environment where the mobile device is located. The positive obstacles and the mobile device are both located on a first side of the plane where the mobile device is located, and the negative obstacles are located on a second side of the plane.

[0013] Determine the first relative display positions of the positive obstacle and the negative obstacle relative to the mobile device;

[0014] Obtain the display modes corresponding to the positive obstacles and the negative obstacles respectively;

[0015] Based on the first relative display position and the display method, the mobile device, the positive obstacle, and the negative obstacle are displayed.

[0016] Thirdly, embodiments of this application provide a control device, including:

[0017] The path determination module is used to determine a globally feasible path from the current global location of the mobile device to the target global location in the global grid map, wherein the globally feasible path includes multiple initial path points;

[0018] The location determination module is used to determine the current local location of the mobile device in the real-time local grid map where the mobile device is located;

[0019] A waypoint determination module is used to determine, from among a plurality of initial waypoints, an unreachable waypoint in the real-time local grid map, wherein the unreachable waypoint is a waypoint that the mobile device has not reached.

[0020] The path search module is used to sequentially search for local feasible paths from the current local location to the corresponding unreachable path point in order of distance from the current local location to the current local location, from farthest to closest.

[0021] The device control module is used to control the mobile device to move based on the local feasible path when a local feasible path from the current local location to the corresponding unreachable path point is found for the first time, and return to execute the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0022] Fourthly, embodiments of this application provide an information display device, comprising:

[0023] An obstacle recognition module is used to identify positive and negative obstacles in the environment where the mobile device is located. The positive obstacles and the mobile device are both located on the first side of the plane where the mobile device is located, and the negative obstacles are located on the second side of the plane.

[0024] A position determination module is used to determine the first relative display positions of the positive obstacle and the negative obstacle relative to the mobile device;

[0025] The mode acquisition module is used to acquire the display modes corresponding to the positive obstacles and the negative obstacles, respectively.

[0026] The information display module is used to display the mobile device, the positive obstacle, and the negative obstacle according to the first relative display position and the display mode.

[0027] Fifthly, embodiments of this application also provide a mobile device, including a memory and a processor, wherein the processor executes the control method provided in embodiments of this application by calling a computer program stored in the memory.

[0028] Sixthly, embodiments of this application also provide a mobile device, including a memory and a processor, wherein the processor executes the information display method provided in embodiments of this application by calling a computer program stored in the memory.

[0029] In this embodiment of the application, a globally feasible path from the current global location of the mobile device to the target global location is determined in a global grid map. This globally feasible path includes multiple initial path points. The current local location of the mobile device in the real-time local grid map where the mobile device is located is determined. Among the multiple initial path points, unreachable path points in the real-time local grid map are determined. These unreachable path points are path points that the mobile device has not yet reached. Local feasible paths from the current local location to the corresponding unreachable path point are searched sequentially according to the order of distance between each unreachable path point and the current local location from farthest to closest. When the first... When a locally feasible path is found from the current local location to the corresponding unreachable path point, the mobile device is controlled to move based on the locally feasible path, and the process returns to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located. This can be achieved by first determining a globally feasible path in the global grid map, and then determining a locally feasible path in the real-time local grid map where the mobile device is located based on the globally feasible path, and controlling the movement of the mobile device based on the locally feasible path. This takes into account the real-time environment in which the mobile device is currently located, such as newly appearing obstacles, thereby improving the movement efficiency of the mobile device and enhancing its safety. Attached Figure Description

[0030] The technical solution and its beneficial effects will become apparent from the following detailed description of specific embodiments of this application, in conjunction with the accompanying drawings.

[0031] Figure 1 is a flowchart illustrating the control method provided in an embodiment of this application.

[0032] Figure 2 is a schematic diagram of the location selection interface provided in an embodiment of this application.

[0033] Figure 3 is a schematic diagram of the globally feasible path provided in the embodiments of this application.

[0034] Figure 4 is a schematic diagram of the first scenario of the control method provided in the embodiments of this application.

[0035] Figure 5 is a schematic diagram of a real-time local raster map provided in an embodiment of this application.

[0036] Figure 6 is a schematic diagram of a second scenario of the control method provided in the embodiments of this application.

[0037] Figure 7 is a schematic diagram of a third scenario of the control method provided in the embodiments of this application.

[0038] Figure 8 is a schematic diagram of the fourth scenario of the control method provided in the embodiments of this application.

[0039] Figure 9 is a schematic diagram of the fifth scenario of the control method provided in the embodiments of this application.

[0040] Figure 10 is a schematic diagram of the sixth scenario of the control method provided in the embodiments of this application.

[0041] Figure 11 is a flowchart illustrating the coarse planning process of feasible paths provided in the embodiments of this application.

[0042] Figure 12 is a flowchart illustrating the detailed planning process of a feasible path provided in an embodiment of this application.

[0043] Figure 13 is a flowchart illustrating the information display method provided in an embodiment of this application.

[0044] Figure 14 is a schematic diagram of the first scenario of the information display method provided in the embodiments of this application.

[0045] Figure 15 is a schematic diagram of a second scenario of the information display method provided in the embodiments of this application.

[0046] Figure 16 is a schematic diagram of a third scenario of the information display method provided in the embodiments of this application.

[0047] Figure 17 is a schematic diagram of the instantaneous arc of the sector in the sector animation provided in the embodiment of this application, where the arc is π / 2.

[0048] Figure 18 is a schematic diagram of the instantaneous arc of the sector in the sector animation provided in the embodiment of this application, where the arc is π.

[0049] Figure 19 is a schematic diagram of the instantaneous arc of the sector in the sector animation provided in the embodiment of this application, where the arc is 2π.

[0050] Figure 20 is a schematic diagram of the structure of the control device provided in the embodiment of this application.

[0051] Figure 21 is a schematic diagram of the structure of the information display device provided in the embodiment of this application.

[0052] Figure 22 is a schematic diagram of the structure of the mobile device provided in an embodiment of this application.

[0053] Figure 23 is a schematic diagram of the structure of another mobile device provided in an embodiment of this application. Detailed Implementation

[0054] Please refer to the illustrations, where the same component symbols represent the same components. The principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application illustrated, and should not be construed as limiting other specific embodiments not detailed herein.

[0055] This application provides a control method, a control device, an information display method, an information display device, and a mobile device. The control method can be executed by the control device provided in this application, or by a mobile device integrating the control device. The control device can be implemented in hardware or software. Similarly, the information display method can be executed by the information display device provided in this application, or by a mobile device integrating the information display device. The information display device can be implemented in hardware or software. The mobile device includes vehicles such as wheelchairs or cars, or robots such as robotic vacuum cleaners or industrial robots, which are equipped with processors and have control or information display capabilities.

[0056] Please refer to Figure 1, which is a flowchart illustrating the control method provided in an embodiment of this application. The process may include:

[0057] In 101, a globally feasible path is determined in the global grid map from the current global location of the mobile device to the target global location. The globally feasible path includes multiple initial waypoints.

[0058] Mobile devices include vehicles such as wheelchairs or cars, robots such as robot vacuum cleaners or industrial robots, or drones.

[0059] Raster maps, such as global raster maps or real-time local raster maps, can be two-dimensional raster maps, three-dimensional raster maps, etc.

[0060] Two-dimensional grid maps, also known as occupancy grid maps, describe the environment as a two-dimensional plane divided into a series of square grids of the same size. Each square grid represents the probability of an obstacle being present in its corresponding two-dimensional area. This type of map is generally used to represent structured environments, and its focus on environmental information is only on whether an obstacle exists at a certain location on the plane where the mobile device is located, without concern for whether there are depressions in front or the specific height of obstacles in front.

[0061] A 3D raster map divides the 3D environment into a series of identically sized cubes, each cube representing a 3D region within the environment and indicating the probability of an obstacle being present in that region. This type of map can comprehensively represent the surrounding environment and is typically visualized as a 3D dense point cloud.

[0062] Compared to two-dimensional raster maps, 2.5D elevation maps store elevation values ​​for corresponding locations in their rasters instead of occupancy probability data.

[0063] In one possible implementation, the mobile device is moved in advance in the area where a global raster map needs to be generated, and the area that the mobile device has moved through is sampled during the movement of the mobile device to generate the global raster map.

[0064] Optionally, to reduce the interference of dynamic obstacles on the generated global grid map, dynamic obstacles in the generated global grid map can be removed during the sampling process of the mobile device to form a global grid map without dynamic obstacles.

[0065] In one possible implementation, after the global grid map is formed, it can be stored in a mobile device for the determination of globally feasible paths, thus enabling the global grid map to be formed once and used multiple times.

[0066] Optionally, the global grid map stored in the removable device can be updated periodically to improve the accuracy of the global grid map, thereby improving the efficiency of global feasible path determination.

[0067] In an optional embodiment, GPS signals, INS data, and sensor fusion technology can be combined to locate the current position of the mobile device, which is then mapped onto a global grid map to determine the current global grid in which the mobile device is located, thus obtaining the current global position of the mobile device.

[0068] In another alternative embodiment, environmental data surrounding the mobile device can be collected using cameras and radar sensors and matched with a global grid map to determine the current global grid in which the mobile device is located, thereby obtaining the current global location of the mobile device.

[0069] In one possible implementation, the destination can be input by the user of the mobile device via a touch screen or by voice, and then the target global grid in the global grid map can be determined to obtain the global location of the target.

[0070] For example, as shown in Figure 2, the display screen configured on the mobile device can display a location selection interface, which can display locations 1 to 4 for the user of the mobile device to select. The user of the mobile device can select any one of locations 1 to 4 as the destination by tapping on the location selection interface with their finger.

[0071] In some embodiments, when determining a globally feasible path from the current global location of a mobile device to a target global location in a global grid map, multiple candidate path points are determined in the global grid map, where candidate path points are grids in an idle state in the global grid map; the connectivity relationship between each candidate path point and candidate path points within a first preset range around it is determined; among the multiple candidate path points, a starting path point closest to the current global location and a target path point closest to the target global location are determined; based on the connectivity relationship, intermediate path points that form a feasible path from the starting path point to the target path point are determined from the multiple candidate path points; the starting path point, the target path point, and the intermediate path points are determined as multiple initial path points, and the feasible path composed of the multiple initial path points is determined as a globally feasible path from the current global location to the target global location.

[0072] Specifically, when the global grid map is a two-dimensional grid map or a three-dimensional grid map, if the probability of storing a grid in the global grid map is not less than a certain occupancy threshold, the grid is considered to be in an occupied state; if the probability is less than the occupancy threshold, the grid is considered to be in an idle state.

[0073] When the global raster map is a 2.5D elevation map, if the elevation value stored in the raster in the global raster map is not less than a certain specific threshold, the raster is considered to be in an occupied state; if the elevation value is less than the threshold, the raster is considered to be in an idle state.

[0074] In some embodiments, when determining multiple candidate path points in a global grid map, the global grid map is divided into multiple grid regions; idle grids are randomly sampled in each grid region as candidate path points to obtain multiple candidate path points.

[0075] Each grid region can be a rectangular area. The size of each grid region can be preset by technicians. For example, if each grid region is 1 meter × 1 meter, and the global grid map is 1000 meters × 1000 meters, then the global grid map includes 1,000,000 grid regions. Each grid region includes multiple grids. For example, if each grid is 0.1 meters × 0.1 meters, then each grid region includes 100 grids.

[0076] After dividing the global raster map into multiple raster regions, random sampling is performed within each raster region. If the sampled raster is an empty raster, the sampling is considered successful, and the empty raster is used as a candidate path point. If the sampled raster is an occupied raster, the sampling is considered unsuccessful, and random sampling continues until a successful sampling is achieved. The sampled empty raster is then used as a candidate path point. Optionally, each raster region may include only one candidate path point. Here, an empty raster is a raster in the global raster map that is currently idle, and an occupied raster is a raster in the global raster map that is currently occupied.

[0077] Understandably, in practical applications, mobile devices are usually not points but have a certain physical size. If the global grid map is not inflated, the planned path may be too close to obstacles, causing the mobile device to be unable to pass or to collide with obstacles during actual movement. Therefore, in some embodiments, when dividing the global grid map into multiple grid regions, the global grid map is first inflated to obtain an inflated grid map; then the inflated grid map is divided into multiple grid regions. By inflating the grid map, the size of the mobile device can be taken into account, ensuring that there is enough space around the planned globally feasible path for the mobile device to move safely.

[0078] The expansion radius can be preset by technicians. For example, the expansion radius can be half the width of the mobile device.

[0079] The first preset range can be set in advance by technicians. For example, the first preset range is the range centered on each candidate path point and within which the actual distance to each candidate path point is 2 meters. When the actual distance between one candidate path point and another candidate path point is not greater than 2 meters, the candidate path point is considered to be a candidate path point within the first preset range around the other candidate path point. Therefore, it can be determined whether a candidate path point is within the first preset range around another candidate path point by calculating the actual distance between the two candidate path points.

[0080] In one possible implementation, taking a two-dimensional global grid map as an example, the actual distance between two candidate path points is equal to the distance between the global grid coordinates of the two candidate path points in the Cartesian coordinate system corresponding to the global grid map, multiplied by the size of the grid in the global grid map.

[0081] Optionally, in this embodiment, the distance between the two coordinates can be Euclidean distance, Hammanton distance, or Chebyshev distance.

[0082] For example, suppose the global grid coordinates of one candidate path point P11 are (2,3) and the global grid coordinates of another candidate path point P12 are (5,7), then the distance between these two candidate path points in the Cartesian coordinate system corresponding to the global grid map is 5.

[0083] Assuming each grid cell in the global raster map is 0.1m x 0.1m, the actual distance between candidate path point P11 and candidate path point P12 is 0.5m. Assuming each grid cell in the global raster map is 0.2m x 0.2m, the actual distance between candidate path point P11 and candidate path point P12 is 1m.

[0084] In some embodiments, when determining the connectivity between each candidate path point and candidate path points within a first preset range around it, a first preset algorithm is used to determine the connectivity between each candidate path point and candidate path points within a first preset range around it.

[0085] The first preset algorithm can be the A* algorithm. The A* algorithm, also known as the A-star algorithm, is a path search algorithm used to search for feasible paths from one path point to another.

[0086] The connectivity between two candidate path points can be either connected or disconnected. If a feasible path from one candidate path point to the other is found using the A* algorithm, then the two candidate path points are considered connected; if no feasible path is found using the A* algorithm, then the two candidate path points are considered disconnected.

[0087] Taking the A* algorithm as an example to determine whether candidate path point P11 is connected to candidate path point P12 within a first preset range, the A* algorithm can be used to search for a feasible path from candidate path point P11 to candidate path point P12. If a feasible path from candidate path point P11 to candidate path point P12 is found, then the connection between candidate path point P11 and candidate path point P12 is determined to be: connected; if no feasible path from candidate path point P11 to candidate path point P12 is found, then the connection between candidate path point P11 and candidate path point P12 is determined to be: not connected.

[0088] Among them, feasible paths, such as globally feasible paths or locally feasible paths, refer to the routes that the mobile device can actually travel, as determined by the mobile device.

[0089] The globally feasible path consists of a series of initial path points between the current global location and the target global location.

[0090] In some embodiments, when determining the starting path point closest to the current global position and the target path point closest to the target global position among multiple candidate path points, a second preset algorithm is used to determine the starting path point closest to the current global position and the target path point closest to the target global position among multiple candidate path points; when determining intermediate path points that form a feasible path from the starting path point to the target path point among multiple candidate path points according to connectivity, the second preset algorithm is used to determine intermediate path points that form a feasible path from the starting path point to the target path point according to connectivity among multiple candidate path points.

[0091] The second preset algorithm can be Dijkstra's algorithm. Dijkstra's algorithm is also a path search algorithm used to find the shortest feasible path from one path point to other path points.

[0092] Specifically, the Dijkstra algorithm searches from multiple candidate path points for the starting path point closest to the current global location of the mobile device and the target path point closest to the target global location. Based on this connectivity, the Dijkstra algorithm then determines intermediate path points from the multiple candidate path points to form a feasible path from the starting path point to the target path point. The starting path point, intermediate path points, and target path point are then identified as multiple initial path points, and the feasible path formed by these initial path points is determined as the globally feasible path from the current global location to the target global location.

[0093] For example, as shown in Figure 3, the globally feasible path is a feasible path composed of the starting path point P1, the intermediate path point P5, the intermediate path point P7, the intermediate path point P11, the intermediate path point P15, and the target path point P20.

[0094] In one possible implementation, as shown in Figure 4, taking a wheelchair as an example of a mobile device, after determining a globally feasible path from the wheelchair's current global location to the target global location in the global grid map, the globally feasible path can also be displayed on a screen configured on the wheelchair to preview the globally feasible path.

[0095] Optionally, a "Start" button can also be displayed on the screen. When the wheelchair user clicks the "Start" button with their finger, the user proceeds to step 102.

[0096] Optionally, wheelchair users can also use their fingers to zoom in / out on the display screen to adjust the size of the content displayed.

[0097] Optionally, a "close" button can be displayed on the screen, which can be clicked by the wheelchair user to close the preview of the globally feasible paths.

[0098] In step 102, the current local location of the mobile device in the real-time local grid map where the mobile device is located is determined.

[0099] Specifically, a real-time local grid map centered on the current location of the mobile device can be generated first, and then the current local location of the mobile device in the real-time local grid map can be determined.

[0100] Taking the generation of a real-time local grid map of a rectangular area 8 meters × 8 meters centered on the current location of the mobile device as an example, the LiDAR on the mobile device continuously scans the surrounding environment, emits laser beams with a certain angular resolution and distance range, and receives the reflected signals to generate a local point cloud map; positioning technology is used to locate the current location of the mobile device; based on this local point cloud map, a rectangular area of ​​8 meters × 8 meters centered on the current location of the mobile device is formed as a real-time local grid map; for each grid, the probability of its storage is set based on whether the object represented by the point cloud point falling into it is an obstacle, such as a positive obstacle or a negative obstacle, to represent the probability that there is an obstacle in the two-dimensional area corresponding to that grid.

[0101] Optionally, the current location of the mobile device can be the location of the rotation center of the mobile device or the location of the center of the mobile device.

[0102] Optionally, the size of the grid cells in the real-time local grid map is the same as the size of the grid cells in the global grid map. For example, if the size of the grid cells in the global grid map is 0.1 m × 0.1 m, then the size of the grid cells in the real-time local grid map is also 0.1 m × 0.1 m.

[0103] Understandably, since the mobile device is constantly moving toward the next waypoint it has not yet reached among multiple initial waypoints, the real-time local raster map in which the mobile device is located is also updated in real time as the location of the mobile device changes.

[0104] Once the real-time local grid map is obtained, the grid in which the mobile device is located is determined, and this grid is identified as the current local location of the mobile device in the real-time local grid map.

[0105] For example, as shown in Figure 5, the grid G ​​in the real-time local grid map M is the current local position of the mobile device L in the real-time local grid map M.

[0106] In 103, unreachable path points in the real-time local grid map are identified among multiple initial waypoints. Unreachable path points are path points that mobile devices have not reached.

[0107] Taking the global raster map and the real-time local raster map as two-dimensional raster maps as an example, since the initial path points are determined based on the global raster map, the coordinates of the initial path points are the coordinates in the Cartesian coordinate system corresponding to the global raster map. Therefore, it is necessary to transform the coordinates of the initial path points to the Cartesian coordinate system corresponding to the real-time local raster map, so as to obtain the coordinates of the initial path points in the Cartesian coordinate system corresponding to the real-time local raster map, and to determine which initial path points are in the real-time local raster map and are path points that the mobile device has not reached.

[0108] Optionally, each time the mobile device reaches an initial waypoint, it records the initial waypoint as a reached waypoint. After determining the initial waypoint in the real-time local grid map, based on the information recorded by the mobile device, the reached waypoints are removed from the determined initial waypoints to obtain the unreached waypoints.

[0109] In one possible implementation, based on the information recorded by the mobile device, reached path points can be removed from multiple initial path points to obtain unreached path points; then, the coordinates of the unreached path points can be transformed to the Cartesian coordinate system corresponding to the real-time local grid map to obtain the coordinates of the unreached path points in the Cartesian coordinate system corresponding to the real-time local grid map; and finally, the unreached path points in the real-time local grid map can be determined based on the coordinates of the unreached path points in the Cartesian coordinate system corresponding to the real-time local grid map.

[0110] In step 104, local feasible paths from the current local position to the corresponding unreachable path point are searched sequentially in order of distance from each unreachable path point to the current local position, from farthest to closest.

[0111] In step 105, when a locally feasible path from the current local location to the corresponding unreachable path point is found for the first time, the movable device is controlled to move based on the locally feasible path, and the process returns to step 102.

[0112] In one possible implementation, the distance between each unreachable path point and the current local location can be determined by calculating the actual distance between each unreachable path point and the current local location. The smaller the actual distance, the closer the unreachable path point is to the current local location; conversely, the larger the actual distance, the farther the unreachable path point is from the current local location.

[0113] For example, suppose the unreachable path points include unreachable path points P5, P7, and P11. The actual distance between unreachable path point P5 and the current local location is 0.8 meters, P7 is 2 meters, and P11 is 3.5 meters. Therefore, P11 is the furthest from the current local location, followed by P7, and P5 is the closest. We can first search for locally feasible paths from the current local location to unreachable path point P11; then search for locally feasible paths from the current local location to unreachable path point P7; and finally search for locally feasible paths from the current local location to unreachable path point P5.

[0114] Optionally, the actual distance between the current local location and the unreached path point = the distance between the grid coordinates of the current local location in the Cartesian coordinate system corresponding to the real-time local grid map and the grid coordinates of the unreached path point in the Cartesian coordinate system corresponding to the real-time local grid map × the size of the grid in the real-time local grid map.

[0115] In some embodiments, when searching for locally feasible paths from the current local location to the corresponding unreachable path point in order of distance from each unreachable path point to the current local location, a locally feasible path from the current local location to the unreachable path point furthest from the current local location is searched. When no locally feasible path from the current local location to the unreachable path point furthest from the current local location is found, the remaining unreachable path points are sequentially determined to be unreachable path points not reached by the mobile device in order of distance from the current local location to the current local location. When the remaining unreachable path points are unreachable path points not reached by the mobile device, a locally feasible path from the current local location to the remaining unreachable path points is searched.

[0116] Considering that mobile devices cyclically search for locally feasible paths and, after finding one, control the device to move towards unreached path points based on that path to eventually reach the target path, it's important to understand that the mobile device searches for locally feasible paths during its movement. Therefore, before searching for a locally feasible path from the current location to the remaining unreached path points, the device might have already reached those points. In this case, there's no need to search for a locally feasible path from the current location to the remaining unreached path points. Instead, before searching for a locally feasible path from the current location to the remaining unreached path points, we can first determine whether the remaining unreached path points are unreachable by the mobile device, in order of increasing distance from the current location. If a remaining unreached path point is indeed unreachable, then we search for a locally feasible path from the current location to that point.

[0117] For example, as shown in Figure 5, assuming the unreachable path points include unreachable path points P5, P7, and P11, and the distance between the current local position and unreachable path point P11 is greater than the distance between the current local position and unreachable path point P7, and the distance between the current local position and unreachable path point P7 is greater than the distance between the current local position and unreachable path point P5, then, firstly, a locally feasible path from the current local position to unreachable path point P11 is searched; if a locally feasible path from the current local position to path point P11 is found, then the mobile device is controlled to move based on this locally feasible path, and the process returns to step 102; if no locally feasible path from the current local position to unreachable path point P11 is found, then it is determined whether path point P7 is an unreachable path point that the mobile device has not reached; if unreachable path point P7 is an unreachable path point that the mobile device has not reached, then a search is performed to find a locally feasible path from the current local position to path point P11. Find a locally feasible path from the current local location to path point P7; if a locally feasible path from the current local location to the unreachable path point P7 is found, control the movement of the mobile device based on the locally feasible path and return to step 102; if no locally feasible path from the current local location to the unreachable path point P7 is found, determine whether the unreachable path point P5 is an unreachable path point that the mobile device has not reached; if the unreachable path point P5 is an unreachable path point that the mobile device has not reached, search for a locally feasible path from the current local location to the unreachable path point P5; if a locally feasible path from the current local location to the unreachable path point P5 is found, control the movement of the mobile device based on the locally feasible path and return to step 102; if the unreachable path point P5 is not an unreachable path point that the mobile device has not reached, that is, an reached path point that the mobile device has reached, then return to step 102.

[0118] Optionally, when determining whether the remaining unreachable path points are unreachable path points that the mobile device has not reached, the determination can be based on the information recorded by the mobile device.

[0119] For example, assuming that the reached path points recorded by the mobile device include path points P1 and P5, and the remaining unreached path points include path points P5 and P7, then path point P7 can be determined as an unreached path point that the mobile device has not yet reached, and path point P5 as a reached path point that the mobile device has already reached.

[0120] In some embodiments, when controlling the movement of a mobile device based on a locally feasible path, the movement speed of the corresponding locally feasible path is obtained; if the movement speed is less than 0, the collision status of the mobile device rotating in place is obtained; if the collision status describes that the mobile device rotating in place will not cause a collision, the mobile device is controlled to rotate in place.

[0121] The movement speed is a vector quantity. In one possible implementation, the movement speed can be represented by polar coordinates. Polar coordinates represent the movement speed using the magnitude of the speed and an angle; that is, the movement speed can be represented by (v, θ). Here, θ is the angle with a certain direction, such as the angle with the forward direction of the mobile device. For example, if the movement speed is (1 m / s, 30°), it means that the mobile device can move at a speed of 1 m / s in a direction with an angle of 30° with the forward direction of the mobile device.

[0122] It is understandable that if the movement speed is represented by polar coordinates, when the mobile device moves to a certain position, such as moving to the next path point that the mobile device has not yet reached in the local feasible path, the magnitude of the movement speed of the mobile device is greater than 0. Therefore, it can be pre-set that when the magnitude of the movement speed of the corresponding local feasible path is greater than 0, the mobile device is controlled to move to the next path point that the mobile device has not yet reached in the local feasible path; when the magnitude of the movement speed of the corresponding local feasible path is less than 0, the mobile device is controlled to rotate in place.

[0123] For example, when the movement speed is obtained at a certain moment, such as time T1, the magnitude of this speed can be compared with the value 0. If the movement speed is less than 0, it is determined that the mobile device needs to be controlled to rotate in place. Considering that the position of the mobile device at time T1 may be a location where rotating in place would cause a collision, the collision status of the mobile device during rotation can be obtained before controlling its rotation at time T1. If the collision status indicates that rotating in place will not cause a collision, then the mobile device is controlled to rotate in place.

[0124] In one possible implementation, after obtaining the collision condition of the mobile device rotating in place, if the collision condition describes that the mobile device rotating in place would cause a collision, then the mobile device is controlled to reverse.

[0125] Specifically, if the collision condition description indicates that rotating the mobile device in place would result in a collision, then the mobile device is controlled to move backward, i.e., in the opposite direction to its forward direction, so that the mobile device can move away from the position where rotating in place would cause a collision. After controlling the mobile device to move backward, the movement speed at the next moment, such as time T2, can be obtained. If the movement speed at time T2 is less than 0, and the collision condition description obtained at time T2 indicates that rotating the mobile device in place will not result in a collision, then the mobile device can be controlled to rotate in place.

[0126] In some embodiments, when obtaining the collision status of a mobile device rotating in place, if it is determined that if the mobile device is controlled to rotate in place, all the grids that the mobile device passes through are in an idle state in the real-time local grid map where the mobile device is located, then the collision status description indicates that the mobile device rotating in place will not cause a collision; if it is determined that if the mobile device is controlled to rotate in place, not all the grids that the mobile device passes through are in an idle state in the real-time local grid map where the mobile device is located, then the collision status description indicates that the mobile device rotating in place will cause a collision.

[0127] Specifically, when acquiring the collision status of a mobile device rotating in place, it can be determined whether, if the mobile device is controlled to rotate in place, all the grids that the mobile device passes through are in an idle state in the real-time local grid map where the mobile device is located. If it is determined that, if the mobile device is controlled to rotate in place, all the grids that the mobile device passes through are in an idle state in the real-time local grid map where the mobile device is located, the collision status description indicates that the mobile device rotating in place will not produce a collision. If it is determined that, if the mobile device is controlled to rotate in place, all the grids that the mobile device passes through are not in an idle state in the real-time local grid map where the mobile device is located, the collision status description indicates that the mobile device rotating in place will produce a collision.

[0128] In some embodiments, after obtaining the movement speed of the corresponding locally feasible path, if the movement speed is greater than 0, the mobile device is controlled to move towards an unreached path point that the mobile device has not yet reached, based on the movement speed.

[0129] For example, when the movement speed at a certain moment, such as time T1, is obtained, the magnitude of that movement speed can be compared with the value 0. If the movement speed at time T1 is greater than 0, it is determined that the mobile device needs to be controlled to move to the next path among the locally feasible path points that the mobile device has not yet reached. Then, at time T1, the mobile device can be controlled to move to the next path point among the locally feasible path points that the mobile device has not yet reached at that movement speed.

[0130] It should be noted that the above is merely an example of controlling the movement of a mobile device based on a locally feasible path and is not intended to limit this application. Any solution provided by related technologies can be used to control the movement of a mobile device based on a locally feasible path, and no specific restrictions are imposed here.

[0131] In some embodiments, after sequentially searching for locally feasible paths from the current local location to the corresponding unreachable path point in order of distance from each unreachable path point to the current local location in ascending order, when no locally feasible path from the current local location to each unreachable path point is found, infeasibility reason information is obtained. The infeasibility reason information describes that there are unreachable path points in an occupied state and / or there are grids in an occupied state within a second preset range around the unreachable path points; based on the infeasibility reason information, the global grid map is updated, and the process returns to step 101.

[0132] The second preset range can be set in advance by technicians. For example, the second preset range is a range centered on each unreached path point, with an actual distance of 1 meter from each unreached path point.

[0133] In one possible real-time mode, the actual distance between an unreachable path point and a grid is equal to the distance between the grid coordinates of the candidate path point and the grid in the Cartesian coordinate system corresponding to the real-time local grid map, multiplied by the size of the grid in the real-time local grid map. When the actual distance between an unreachable path point and a grid is not greater than 1 meter, the grid is considered to be a grid within the second preset range around the unreachable path point. Therefore, it is possible to determine whether a grid is within the second preset range around an unreachable path point by calculating the actual distance between the unreachable path point and the grid.

[0134] Specifically, since the infeasibility information is obtained based on a real-time local grid map, when there is an unreachable path point in an occupied state, the grid coordinates of that unreachable path point in the global grid map can be determined, and the grid represented by those grid coordinates in the global grid map can be updated from an idle state to an occupied state. When there are grids in an occupied state within a second preset range around an unreachable path point, the grid coordinates of those occupied grids in the global grid map can be determined, and the grid represented by those grid coordinates in the global grid map can be updated from an idle state to an occupied state, thereby updating the global grid map.

[0135] In some embodiments, when no locally feasible path from the current local location to each unreachable path point is found, and infeasibility reason information is obtained, when no locally feasible path from the current local location to each unreachable path point is found, a locally feasible path from the current local location to the unreachable path point closest to the current local location is searched after a preset time period; if no locally feasible path from the current local location to the unreachable path point closest to the current local location is found, infeasibility reason information is obtained.

[0136] The preset duration can be set by technicians. For example, the preset duration can be 1 second, 2 seconds, 3 seconds, etc.

[0137] For example, assuming the unreachable path points include P5, P7, and P11, and the current local position is closest to path point P5, if no locally feasible path is found from the current local position to reach P5, P7, and P11 respectively, a locally feasible path can be searched for from the current local position to P5 after 3 seconds. If a locally feasible path is found from the current local position to P5, it can be determined that the path was found because of the removal of an obstacle. Based on the locally feasible path, the mobile device is controlled to move, and the process returns to step 102. If no locally feasible path is found from the current local position to P5, it can be determined that the path cannot be found because of the presence of an obstacle. In this case, the infeasibility reason information is obtained. Based on the infeasibility reason information, the global grid map is updated, and the process returns to step 101.

[0138] In one possible implementation, after controlling the movement of the mobile device based on a locally feasible path, the process can immediately return to step 102.

[0139] For example, assuming that the unreachable path points include P5, P7 and P11, and the current local position is farthest from the unreachable path point P11, and a locally feasible path from the current local position to the unreachable path point P11 is found, then after controlling the movement of the mobile device based on the locally feasible path, we can immediately return to execute step 102.

[0140] In another possible implementation, after the mobile device moves based on the locally feasible path, it can return to step 102 after the mobile device reaches the unreached path point that is furthest from the current local location.

[0141] For example, assuming that the unreachable path points include P5, P7 and P11, and the current local position is farthest from the unreachable path point P11, and a locally feasible path from the current local position to the unreachable path point P11 is found, then the mobile device can be moved to the unreachable path point P11 based on the locally feasible path before returning to execute step 102.

[0142] In one possible implementation, after the mobile device moves based on the locally feasible path control, if there are new unreached path points in the real-time local grid map where the mobile device is located, the process can return to step 102.

[0143] For example, assuming that the unreachable path points include unreachable path points P5, P7 and P11, and the current local location is farthest from unreachable path point P11, and a locally feasible path from the current local location to unreachable path point P11 is found, then, when a new unreachable path point P15 exists in the real-time local grid map where the mobile device is located, the process can return to step 102.

[0144] In some embodiments, the mobile device provides two display modes, and the user of the mobile device can switch modes to switch the mobile device to the corresponding display mode.

[0145] When the mobile device is in the first display mode, if the mobile device is moving, it can only display basic information such as the status bar (such as current battery level, time, etc.), the moving speed of the mobile device, and the gear of the mobile device; when the mobile device stops moving, if the mobile device does not move after waiting for a set time (such as 3 seconds), it will only display basic information unrelated to the movement of the mobile device, such as the status bar (such as current battery level, time, etc.).

[0146] When the mobile device is in the second mode, the mobile device, its surrounding environment (positive and negative obstacles) can be displayed. Furthermore, once a globally feasible path is obtained, it can also be displayed simultaneously; similarly, once a locally feasible path is obtained, it can also be displayed simultaneously. Optionally, when the mobile device is in the second mode, in addition to displaying the above information, the information displayed when the mobile device is in the first mode can also be displayed simultaneously.

[0147] In an optional embodiment, when it is necessary to display the mobile device, positive obstacles and negative obstacles in the environment where the mobile device is located, the positive obstacles and negative obstacles in the environment where the mobile device is located are identified; the relative display positions of the positive obstacles and negative obstacles relative to the mobile device are determined; the display modes corresponding to the positive obstacles and negative obstacles are obtained; and the mobile device, positive obstacles and negative obstacles are displayed according to the relative display positions of the positive obstacles and negative obstacles relative to the mobile device and the display modes corresponding to the positive obstacles and negative obstacles.

[0148] In this scenario, both positive obstacles and mobile devices are located on the first side of the plane where the mobile device is located, while negative obstacles are located on the second side of the plane where the mobile device is located.

[0149] Taking a wheelchair as an example of a mobile device, since the wheelchair is above the ground, a positive obstacle can be an object that is above the ground and whose height exceeds a certain threshold (e.g., 2 cm, 3 cm, or 10 cm), such as a person, animal, furniture, or building. A negative obstacle can be an object that is below the ground and whose depth is below a certain threshold (e.g., -2 cm, -3 cm, or -10 cm), such as steps, ramps, or potholes.

[0150] It should be noted that the embodiments of this application do not limit the specific implementation of identifying positive and negative obstacles in the environment of the mobile device. Any method provided by related technologies can be used to identify positive and negative obstacles in the environment of the mobile device.

[0151] In one possible implementation, the relative display positions of the positive and negative obstacles relative to the mobile device can be determined based on the relative positions of the positive and negative obstacles relative to the mobile device in the actual environment. Thus, the relative positions of the positive and negative obstacles relative to the mobile device in the actual environment can be intuitively obtained through the content displayed on the mobile device.

[0152] The display methods for positive and negative obstacles are different.

[0153] In one possible implementation, the positive obstacle is displayed as a solid line, and the negative obstacle is displayed as a dashed line.

[0154] In another possible implementation, the positive obstacle is displayed as a line with a first preset line width, and the negative obstacle is displayed as a line with a second preset line width.

[0155] In one possible implementation, the positive obstacle is displayed as a solid line with a first preset line width, and the negative obstacle is displayed as a dashed line with a second preset line width.

[0156] The first preset line width and the second preset line width are different. The first preset line width and the second preset line width can be preset by those skilled in the art. For example, in order to enable users of mobile devices to better distinguish between positive and negative obstacles, the first preset line width can be 10 pixels and the second preset line width can be 40 pixels.

[0157] Understandably, displaying positive and negative obstacles in different ways allows users of mobile devices to intuitively understand the different environmental conditions of their mobile devices through the content displayed on the screen, thereby enabling them to make better decisions and enhance safety.

[0158] For example, when a wheelchair user sees a positive obstacle such as a building, they can adjust their direction of travel to maintain a safe distance from the obstacle and avoid collision. When they see a negative obstacle such as a pothole, they can slow down in advance and adjust their direction of travel to move the wheelchair away from the obstacle and avoid falling into it and getting injured.

[0159] The following explanation will take the example of displaying the wheelchair, positive obstacles, and negative obstacles through a display screen with a size of 800 pixels × 510 pixels provided by the wheelchair.

[0160] Assuming 1 pixel corresponds to an actual distance of 0.01 meters, the interface coordinate system corresponding to the display interface has its origin at the top left corner, with the positive X-axis pointing horizontally to the right and the positive Y-axis pointing vertically downwards. One unit of the interface coordinate system represents 1 pixel. The vehicle coordinate system corresponding to the wheelchair has its origin at the wheelchair's rotation center, with the wheelchair's forward direction as the positive X-axis, the left side of the wheelchair as the positive Y-axis, and the direction perpendicular to the ground above the wheelchair as the positive Z-axis. One unit of the vehicle coordinate system represents an actual distance of 0.01 meters. The wheelchair's dimensions are 1 meter × 0.6 meters. The wheelchair's rotation center is 0.9 meters from the front of the wheelchair and 0.3 meters from the left end of the wheelchair. When displaying the wheelchair image on the display interface, the wheelchair image needs to be centered horizontally, and the wheelchair image needs to be 100 pixels from the bottom edge of the display interface. Therefore, the display coordinates of the rotation center in the interface coordinate system corresponding to the display interface are (400, 400). The display size of the wheelchair image is 60 pixels × 100 pixels. The forward direction of the wheelchair in the wheelchair image is the same as the negative Y-axis direction of the interface coordinate system.

[0161] First, acquire environmental point cloud points within the following ranges: those in front of the wheelchair with an actual distance component of no more than 4 meters from the wheelchair's rotation center in the X-axis direction of the vehicle coordinate system; those on the left and right sides of the wheelchair with an actual distance component of no more than 4 meters from the wheelchair's rotation center in the Y-axis direction of the vehicle coordinate system; and those behind the wheelchair with an actual distance component of no more than 1.1 meters from the wheelchair's rotation center in the X-axis direction of the vehicle coordinate system.

[0162] Then, based on the environmental point cloud points within the above range, determine the relative display coordinates of the reference point cloud points corresponding to each positive obstacle within the above range relative to the rotation center (400, 400) in the interface coordinate system, and the relative display coordinates of the reference point cloud points corresponding to each negative obstacle relative to the rotation center (400, 400) in the interface coordinate system.

[0163] Specifically, the environmental point cloud points within the above-mentioned range are first identified to identify point cloud clusters representing each positive obstacle and point cloud clusters representing each negative obstacle.

[0164] Next, ignoring the Z-axis coordinates of the point cloud points, candidate point cloud points in each of the multiple preset directions are determined from the point cloud point clusters representing each positive obstacle; among the candidate point cloud points in each preset direction, the reference point cloud point closest to the rotation center is determined, thus obtaining the reference point cloud point corresponding to each positive obstacle; based on the relative position of the reference point cloud point corresponding to each positive obstacle with respect to the rotation center in the vehicle coordinate system, the relative display position of the reference point cloud point corresponding to each positive obstacle with respect to the rotation center in the interface coordinate system is determined, and the relative display position of the reference point cloud point corresponding to each positive obstacle with respect to the rotation center in the interface coordinate system is taken as the first relative display position of each positive obstacle with respect to the wheelchair.

[0165] Similarly, ignoring the Z-axis coordinates of the point cloud points, candidate point cloud points in each of the multiple preset directions are determined from the point cloud point clusters representing each negative obstacle; the reference point cloud point closest to the rotation center is determined from the candidate point cloud points in each preset direction, thus obtaining the reference point cloud point corresponding to each negative obstacle; based on the relative position of the reference point cloud point corresponding to each negative obstacle with respect to the rotation center in the vehicle coordinate system, the relative display position of the reference point cloud point corresponding to each negative obstacle with respect to the rotation center in the interface coordinate system is determined, and the relative display position of the reference point cloud point corresponding to each negative obstacle with respect to the rotation center in the interface coordinate system is taken as the first relative display position of each negative obstacle with respect to the wheelchair.

[0166] The preset directions can be obtained by dividing a 360-degree circular space into preset intervals, with the positive X-axis of the vehicle coordinate system as 0 degrees. These preset degrees can be pre-set by those skilled in the art. For example, preset degrees could be 0.1 degrees, 0.5 degrees, 1 degree, etc.

[0167] Understandably, since the coordinates of the reference point cloud corresponding to each positive obstacle are in the vehicle coordinate system, and the origin of the vehicle coordinate system is the rotation center of the wheelchair, the coordinates of the reference point cloud corresponding to each positive obstacle in the vehicle coordinate system represent the relative position of each positive obstacle with respect to the rotation center in the vehicle coordinate system. Based on the coordinates of the reference point cloud corresponding to each positive obstacle in the vehicle coordinate system and the coordinates of the rotation center in the interface coordinate system, the relative display coordinates of the reference point cloud corresponding to each positive obstacle with respect to the rotation center in the interface coordinate system can be determined.

[0168] For example, assuming that the coordinates of a reference point cloud corresponding to a certain positive obstacle in the vehicle coordinate system are (300, 300), then the relative display coordinates of the reference point cloud relative to the rotation center in the interface coordinate system are (100, 100).

[0169] Similarly, since the coordinates of the reference point cloud corresponding to each negative obstacle are in the vehicle coordinate system, and the origin of the vehicle coordinate system is the rotation center of the wheelchair, the coordinates of the reference point cloud corresponding to each negative obstacle in the vehicle coordinate system represent the relative position of each negative obstacle with respect to the rotation center in the vehicle coordinate system. Based on the coordinates of the reference point cloud corresponding to each negative obstacle in the vehicle coordinate system and the coordinates of the rotation center in the interface coordinate system, the relative display coordinates of the reference point cloud corresponding to each negative obstacle with respect to the rotation center in the interface coordinate system can be determined.

[0170] For example, assuming that the coordinates of a reference point cloud corresponding to a negative obstacle in the vehicle coordinate system are (-100, 300), then the relative display coordinates of the reference point cloud relative to the rotation center in the interface coordinate system are (500, 100).

[0171] Please refer to Figure 6. After obtaining the relative display coordinates of each positive obstacle and each negative obstacle, a wheelchair image is displayed on the display interface based on the display coordinates (400, 400). The display size of the wheelchair image is 60 pixels × 100 pixels. A curve fitting operation is performed based on the relative display coordinates of each positive obstacle, and a partial outline of each positive obstacle is drawn on the display interface in the form of a solid line. Similarly, a curve fitting operation is performed based on the relative display coordinates of each negative obstacle, and a partial outline of each negative obstacle is drawn on the display interface in the form of a dashed line, thereby realizing the display of the wheelchair, positive obstacles, and negative obstacles.

[0172] In an optional embodiment, a highlight can be displayed 5 pixels to the left of the top left of the wheelchair image; a shadow can be displayed 5 pixels to the right and 10 pixels down from the center of the wheelchair image to add lighting effects to the wheelchair, making the wheelchair user more focused on the wheelchair image and improving its aesthetics.

[0173] In an optional embodiment, the relative display coordinates of the point cloud points in the point cloud cluster representing each positive obstacle relative to the rotation center in the interface coordinate system can be determined directly based on the coordinates of the point cloud points in the point cloud cluster representing each positive obstacle in the vehicle coordinate system and the display coordinates of the rotation center; the relative display coordinates of the point cloud points in the point cloud cluster representing each negative obstacle relative to the rotation center in the interface coordinate system can also be determined based on the coordinates of the point cloud points in the point cloud cluster representing each negative obstacle in the vehicle coordinate system and the display coordinates of the rotation center; then, a curve fitting operation is performed based on the relative display coordinates of each positive obstacle, and the outline of each positive obstacle is drawn on the display interface in the form of a solid line; a curve fitting operation is performed based on the relative display coordinates of each negative obstacle, and the outline of each negative obstacle is drawn on the display interface in the form of a dashed line, so as to realize the display of positive and negative obstacles.

[0174] In some embodiments, after obtaining the globally feasible path, the relative display position of the globally feasible path relative to the mobile device can also be determined; then, when displaying the mobile device, positive obstacles, and negative obstacles according to the relative display positions of positive and negative obstacles relative to the mobile device, and the display modes corresponding to positive and negative obstacles, the mobile device, positive obstacles, and negative obstacles are displayed according to the relative display positions of positive obstacles, negative obstacles, and the globally feasible path relative to the mobile device, and the display modes corresponding to positive and negative obstacles.

[0175] Optionally, the relative display position of the global feasible path with respect to the mobile device can be determined based on the relative positions of the initial path points that make up the global feasible path with respect to the mobile device in the actual environment.

[0176] Continuing with the example of displaying wheelchairs, positive obstacles, negative obstacles, and global feasible paths through the aforementioned display interface, after obtaining the global feasible path, the coordinates of each initial path point in the vehicle coordinate system can be determined; based on the coordinates of each initial path point in the vehicle coordinate system and the display coordinates of the rotation center, the relative display coordinates of each initial path point relative to the rotation center in the interface coordinate system can be determined.

[0177] It should be noted that if there is an initial path point whose relative display coordinates exceed the range that the display interface can display, the relative display coordinates of that initial path point can be ignored, and only the relative display coordinates of the remaining initial path points can be retained.

[0178] Please refer to Figure 7. After obtaining the relative display coordinates of each positive obstacle, the relative display coordinates of each negative obstacle, and the relative display coordinates of the initial path point, the wheelchair, positive obstacles, and negative obstacles are displayed on the display interface. For specific implementation, please refer to the previous embodiments, which will not be repeated here. Based on the display coordinates (400, 400) and the relative display coordinates of the initial path point, a curve fitting operation is performed to draw a global feasible path on the display interface in the form of a solid line with a third preset line width, so as to realize the display of the wheelchair, positive obstacles, negative obstacles, and the global feasible path.

[0179] The third preset line width can be preset by those skilled in the art. For example, the third preset line width can be 20 pixels.

[0180] Understandably, due to limitations in the display interface size and the actual distance corresponding to 1 pixel, when the distance component of the globally feasible path in the X-axis direction is large, only the portion of the globally feasible path closest to the wheelchair can be displayed. If more of the globally feasible path needs to be displayed, the size of the display interface and / or the actual distance corresponding to 1 pixel can be adjusted. Understandably, as the size of the display interface and / or the actual distance corresponding to 1 pixel is adjusted, the actual range of positive and negative obstacles displayed on the display interface, as well as the size of the wheelchair, positive obstacles, and negative obstacles, will also change accordingly.

[0181] It is understandable that when the initial path point is in front of the wheelchair in the actual environment, the relative display coordinates of the initial path point are also in front of the display coordinates in the direction of the wheelchair's movement. Therefore, the globally feasible path will also be displayed in front of the wheelchair, as shown in Figure 7. When the initial path point is behind the wheelchair in the actual environment, the relative display coordinates of the initial path point are also behind the display coordinates in the direction of the wheelchair's movement. Therefore, the globally feasible path will also be displayed behind the wheelchair, as shown in Figure 8.

[0182] In some embodiments, during the process of controlling the movement of a mobile device based on a locally feasible path, the mobile device, positive obstacles, negative obstacles, and paths not traversed by the mobile device within the locally feasible path can also be displayed. That is, during the process of controlling the movement of a mobile device based on a locally feasible path, positive and negative obstacles in the environment where the mobile device is located are identified; the relative display positions of the positive obstacles, negative obstacles, and locally feasible paths relative to the mobile device are determined; the display modes corresponding to the positive and negative obstacles are obtained; and the mobile device, positive obstacles, negative obstacles, and paths not traversed by the mobile device within the locally feasible path are displayed according to the relative display positions and display modes.

[0183] Optionally, the relative display position of the locally feasible path with respect to the mobile device can be determined based on the relative positions of the unreachable path points that make up the locally feasible path with respect to the mobile device in the actual environment.

[0184] Continuing with the example of displaying a wheelchair, positive obstacles, negative obstacles, and partially feasible paths through the aforementioned display interface, during the process of controlling the wheelchair's movement based on the partially feasible path, the unreached path points that the wheelchair has not yet traversed in the partially feasible path are obtained, and the coordinates of each unreached path point in the vehicle coordinate system are determined. Based on the coordinates of each unreached path point in the vehicle coordinate system and the display coordinates of the rotation center, the relative display coordinates of each unreached path point relative to the rotation center in the interface coordinate system are determined.

[0185] It should be noted that if there are unreachable path points whose relative display coordinates are beyond the range that the display interface can show, the relative display coordinates of the unreachable path points can be ignored, and only the relative display coordinates of the remaining unreachable path points can be retained.

[0186] Please refer to Figure 9. After obtaining the relative display coordinates of each positive obstacle, each negative obstacle, and the relative display coordinates of the unreached path points, the wheelchair, positive obstacles, and negative obstacles are displayed on the display interface. For specific implementation details, please refer to the previous embodiments, which will not be repeated here. Based on the display coordinates (400, 400) and the relative display coordinates of the unreached path points, a curve fitting operation is performed to draw a partially feasible path on the display interface in the form of a solid line with a fourth preset line width, so as to realize the display of the wheelchair, positive obstacles, negative obstacles, and the path that the wheelchair has not passed through among the partially feasible paths.

[0187] The fourth preset line width can be preset by someone skilled in the art. The fourth preset line width can be the same as or different from the third preset line width. For example, the fourth preset line width can be 20 pixels, 30 pixels, 40 pixels, etc.

[0188] It is understandable that when an unreachable path point is in front of the wheelchair in the actual environment, the relative displayed coordinates of the unreachable path point are also in front of the displayed coordinates in the direction of the wheelchair's movement. Therefore, a partially feasible path will also be displayed in front of the wheelchair, as shown in Figure 9. When an unreachable path point is behind the wheelchair in the actual environment, the relative displayed coordinates of the unreachable path point are also behind the displayed coordinates in the direction of the wheelchair's movement. Therefore, a partially feasible path will also be displayed behind the wheelchair, as shown in Figure 10.

[0189] In one possible implementation, the outlines of positive or negative obstacles that are close to the wheelchair can also be represented by different colors. For example, the outlines of positive or negative obstacles that are farther from the wheelchair can be displayed in white, while the outlines of positive or negative obstacles that are closer to the wheelchair can be displayed in red.

[0190] In one possible implementation, the display color of the locally feasible path can also be set. For example, the display color of the locally feasible path can be preset by a technician, or it can be set by the wheelchair user.

[0191] Optionally, the wheelchair image can be displayed at the top layer of the display interface, while the partially feasible path can be displayed at the next lower layer to avoid the partially feasible path obstructing the wheelchair; or, if the partially feasible path obstructs the wheelchair, the part of the movement trajectory that obstructs the wheelchair can be omitted.

[0192] Optionally, a "pause" button can also be displayed on the screen. When the wheelchair user taps the "pause" button with their finger, the wheelchair stops moving, and a "start" button is displayed instead of a "pause" button on the screen. When the wheelchair user taps the "start" button with their finger, the wheelchair continues to move based on the locally feasible path, and a "pause" button is displayed instead of a "start" button on the screen.

[0193] Optionally, the progress of the journey can also be displayed on the screen according to the percentage of the globally feasible path completed.

[0194] Optionally, while displaying the content shown in Figure 9 on the screen, a "view switch" button can also be displayed on the screen. When the wheelchair user clicks the "view switch" button, the display screen switches from showing the content shown in Figure 9 to showing the content shown in Figure 4. After switching to the content shown in Figure 4, a "view switch" button can also be displayed on the screen. When the wheelchair user clicks the "view switch" button, the display screen switches from showing the content shown in Figure 4 to showing the content shown in Figure 9.

[0195] Optionally, wheelchair users can also use their fingers to zoom in / out on the display screen to zoom in / out on the content displayed on the screen. After zooming in / out on the content displayed on the screen, a "view restore" button can be displayed on the screen. When the wheelchair user clicks the "view restore" button, the display screen will switch from showing the zoomed-out content to showing the content before zooming in / out.

[0196] Optionally, a "Close" button can be displayed on the screen. When the wheelchair user taps the "Close" button, a confirmation screen can be displayed to ask whether to stop the current movement. If the wheelchair user confirms to stop the current movement, the wheelchair will stop moving.

[0197] Optionally, when the wheelchair approaches its destination, the user can be notified in advance by displaying or broadcasting a prompt message, and the wheelchair can be brought to a smooth stop at a suitable location to complete the journey.

[0198] In this embodiment, a globally feasible path from the current global location of the mobile device to the target global location is determined in the global grid map. This globally feasible path includes multiple initial path points. The current local location of the mobile device in the real-time local grid map where the mobile device is located is determined. Unreachable path points in the real-time local grid map are identified from among the multiple initial path points; these unreachable path points are path points that the mobile device has not yet reached. Local feasible paths from the current local location to the corresponding unreachable path points are searched sequentially according to the order of distance between each unreachable path point and the current local location, from farthest to closest. When the first search finds a path from the current local location to the target global location... When a local feasible path is reached from the previous local position to the corresponding unreachable path point, the mobile device is controlled to move based on the local feasible path, and then the process returns to determine the current local position of the mobile device in the real-time local grid map where the mobile device is located. The global feasible path can be determined first in the global grid map, and then a local feasible path can be determined in the real-time local grid map where the mobile device is located based on the global feasible path. The movement of the mobile device is controlled based on the local feasible path. This takes into account the real-time environment in which the mobile device is currently located, such as newly appearing obstacles, thereby improving the movement efficiency of the mobile device and enhancing its safety.

[0199] Please refer to Figures 11 and 12 together. Figure 11 is a flowchart illustrating the coarse planning process of a feasible path provided in the embodiment of this application, and Figure 12 is a flowchart illustrating the fine planning process of a feasible path provided in the embodiment of this application.

[0200] The following will further illustrate the control method provided in the embodiments of this application, using a wheelchair as an example and a two-dimensional grid map as an example.

[0201] The wheelchair system comprises a first planner, a second planner, and a third planner. The first planner performs coarse planning of feasible paths to determine globally feasible paths. The second planner performs fine planning of optional paths based on the globally feasible paths to determine locally feasible paths, and sends the initial speed of each moment corresponding to the locally feasible path to the third planner in chronological order. The third planner adjusts the received initial speed based on current real-time conditions, such as the current environment and the wheelchair's current real-time state, to determine a target speed for smooth wheelchair movement, and then controls the wheelchair to drive autonomously at the determined target speed.

[0202] The coarse planning process for feasible paths by the first planner is as follows:

[0203] 2011. Obtain the global raster map.

[0204] In this embodiment, a wheelchair can be used to drive in the area where a global grid map needs to be generated beforehand, and the area traversed by the wheelchair can be sampled during the wheelchair's movement to generate the global grid map. Optionally, to reduce the interference of dynamic obstacles on the generated global grid map, dynamic obstacles in the generated global grid map can be removed during the wheelchair sampling process to form a global grid map without dynamic obstacles.

[0205] 2012. Dilate the global raster map to obtain the dilated raster map.

[0206] The expansion radius can be preset by technicians. For example, the expansion radius can be half the width of the wheelchair. For instance, assuming the width of the wheelchair is 0.6 meters, the expansion radius would be 0.3 meters.

[0207] 2013. The expanded raster map was divided into multiple raster regions, and idle raster cells in each raster region were sampled as candidate path points to obtain multiple candidate path points.

[0208] Each grid area can be a rectangular area. The size of each grid area can be preset by technicians. For example, the size of each grid area is 1 meter × 1 meter. Each grid area includes multiple grids. For example, if the size of each grid is 0.1 meter × 0.1 meter, then each grid area includes 100 grids.

[0209] After dividing the dilated raster map into multiple raster regions, random sampling is performed within each raster region. If the sampled raster is an empty raster, the sampling is considered successful, and the empty raster is used as a candidate path point. If the sampled raster is an occupied raster, the sampling is considered unsuccessful, and random sampling continues until a successful sampling is performed. The sampled empty raster is then used as a candidate path point. Optionally, each raster region includes one candidate path point.

[0210] 2014. The connectivity between each candidate path point and the candidate path points within a first preset range around it is determined using the A* algorithm.

[0211] The connectivity between two candidate path points can be either connected or disconnected. If a feasible path from one candidate path point to the other is found using the A* algorithm, then the two candidate path points are considered connected; if no feasible path is found using the A* algorithm, then the two candidate path points are considered disconnected.

[0212] For example, taking the A* algorithm as an example to determine whether candidate path point P11 is connected to candidate path point P12 within a first preset range, the A* algorithm can be used to search for a feasible path from candidate path point P11 to candidate path point P12. If a feasible path from candidate path point P11 to candidate path point P12 is found, then the connection between candidate path point P11 and candidate path point P12 is determined to be: connected; if no feasible path from candidate path point P11 to candidate path point P12 is found, then the connection between candidate path point P11 and candidate path point P12 is determined to be: not connected.

[0213] The first preset range can be set in advance by technicians. For example, the first preset range is a range centered on each candidate path point, with an actual distance of 2 meters from each candidate path point.

[0214] In 2015, using Dijkstra's algorithm, a globally feasible path was searched from the wheelchair's current global position to the target global position based on the connectivity relationship. The globally feasible path includes multiple initial path points.

[0215] Specifically, the Dijkstra algorithm searches from multiple candidate path points for the starting path point closest to the wheelchair's current global position and the target path point closest to the target global position. Based on this connectivity, the Dijkstra algorithm selects intermediate path points from the multiple candidate path points to form a feasible path from the starting path point to the target path point. The starting path point, intermediate path points, and target path point are determined as multiple initial path points, and the feasible path formed by the multiple initial path points is determined as the globally feasible path from the current global position to the target global position.

[0216] The second planner performs detailed planning of feasible paths as follows:

[0217] 2021. Determine the current local location of the wheelchair in the real-time local grid map where the wheelchair is located.

[0218] Specifically, a real-time local raster map centered on the current location of the wheelchair can be generated first.

[0219] Optionally, the size of the grid cells in the real-time local grid map is the same as the size of the grid cells in the global grid map. For example, if the size of the grid cells in the global grid map is 0.1 m × 0.1 m, then the size of the grid cells in the real-time local grid map is also 0.1 m × 0.1 m.

[0220] Once the real-time local grid map is obtained, the grid in which the wheelchair is located is determined, and this grid is identified as the current local position of the wheelchair in the real-time local grid map.

[0221] 2022. Select unreachable path points in the real-time local grid map from multiple initial waypoints. Unreachable path points are path points that the wheelchair cannot reach.

[0222] Since the initial path points are determined based on the global grid map, their coordinates are in the Cartesian coordinate system corresponding to the global grid map. However, the detailed planning of feasible paths is based on the real-time local grid map. Therefore, it is necessary to transform the coordinates of the initial path points to the Cartesian coordinate system corresponding to the real-time local grid map. This will allow us to determine which initial path points are located in the real-time local grid map and are path points that the wheelchair cannot reach.

[0223] 2023. Search for a locally feasible path from the current local location to the unreachable point that is furthest from the current local location.

[0224] Specifically, the unreachable path point furthest from the current local location can be determined by calculating the actual distance between the current local location and each unreachable path point.

[0225] In this context, the smaller the actual distance, the closer the point is to the current local location; conversely, the larger the actual distance, the farther the point is from the current local location. For example, suppose the unreached path points include P5, P7, and P11. The actual distance between unreached path point P5 and the current local location is 0.8 meters, the actual distance between unreached path point P7 and the current local location is 2 meters, and the actual distance between unreached path point P11 and the current local location is 3.5 meters. Therefore, it is determined that unreached path point P11 is the farthest from the current local location, followed by unreached path point P7, and unreached path point P5 is the closest.

[0226] 2024. When no local feasible path is found from the current local location to the unreachable path point that is furthest from the current local location, the remaining unreachable path points are sequentially determined in order of their distance from the current local location, from farthest to closest, to determine whether the remaining unreachable path points are unreachable path points that the wheelchair cannot reach.

[0227] 2025. When the remaining unreachable path points are those that the wheelchair cannot reach, search for locally feasible paths from the current local location to the remaining unreachable path points.

[0228] 2026. When a locally feasible path from the current local location to the corresponding unreachable path point is found for the first time, the wheelchair is moved based on the locally feasible path, and the process returns to step 2021.

[0229] Considering that the second planner iteratively refines feasible path planning to search for locally feasible paths, and after finding a locally feasible path, controls the wheelchair to move towards the path points it has not yet reached, ultimately reaching the target path point. In other words, the second planner performs detailed feasible path planning during the wheelchair's movement. Therefore, before searching for locally feasible paths from the current local position to the remaining unreachable path points, the wheelchair may have already reached those points. Thus, there's no need to search for locally feasible paths from the current local position to the remaining unreachable path points. Therefore, before searching for locally feasible paths from the current local position to the remaining unreachable path points, we can first determine whether the remaining unreachable path points are unreachable path points that the wheelchair has not reached, in order of increasing distance from the current local position. If a remaining unreachable path point is indeed unreachable by the wheelchair, then we search for locally feasible paths from the current local position to the remaining unreachable path points.

[0230] For example, suppose the unreachable path points include unreachable path points P5, P7, and P11. The distance between the current local position and unreachable path point P11 is greater than the distance between the current local position and unreachable path point P7, and the distance between the current local position and unreachable path point P7 is greater than the distance between the current local position and unreachable path point P5. Then, firstly, search for a locally feasible path from the current local position to unreachable path point P11. If a locally feasible path from the current local position to path point P11 is found, control the wheelchair movement based on this locally feasible path and return to step 2021. If no locally feasible path from the current local position to unreachable path point P11 is found, determine whether path point P7 is an unreachable path point that the wheelchair has not reached. If unreachable path point P7 is an unreachable path point that the wheelchair has not reached, then search for a locally feasible path from the current local position to path point P11. If a locally feasible path from the current local position to the unreachable path point P7 is found, the wheelchair is moved based on this locally feasible path, and the process returns to step 2021. If no locally feasible path from the current local position to the unreachable path point P7 is found, it is determined whether the unreachable path point P5 is an unreachable path point that the wheelchair has not reached. If the unreachable path point P5 is an unreachable path point that the wheelchair has not reached, a locally feasible path from the current local position to the unreachable path point P5 is searched. If a locally feasible path from the current local position to the unreachable path point P5 is found, the wheelchair is moved based on this locally feasible path, and the process returns to step 2021. If the unreachable path point P5 is not an unreachable path point that the wheelchair has not reached, i.e., it is a reached path point that the wheelchair has reached, the process returns to step 2021.

[0231] 2027. When no locally feasible path from the current local location to each unreachable path point is found, search for a locally feasible path from the current local location to the unreachable path point closest to the current local location after a preset time.

[0232] 2028. If no locally feasible path is found from the current local location to the nearest unreachable path point, obtain the reason why it is infeasible.

[0233] 2029. Send the infeasibility cause information to the first planner so that the first planner updates the global grid map based on the infeasibility cause information and returns to execution step 2012.

[0234] 2030. If a locally feasible path is found that leads from the current local location to the nearest unreachable path point, then the wheelchair is moved based on the locally feasible path, and the process returns to step 2021.

[0235] The preset duration can be set by technical personnel. For example, the preset duration can be 1 second, 2 seconds, 3 seconds, etc.

[0236] The description of the infeasibility reason is that there are unreachable path points in an occupied state and / or there are occupied grids within a second preset range around the unreachable path points. The second preset range can be preset by a technician. For example, the second preset range is a range centered on each unreachable path point, with an actual distance of 1 meter from each unreachable path point.

[0237] For example, assuming the unreachable path points include P5, P7, and P11, and the current local position is closest to path point P5, if no locally feasible paths are found from the current local position to P5, P7, and P11 respectively, a locally feasible path can be searched for from the current local position to P5 after 3 seconds. If a locally feasible path is found from the current local position to P5, it can be determined that the path may have been found because an obstacle has been removed. If a locally feasible path from the current local position to the unreachable path point P5 is found, the wheelchair is moved based on the locally feasible path, and the process returns to step 2021. If no locally feasible path from the current local position to the unreachable path point P5 is found, it can be determined that the existence of obstacles prevents the search of a locally feasible path from the current local position to the unreachable path point P5. In this case, the infeasibility reason information is obtained and sent to the first planner so that the first planner updates the global grid map based on the infeasibility reason information and returns to step 2012.

[0238] In an optional embodiment, the second planner can determine the initial speed at each moment based on the locally feasible path and send the determined initial speed to the third planner. However, when determining the initial speed, the second planner only considers the relative positions between the path points that make up the locally feasible path, without considering the current real-time conditions, such as obstacles in the current real-time environment or the current speed of the wheelchair. Therefore, if the wheelchair is controlled to move at this initial speed, it may collide with obstacles or cause discomfort to the wheelchair occupant due to sudden acceleration. Therefore, after receiving the initial speed, the third planner can adjust the initial speed based on the current real-time conditions to obtain a target speed and control the wheelchair to move at that target speed to avoid collisions with obstacles or discomfort to the wheelchair occupant due to sudden acceleration, thus achieving safe and comfortable autonomous driving.

[0239] Please refer to Figure 13, which is a flowchart illustrating the information display method provided in an embodiment of this application. The process may include:

[0240] In 301, positive and negative obstacles in the environment of the mobile device are identified. Both the positive obstacles and the mobile device are located on the first side of the plane in which the mobile device is located, and the negative obstacles are located on the second side of the plane.

[0241] In step 302, the first relative display positions of the positive and negative obstacles relative to the mobile device are determined.

[0242] In 303, obtain the display methods corresponding to positive and negative obstacles respectively.

[0243] In 304, the movable device, positive obstacle, and negative obstacle are displayed according to the first relative display position and display method.

[0244] It should be noted that the specific implementation of steps 301 to 304 can be found in the previous embodiments, and will not be repeated here.

[0245] In some embodiments, before displaying the mobile device, positive obstacle, and negative obstacle according to the first relative display position and display mode, the moving speed of the mobile device is obtained; based on the moving speed, the moving trajectory of the mobile device relative to the second relative display position of the mobile device is determined; when displaying the mobile device, positive obstacle, and negative obstacle according to the first relative display position and display mode, the mobile device, positive obstacle, negative obstacle, and moving trajectory are displayed according to the first relative display position, the second relative display position, and the display mode.

[0246] The movement speed is a vector. In one possible implementation, the movement speed can be represented by two components. Taking a wheelchair as an example, one component is the component v in the X-axis direction of the vehicle coordinate system corresponding to the wheelchair. x The other component is the Y-axis component v in the vehicle coordinate system corresponding to the wheelchair. y It can be denoted as movement speed. The vehicle coordinate system has the center of rotation of the wheelchair as the origin, the forward direction of the wheelchair as the positive X-axis, the left side of the wheelchair as the positive Y-axis, and the direction perpendicular to the ground where the wheelchair is located as the positive Z-axis.

[0247] Speed ​​of movement When V x When V is positive, the direction of the moving speed is forward; when V is positive... x When V is negative, the direction of the moving speed is backward; when V... y When V is positive, the direction of the moving speed is to the left; when V is positive... y When the value is negative, the direction of the movement speed is to the right. For example, if This indicates that the direction of the movement speed is to the left and rear; if This indicates that the direction of the movement speed is directly forward.

[0248] Optionally, the movement speed can be the movement speed indicated by the speed control operation performed by the user of the mobile device. Taking speed control operation via a joystick mounted on a wheelchair as an example, the wheelchair user can operate the joystick to perform speed control operation, and the mobile device determines the speed direction based on the angle of the joystick and the speed magnitude based on the displacement of the joystick, thus obtaining the movement speed of the mobile device.

[0249] The movement trajectory of a mobile device can be the predicted movement trajectory of the mobile device within a preset time period, such as the next 2 seconds, 5 seconds, or 10 seconds.

[0250] In an optional embodiment, the display position of the mobile device can be preset. Taking the display of a wheelchair, positive obstacles, and negative obstacles on the display screen as an example, the display position of the wheelchair can be the center of the display screen; considering that the wheelchair moves forward in most cases, the display position of the wheelchair can also be a position near the lower boundary of the display screen, so as to display more of the positive or negative obstacles in front of the wheelchair, so that the wheelchair rider can better understand the distribution of positive or negative obstacles in front of the wheelchair through the display screen.

[0251] In another optional embodiment, during the movement of the mobile device, the display position of the mobile device can also be determined based on the movement speed. Before determining the second relative display position of the mobile device's movement trajectory relative to the mobile device based on the movement speed, the display position of the mobile device is determined based on the movement speed. When determining the second relative display position of the mobile device's movement trajectory relative to the mobile device based on the movement speed, the second relative display position of the mobile device's movement trajectory relative to the mobile device is determined based on the movement speed and the display position. When displaying the mobile device, positive obstacle, negative obstacle, and movement trajectory based on the first relative display position, the second relative display position, and the display mode, the mobile device, positive obstacle, negative obstacle, and movement trajectory are displayed based on the display position, the first relative display position, the second relative display position, and the display mode.

[0252] It is understandable that when a mobile device moves forward, and the speed component of the mobile device in the forward direction is large, a larger display area is needed in front of the mobile device to display the movement trajectory; when a mobile device moves backward, and the speed component of the mobile device in the backward direction is large, a larger display area is needed behind the mobile device to display the movement trajectory. Therefore, the display position of the mobile device can be determined based on the speed of the mobile device.

[0253] In one possible implementation, taking a wheelchair as an example, the display position of the mobile device is determined to be near the lower boundary of the display interface, based on the criterion of displaying more positive or negative obstacles in front of the wheelchair. When the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is positive, or when the velocity component is negative and its absolute value is not greater than a speed threshold, the display position of the mobile device is determined based on the absolute value of the velocity component. For example, a preset mapping relationship between the absolute value of the velocity component and the display position of the mobile device can be established in advance. The display position of the mobile device is determined based on the absolute value of the velocity component and the preset mapping relationship. The larger the absolute value of the velocity component, the closer the wheelchair is to the upper boundary of the display interface.

[0254] The speed threshold can be preset by technicians. Optionally, the speed threshold can range from (0.05vmax, 0.95vmax), where vmax is the maximum speed the wheelchair can reach in the backward direction.

[0255] In another possible implementation, taking a wheelchair as an example of a mobile device, when the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is positive, the display position of the wheelchair is determined to be near the lower boundary of the display interface; when the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is negative, the display position of the wheelchair is determined to be near the upper boundary of the display interface.

[0256] Optionally, the display width of the movable device and the movement trajectory is the same in the first display direction.

[0257] The first display direction can be a direction perpendicular to the forward movement direction of the display device on the display screen. For example, if the forward movement direction of the mobile device on the display screen is perpendicular to the width direction of the display screen, then the first display direction is the width direction of the display screen; if the forward movement direction of the mobile device is perpendicular to the height direction of the display screen, then the first display direction is the height direction of the display screen.

[0258] Understandably, by making the display width of the mobile device and the movement trajectory the same in the first display direction, the user of the mobile device can know whether the mobile device will collide with surrounding positive or negative obstacles if it continues to move along the movement trajectory. Thus, knowing that the mobile device will collide with surrounding positive or negative obstacles if it continues to move along the movement trajectory, the user can adjust the movement speed of the wheelchair in advance, such as adjusting the linear speed and / or angular speed, to avoid collisions with surrounding positive or negative obstacles.

[0259] Continuing with the example of displaying a wheelchair, positive obstacle, negative obstacle, and movement trajectory using the display interface described in the previous embodiment, when the speed component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is positive, or when the speed component is negative and the absolute value of the speed component is not greater than a speed threshold, the display coordinates of the rotation center in the display interface are set to (400, 400). That is, the distance between the wheelchair and the lower boundary of the display interface is always kept equal to the display width of the wheelchair in the Y-axis direction, which is 100 pixels. When the speed component is negative and the absolute value of the speed component is greater than the speed threshold, the range of the Y-axis coordinates of the rotation center in the display interface is (190, 400). That is, the minimum distance between the wheelchair and the upper boundary of the display interface is equal to the display width of the wheelchair in the Y-axis direction, and the larger the absolute value of the speed component, the smaller the value.

[0260] After obtaining the wheelchair's moving speed, the coordinates of the wheelchair's rotation center in the vehicle coordinate system are predicted at each moment within a preset time period based on the wheelchair's moving speed. Ignoring the Z-axis coordinate of each of the above coordinates, the relative display coordinates of the rotation center in the interface coordinate system at each moment within the preset time period are determined based on the above coordinates and the display coordinates. The relative display coordinates of the rotation center in the interface coordinate system at each moment within the preset time period are determined as the second relative display position of the movement trajectory relative to the wheelchair.

[0261] Please refer to Figure 14. If the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is positive, after obtaining the relative display coordinates of each positive obstacle, the relative display coordinates of each negative obstacle, and the relative display coordinates of the rotation center, the wheelchair, positive obstacles, and negative obstacles are displayed on the display interface. For specific implementation, please refer to the previous embodiments, which will not be repeated here. Based on the display coordinates (400, 400) and the relative display coordinates of the rotation center, a curve fitting operation is performed, and the movement trajectory of the wheelchair is drawn on the display interface in the form of a solid line with a fifth preset line width, so as to realize the display of the wheelchair, positive obstacles, negative obstacles, and movement trajectory.

[0262] It should be noted that when the velocity component in the positive X-axis direction is positive, if the velocity component is large, the relative display coordinates of the rotation center may be beyond the range that the display interface can display. In this case, the relative display coordinates can be ignored, and the wheelchair's movement trajectory can be drawn based only on the remaining relative display coordinates.

[0263] The fifth preset line width can be preset by those skilled in the art. For example, the fifth preset line width can be 60 pixels, which is the same as the display width of the wheelchair image in the X-axis direction of the interface coordinate system. This allows the wheelchair user to intuitively understand whether the wheelchair will collide with an obstacle at the current speed by observing the movement trajectory displayed on the interface.

[0264] It is understandable that if the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is negative, the relative display coordinates of the rotation center are all behind the display coordinates (400, 400). Therefore, when the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is negative, and the absolute value of this velocity component is not greater than the velocity threshold, the difference from the above example is that the movement trajectory is displayed behind the wheelchair, as shown in Figure 15.

[0265] When the velocity component of the wheelchair's movement speed in the X-axis direction of the vehicle coordinate system is negative, and the absolute value of this velocity component is greater than a certain velocity threshold, the Y-axis coordinate corresponding to the absolute value of this velocity component can be determined. Assuming the Y-axis coordinate corresponding to the absolute value of this velocity component is 240, the display coordinates of the rotation center can be determined as (400, 240). The difference from the previous example is that this example is based on environmental point cloud points within a range: those in front of the wheelchair with an actual distance component of no more than 2.4 meters from the wheelchair's rotation center in the X-axis direction; those on the left and right sides of the wheelchair with an actual distance component of no more than 4 meters from the wheelchair's rotation center in the Y-axis direction; and those behind the wheelchair with an actual distance component of no more than 2.7 meters from the wheelchair's rotation center in the X-axis direction. The relative display coordinates of the reference point cloud point corresponding to each negative obstacle within this range relative to the rotation center (400, 240) in the interface coordinate system are determined. The relative display coordinates of the reference point cloud points corresponding to each negative obstacle and the rotation center in the interface coordinate system are obtained. Please refer to Figure 16. After obtaining the relative display coordinates of each positive obstacle, each negative obstacle, and the rotation center, the wheelchair, positive obstacle, and negative obstacle are displayed on the display interface. For specific implementation, please refer to the previous embodiments, which will not be repeated here. Based on the display coordinates (400, 240) and the relative display coordinates of the rotation center, a curve fitting operation is performed, and the movement trajectory of the wheelchair is drawn on the display interface in the form of a solid line with a fifth preset line width.

[0266] In some embodiments, after displaying the mobile device, positive obstacle, and negative obstacle according to the first relative display position and display mode, an input rotation operation is received; in response to the rotation operation, a prompt animation is displayed to prompt the rotation of the mobile device; the mobile device is controlled to rotate in place, and during the rotation of the mobile device in place, the process returns to step 301.

[0267] It is understandable that users of mobile devices may also control the mobile device to rotate in place to adjust its orientation. To avoid discomfort caused by sudden rotation, after receiving the user's input of a rotation operation, a prompt animation can be displayed in response to the rotation operation to inform the user that the mobile device will rotate in place. Then, the mobile device can be controlled to rotate in place, and during the rotation, the process can return to step 301 to display the mobile device, positive obstacles, and negative obstacles, so that the user can intuitively see the changes in positive and negative obstacles in the environment where the mobile device is located as the mobile device rotates.

[0268] In one possible implementation, the prompt animation includes a fan-shaped animation, wherein the fan-shaped animation includes a fan that dynamically expands from 0 radians to a target radian, the target radian being determined according to the rotation angle indicated by the in-place rotation operation, the display position of the apex of the fan being determined according to the rotation center of the mobile device's in-place rotation, and the radius of the fan being determined according to the display position and the display width of the mobile device in a second display direction, thereby enabling the user of the mobile device to intuitively understand the rotation angle of the wheelchair based on the fan-shaped animation, and whether the wheelchair is likely to collide with a positive or negative obstacle during rotation.

[0269] The target radius is positively correlated with the rotation angle. That is, the larger the rotation angle, the larger the target radius; the smaller the rotation angle, the smaller the target radius.

[0270] For example, please refer to Figures 17 to 19 together. Figure 17 is a schematic diagram of the moment when the arc of the sector in the sector animation provided in the embodiment of this application is π / 2. Figure 18 is a schematic diagram of the moment when the arc of the sector in the sector animation provided in the embodiment of this application is π. Figure 19 is a schematic diagram of the moment when the arc of the sector in the sector animation provided in the embodiment of this application is 2π. Taking a wheelchair as an example of a mobile device, when the rotation angle is 360 degrees, the dynamic process of the sector expanding from 0 arcs to π / 2 arcs, then to π arcs, and finally to 2π arcs will be displayed on the display interface provided by the display screen.

[0271] For example, as shown in Figure 17, taking a wheelchair as an example of a mobile device, the center of rotation of the wheelchair is the center of the rear wheel. Therefore, the display position of the vertex of the sector can be the same as the display position of the center of the rear wheel of the wheelchair. For instance, if the display coordinates of the center of the rear wheel of the wheelchair in the interface coordinate system corresponding to the display interface are D, then the display coordinates of the vertex of the sector in the interface coordinate system are also D. The interface coordinate system is established with the upper left corner of the display interface as the origin, the positive X-axis pointing horizontally to the right, and the positive Y-axis pointing vertically downwards.

[0272] The second display direction can be a direction parallel to the forward movement of the display device on the display screen. For example, if the forward movement of the mobile device on the display screen is parallel to the height direction of the display screen, then the second display direction is the height direction of the display screen; if the forward movement of the mobile device on the display screen is parallel to the width direction of the display screen, then the second display direction is the width direction of the display screen.

[0273] For example, assuming the display width of the mobile device in the second display direction is 1000 pixels, and the rotation center is located at a display position 100 pixels away from the rear of the mobile device, then the display radius of the fan shape can be 900 pixels, or slightly larger than 900 pixels.

[0274] In this embodiment, positive and negative obstacles in the environment where the mobile device is located are identified. Both the positive obstacle and the mobile device are located on the first side of the plane where the mobile device is located, and the negative obstacle is located on the second side of the plane. The first relative display positions of the positive and negative obstacles relative to the mobile device are determined. The display modes corresponding to the positive and negative obstacles are obtained. According to the first relative display positions and display modes, the mobile device, positive obstacles, and negative obstacles are displayed. Positive and negative obstacles can be displayed in different ways, which allows users of the mobile device to intuitively understand the positive and negative obstacles in the environment where the mobile device is located through the content displayed on the screen, thereby making better different actions and enhancing safety.

[0275] In one possible implementation, the display color of the movement trajectory can also be set. For example, the display color of the movement trajectory can be preset by a technician, or it can be set by the wheelchair user.

[0276] Optionally, the wheelchair image, the outline of the positive obstacle, and the negative obstacle can be displayed at the top layer of the display interface, while the movement trajectory can be displayed at the next lower layer to avoid the movement trajectory obstructing the wheelchair, the positive obstacle, or the negative obstacle; or, if the movement trajectory would obstruct the wheelchair, the positive obstacle, or the negative obstacle, the part of the movement trajectory that obstructs the wheelchair, the positive obstacle, or the negative obstacle can be not displayed.

[0277] Please refer to Figure 20, which is a schematic diagram of the control device provided in an embodiment of this application. The control device includes: a path determination module 401, a position determination module 402, a path point determination module 403, a path search module 404, and a device control module 404.

[0278] The path determination module 401 is used to determine a globally feasible path from the current global location of the mobile device to the target global location in the global grid map. The globally feasible path includes multiple initial path points.

[0279] The location determination module 402 is used to determine the current local location of the mobile device in the real-time local grid map where the mobile device is located;

[0280] The waypoint determination module 403 is used to determine unreachable waypoints in the real-time local grid map from multiple initial waypoints. Unreachable waypoints are waypoints that the mobile device has not reached.

[0281] The path search module 404 is used to search for local feasible paths from the current local location to the corresponding unreachable path point in order of distance from each unreachable path point to the current local location, from farthest to closest.

[0282] The device control module 405 is used to control the movement of the mobile device based on the local feasible path when a local feasible path from the current local location to the corresponding unreachable path point is found for the first time, and return to execute the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0283] In an alternative embodiment, the path determination module 401 is specifically configured to perform:

[0284] Multiple candidate path points are identified in the global grid map. The candidate path points are grid cells in the global grid map that are in an idle state.

[0285] Determine the connectivity between each candidate path point and other candidate path points within a first preset range around it;

[0286] Among multiple candidate path points, determine the starting path point that is closest to the current global position, and the target path point that is closest to the target global position;

[0287] Based on connectivity, intermediate path points are determined from multiple candidate path points to form a feasible path from the starting path point to the target path point.

[0288] The starting path point, the target path point, and intermediate path points are determined as multiple initial path points, and the feasible path composed of multiple initial path points is determined as the globally feasible path from the current global position to the target global position.

[0289] In an alternative embodiment, the path determination module 401 is specifically configured to perform:

[0290] The connectivity between each candidate path point and its surrounding candidate path points within a first preset range is determined using a first preset algorithm.

[0291] The second preset algorithm determines the starting path point closest to the current global position and the target path point closest to the target global position from multiple candidate path points.

[0292] Using a second preset algorithm, intermediate path points are determined from multiple candidate path points based on connectivity to form a feasible path from the starting path point to the target path point.

[0293] In an alternative embodiment, the path determination module 401 is specifically configured to perform:

[0294] Divide the global raster map into multiple raster regions;

[0295] In each grid area, idle grid cells are randomly sampled as candidate path points to obtain multiple candidate path points. Idle grid cells are grid cells in an idle state in the global grid map.

[0296] In an optional embodiment, the control device further includes a map update module, which is specifically used to perform:

[0297] When no local feasible path is found from the current local location to each unreachable path point, obtain infeasibility reason information. The infeasibility reason information describes that there are unreachable path points in an occupied state and / or there are grids in an occupied state within a second preset range around the unreachable path points.

[0298] Based on the infeasibility information, update the global grid map and return to the step of determining a globally feasible path from the current global location of the mobile device to the target global location in the global grid map.

[0299] In an alternative embodiment, the map update module is specifically configured to perform:

[0300] When no locally feasible path from the current local location to each unreachable path point is found, search for a locally feasible path from the current local location to the unreachable path point closest to the current local location after a preset time.

[0301] If no locally feasible path is found from the current local location to the nearest unreachable path point, then obtain the reason why it is not feasible.

[0302] In an alternative embodiment, the device control module 405 is specifically configured to perform:

[0303] When the mobile device moves to the unreached waypoint that is furthest from the current local location, return to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0304] In an alternative embodiment, the device control module 405 is specifically configured to perform:

[0305] When the mobile device moves to a point in the real-time local grid map where the mobile device is located, and there is a new unreached path point, return to the step of determining the current local location of the mobile device in the real-time local grid map.

[0306] In an alternative embodiment, the path search module 404 is specifically configured to perform:

[0307] Search for a locally feasible path from the current local location to the unreachable path point that is furthest from the current local location;

[0308] When no local feasible path is found from the current local location to the unreachable path point that is furthest from the current local location, the remaining unreachable path points are sequentially determined in order of their distance from the current local location, from farthest to closest, to determine whether the remaining unreachable path points are unreachable path points that the mobile device has not reached.

[0309] When the remaining unreachable path points are unreachable path points that the mobile device cannot reach, search for a locally feasible path from the current local location to the remaining unreachable path points.

[0310] In an alternative embodiment, the device control module 405 is specifically configured to perform:

[0311] Obtain the movement speed of the corresponding locally feasible path;

[0312] If the movement speed is less than 0, obtain the collision status of the mobile device rotating in place;

[0313] If the collision condition description states that rotating the mobile device in place will not result in a collision, then control the mobile device to rotate in place.

[0314] In an alternative embodiment, the device control module 405 is specifically configured to perform:

[0315] If the collision condition description states that a collision would occur if the mobile device were to rotate in place, then control the mobile device to move backward.

[0316] In an alternative embodiment, the device control module 405 is specifically configured to perform:

[0317] When it is determined that if the mobile device is controlled to rotate in place, the grid that the mobile device passes through are all grids that are in an idle state in the real-time local grid map where the mobile device is located, it is determined that the collision situation description is that the mobile device will not cause a collision when rotating in place.

[0318] When it is determined that if the mobile device is controlled to rotate in place, the grids that the mobile device passes through are not all grids that are in an idle state in the real-time local grid map where the mobile device is located, it is determined that the collision condition description indicates that the mobile device will collide when rotating in place.

[0319] In an alternative embodiment, the device control module 405 is specifically configured to perform:

[0320] If the movement speed is greater than 0, then based on the movement speed, control the mobile device to move towards an unreached path point that the mobile device has not yet reached.

[0321] In an optional embodiment, the control device further includes an information display module, which is specifically used to perform:

[0322] During the process of controlling the movement of a mobile device based on a locally feasible path, positive and negative obstacles in the environment where the mobile device is located are identified. Both the positive obstacles and the mobile device are located on the first side of the plane where the mobile device is located, and the negative obstacles are located on the second side of the plane.

[0323] Determine the relative display positions of positive obstacles, negative obstacles, and locally feasible paths relative to the mobile device;

[0324] Obtain the display methods corresponding to positive and negative obstacles respectively;

[0325] Based on the relative display position and display method, display mobile devices, positive obstacles, negative obstacles, and paths that mobile devices have not yet traversed in the partially feasible paths.

[0326] Please refer to Figure 21, which is a schematic diagram of the structure of the information display device provided in an embodiment of this application. The information display device includes: an obstacle recognition module 501, a position determination module 502, a mode acquisition module 503, and an information display module 504.

[0327] The obstacle recognition module 501 is used to identify positive and negative obstacles in the environment where the mobile device is located. Both the positive obstacle and the mobile device are located on the first side of the plane where the mobile device is located, and the negative obstacle is located on the second side of the plane.

[0328] The position determination module 502 is used to determine the first relative display positions of positive and negative obstacles relative to the mobile device.

[0329] The mode acquisition module 503 is used to acquire the display modes corresponding to positive obstacles and negative obstacles respectively;

[0330] The information display module 504 is used to display the mobile device, positive obstacle and negative obstacle according to the first relative display position and display mode.

[0331] In an optional embodiment, the positive obstacle is displayed as a solid line, and the negative obstacle is displayed as a dashed line.

[0332] In an alternative embodiment, the location determination module 502 is specifically configured to perform:

[0333] Get the movement speed of the mobile device;

[0334] Based on the movement speed, determine the second relative display position of the movement trajectory relative to the movable device;

[0335] The information display module 504 is specifically used to perform:

[0336] Based on the first relative display position, the second relative display position, and the display method, the mobile device, positive obstacle, negative obstacle, and movement trajectory are displayed.

[0337] In an alternative embodiment, the location determination module 502 is specifically configured to perform:

[0338] The display position of the mobile device is determined based on the moving speed;

[0339] Based on the moving speed and the display position, determine the moving trajectory of the mobile device relative to the second relative display position of the mobile device;

[0340] The information display module 504 is specifically used to perform:

[0341] Based on the display position, the first relative display position, the second relative display position, and the display mode, the mobile device, the positive obstacle, the negative obstacle, and the movement trajectory are displayed.

[0342] In an alternative embodiment, the movable device and the movement trajectory have the same display width in the first display direction.

[0343] In an optional embodiment, the information display device further includes a rotation control module, which is specifically used to perform:

[0344] Receive input for in-situ rotation operation;

[0345] In response to a spin-in-place operation, a prompt animation is displayed to indicate when the mobile device is being rotated in place.

[0346] Control the mobile device to rotate in place, and while the mobile device is rotating in place, return to perform the step of identifying positive and negative obstacles in the environment in which the mobile device is located.

[0347] In an optional embodiment, the prompt animation includes a fan-shaped animation, wherein the fan-shaped animation includes a fan that dynamically expands from 0 radians to a target radian, the target radian being determined according to the rotation angle indicated by the in-place rotation operation, the display position of the vertex of the fan being determined according to the rotation center of the mobile device in-place rotation, and the radius of the fan being determined according to the display position and the display width of the mobile device in the second display direction.

[0348] This application also provides a mobile device, including a memory and a processor, wherein the processor executes the control method provided in this embodiment by calling a computer program stored in the memory.

[0349] For example, as shown in Figure 22, which illustrates the structural diagram of the mobile device involved in the embodiments of this application, specifically:

[0350] The mobile device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that the mobile device structure shown in FIG22 does not constitute a limitation on the mobile device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0351] The processor 601 is the control center of the mobile device. It connects various parts of the mobile device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, thereby controlling the mobile device as a whole. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, wheelchair user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.

[0352] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the removable device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0353] The mobile device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 is logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0354] The mobile device may also include an input unit 604, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to wheelchair occupant settings and function control.

[0355] Although not shown, the mobile device may also include a display unit, such as a display screen, which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the mobile device loads the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 601 runs the applications stored in the memory 602 to realize various functions, as follows:

[0356] Determine a globally feasible path from the current global location of the mobile device to the target global location in the global grid map. The globally feasible path includes multiple initial waypoints.

[0357] Determine the current local location of the mobile device within the real-time local grid map where the mobile device is located;

[0358] Identify unreachable path points in the real-time local grid map from multiple initial waypoints. Unreachable path points are path points that mobile devices have not reached.

[0359] Search for local feasible paths from the current local location to the corresponding unreachable path point in order of distance from each unreachable path point to the current local location, from farthest to closest.

[0360] When a locally feasible path is found from the current local location to the corresponding unreachable path point for the first time, the mobile device is controlled to move based on the locally feasible path, and the process returns to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0361] Optionally, a globally feasible path is determined in the global grid map from the current global location of the mobile device to the target global location, including:

[0362] Multiple candidate path points are identified in the global grid map. The candidate path points are grid cells in the global grid map that are in an idle state.

[0363] Determine the connectivity between each candidate path point and other candidate path points within a first preset range around it;

[0364] Among multiple candidate path points, determine the starting path point that is closest to the current global position, and the target path point that is closest to the target global position;

[0365] Based on connectivity, intermediate path points are determined from multiple candidate path points to form a feasible path from the starting path point to the target path point.

[0366] The starting path point, the target path point, and intermediate path points are determined as multiple initial path points, and the feasible path composed of multiple initial path points is determined as the globally feasible path from the current global position to the target global position.

[0367] Optionally, determining the connectivity between each candidate path point and candidate path points within a first preset range includes:

[0368] The connectivity between each candidate path point and its surrounding candidate path points within a first preset range is determined using a first preset algorithm.

[0369] Among multiple candidate pathpoints, determine the starting pathpoint closest to the current global position and the target pathpoint closest to the target global position, including:

[0370] The second preset algorithm determines the starting path point closest to the current global position and the target path point closest to the target global position from multiple candidate path points.

[0371] Based on connectivity, intermediate path points are determined from multiple candidate path points to form a feasible path from the starting path point to the target path point, including:

[0372] Using a second preset algorithm, intermediate path points are determined from multiple candidate path points based on connectivity to form a feasible path from the starting path point to the target path point.

[0373] Optionally, multiple candidate waypoints are identified in the global raster map, including:

[0374] Divide the global raster map into multiple raster regions;

[0375] In each grid area, idle grid cells are randomly sampled as candidate path points to obtain multiple candidate path points. Idle grid cells are grid cells in an idle state in the global grid map.

[0376] Optionally, after sequentially searching for locally feasible paths from the current local location to the corresponding unreachable path points in order of increasing distance from each unreachable path point to the current local location, the process further includes:

[0377] When no local feasible path is found from the current local location to each unreachable path point, obtain infeasibility reason information. The infeasibility reason information describes that there are unreachable path points in an occupied state and / or there are grids in an occupied state within a second preset range around the unreachable path points.

[0378] Based on the infeasibility information, update the global grid map and return to the step of determining a globally feasible path from the current global location of the mobile device to the target global location in the global grid map.

[0379] Optionally, when no locally feasible path is found from the current local location to each unreachable path point, information on the reasons for infeasibility is obtained, including:

[0380] When no locally feasible path from the current local location to each unreachable path point is found, search for a locally feasible path from the current local location to the unreachable path point closest to the current local location after a preset time.

[0381] If no locally feasible path is found from the current local location to the nearest unreachable path point, then obtain the reason why it is not feasible.

[0382] Optionally, returning to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located includes:

[0383] When the mobile device moves to the unreached waypoint that is furthest from the current local location, return to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0384] Optionally, returning to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located includes:

[0385] When the mobile device moves to a point in the real-time local grid map where the mobile device is located, and there is a new unreached path point, return to the step of determining the current local location of the mobile device in the real-time local grid map.

[0386] Optionally, according to the order of distance from each unreachable path point to the current local location from farthest to nearest, locally feasible paths from the current local location to the corresponding unreachable path point are searched sequentially, including:

[0387] Search for a locally feasible path from the current local location to the unreachable path point that is furthest from the current local location;

[0388] When no local feasible path is found from the current local location to the unreachable path point that is furthest from the current local location, the remaining unreachable path points are sequentially determined in order of their distance from the current local location, from farthest to closest, to determine whether the remaining unreachable path points are unreachable path points that the mobile device has not reached.

[0389] When the remaining unreachable path points are unreachable path points that the mobile device cannot reach, search for a locally feasible path from the current local location to the remaining unreachable path points.

[0390] Optionally, controlling the movement of mobile devices based on locally feasible paths includes:

[0391] Obtain the movement speed of the corresponding locally feasible path;

[0392] If the movement speed is less than 0, obtain the collision status of the mobile device rotating in place;

[0393] If the collision condition description states that rotating the mobile device in place will not result in a collision, then control the mobile device to rotate in place.

[0394] Optionally, after obtaining the collision status of the mobile device rotating in place, the following steps are also included:

[0395] If the collision condition description states that a collision would occur if the mobile device were to rotate in place, then control the mobile device to move backward.

[0396] Optionally, the collision status of the mobile device rotating in place is obtained, including:

[0397] When it is determined that if the mobile device is controlled to rotate in place, the grid that the mobile device passes through are all grids that are in an idle state in the real-time local grid map where the mobile device is located, it is determined that the collision situation description is that the mobile device will not cause a collision when rotating in place.

[0398] When it is determined that if the mobile device is controlled to rotate in place, the grids that the mobile device passes through are not all grids that are in an idle state in the real-time local grid map where the mobile device is located, it is determined that the collision condition description indicates that the mobile device will collide when rotating in place.

[0399] Optionally, after obtaining the movement speed of the corresponding locally feasible path, the method further includes:

[0400] If the movement speed is greater than 0, then based on the movement speed, control the mobile device to move towards an unreached path point that the mobile device has not yet reached.

[0401] Optionally, the control method further includes:

[0402] During the process of controlling the movement of a mobile device based on a locally feasible path, positive and negative obstacles in the environment where the mobile device is located are identified. Both the positive obstacles and the mobile device are located on the first side of the plane where the mobile device is located, and the negative obstacles are located on the second side of the plane.

[0403] Determine the relative display positions of positive obstacles, negative obstacles, and locally feasible paths relative to the mobile device;

[0404] Obtain the display methods corresponding to positive and negative obstacles respectively;

[0405] Based on the relative display position and display method, the system displays mobile devices, positive obstacles, negative obstacles, and paths that the mobile device has not yet traversed in the partially feasible paths.

[0406] This application also provides a mobile device, including a memory and a processor, wherein the processor executes the information display method provided in this embodiment by calling a computer program stored in the memory.

[0407] For example, as shown in Figure 23, which illustrates the structural diagram of the mobile device involved in the embodiments of this application, specifically:

[0408] The mobile device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that the mobile device structure shown in FIG23 does not constitute a limitation on the mobile device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0409] The processor 701 is the control center of the mobile device. It connects various parts of the mobile device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702, thereby controlling the mobile device as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, wheelchair user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701.

[0410] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the removable device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0411] The mobile device also includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 is logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0412] The mobile device may also include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to wheelchair occupant settings and function control.

[0413] Although not shown, the mobile device may also include a display unit, such as a display screen, which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the mobile device loads the executable files corresponding to the processes of one or more applications into the memory 702 according to the following instructions, and the processor 701 runs the applications stored in the memory 702 to realize various functions, as follows:

[0414] Identify positive and negative obstacles in the environment where the mobile device is located. Both positive obstacles and the mobile device are located on the first side of the plane where the mobile device is located, while negative obstacles are located on the second side of the plane.

[0415] Determine the first relative display positions of positive and negative obstacles relative to the mobile device;

[0416] Obtain the display methods corresponding to positive and negative obstacles respectively;

[0417] Based on the first relative display position and display method, the mobile device, positive obstacle, and negative obstacle are displayed.

[0418] Optionally, positive obstacles are displayed as solid lines, while negative obstacles are displayed as dashed lines.

[0419] Optionally, before displaying the mobile device, positive obstacle, and negative obstacle according to the first relative display position and display method, the method further includes:

[0420] Get the movement speed of the mobile device;

[0421] Determine the movement trajectory of the mobile device based on its movement speed;

[0422] Obtain the second relative display position of the movement trajectory relative to the mobile device;

[0423] Based on the first relative display position and display method, the movable device, positive obstacle, and negative obstacle are displayed, including:

[0424] Based on the first relative display position, the second relative display position, and the display method, the mobile device, positive obstacle, negative obstacle, and movement trajectory are displayed.

[0425] Optionally, before determining the movement trajectory of the mobile device relative to the second relative display position of the mobile device based on the movement speed, the method further includes:

[0426] Determine the display position of the mobile device based on its movement speed;

[0427] Based on the moving speed, determine the moving trajectory of the mobile device relative to the second relative display position of the mobile device, including:

[0428] Based on the moving speed and display position, determine the moving trajectory of the mobile device relative to the second relative display position of the mobile device;

[0429] Based on the first relative display position, the second relative display position, and the display method, the movable device, positive obstacle, negative obstacle, and movement trajectory are displayed, including:

[0430] Based on the display position, the first relative display position, the second relative display position, and the display method, the mobile device, positive obstacle, negative obstacle, and movement trajectory are displayed.

[0431] Optionally, the display width of the movable device and the movement trajectory is the same in the first display direction.

[0432] Optionally, after displaying the mobile device, positive obstacle, and negative obstacle according to the first relative display position and display method, the method further includes:

[0433] Receive input for in-situ rotation operation;

[0434] In response to a spin-in-place operation, a prompt animation is displayed to indicate when the mobile device is being rotated in place.

[0435] Control the mobile device to rotate in place, and while the mobile device is rotating in place, return to perform the step of identifying positive and negative obstacles in the environment in which the mobile device is located.

[0436] Optionally, the prompt animation includes a fan-shaped animation, wherein the fan-shaped animation includes a fan that dynamically expands from 0 radians to a target radian, the target radian being determined according to the rotation angle indicated by the in-place rotation operation, the display position of the vertex of the fan being determined according to the rotation center of the mobile device in-place rotation, and the radius of the fan being determined according to the display position and the display width of the mobile device in the second display direction.

[0437] This application also provides a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the control methods provided in this application. For example, the instructions can execute the following steps:

[0438] In the global grid map, determine a globally feasible path from the current global location of the mobile device to the target global location. The globally feasible path includes multiple initial path points. Determine the current local location of the mobile device in the real-time local grid map where the mobile device is located. Among the multiple initial path points, determine the unreachable path points in the real-time local grid map. Unreachable path points are path points that the mobile device has not yet reached. Search for local feasible paths from the current local location to the corresponding unreachable path points in order of distance from each unreachable path point to the current local location, from farthest to closest. When a local feasible path from the current local location to the corresponding unreachable path point is found for the first time, control the movement of the mobile device based on the local feasible path, and return to execute the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

[0439] This application also provides a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the information display methods provided in this application. For example, the instructions can execute the following steps:

[0440] Identify positive and negative obstacles in the environment where the mobile device is located. Both the positive obstacle and the mobile device are located on the first side of the plane where the mobile device is located, and the negative obstacle is located on the second side of the plane. Determine the first relative display position of the positive and negative obstacles relative to the mobile device. Obtain the display mode corresponding to the positive and negative obstacles respectively. Display the mobile device, the positive obstacle, and the negative obstacle according to the first relative display position and the display mode.

[0441] The aforementioned storage media may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0442] Since the instructions stored in the aforementioned storage medium can execute the steps of any of the control methods or information display methods provided in the embodiments of this application, the beneficial effects that any of the control methods or information display methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the preceding embodiments, which will not be repeated here.

[0443] According to one aspect of this application, a computer program content or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a mobile device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the mobile device to perform the control method or information display method provided in the above-described embodiments.

[0444] It should be noted that this application does not restrict the execution order of each step, and the execution order of each step can be adjusted according to actual needs.

[0445] It should also be noted that the descriptions of the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to other embodiments. This application will not repeat them here.

[0446] The control method, information display method, device, and mobile device provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A control method, comprising: In a global grid map, a globally feasible path is determined from the current global location of the mobile device to the target global location, the globally feasible path including multiple initial path points; Determine the current local position of the mobile device in the real-time local grid map where the mobile device is located; Among the multiple initial waypoints, an unreachable waypoint in the real-time local grid map is determined, and the unreachable waypoint is a waypoint that the mobile device has not reached. According to the order of distance from each unreachable path point to the current local position from farthest to closest, search for local feasible paths from the current local position to the corresponding unreachable path point. When a locally feasible path from the current local location to the corresponding unreachable path point is found for the first time, the mobile device is controlled to move based on the locally feasible path, and the process returns to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

2. The control method according to claim 1, wherein, Determining a globally feasible path from the current global location of a mobile device to the target global location in a global grid map includes: Multiple candidate path points are determined in the global grid map, and the candidate path points are grids in the global grid map that are in an idle state; Determine the connectivity between each candidate path point and the candidate path points within a first preset range around it; Among the candidate path points, determine the starting path point that is closest to the current global position, and the target path point that is closest to the target global position; Based on the connectivity relationship, intermediate path points are determined from among the multiple candidate path points to form a feasible path from the starting path point to the target path point; The starting path point, the target path point, and the intermediate path point are determined as multiple initial path points, and the feasible path composed of the multiple initial path points is determined as a globally feasible path from the current global position to the target global position.

3. The control method according to claim 2, wherein, Determining the connectivity between each candidate path point and other candidate path points within a first preset range includes: The connectivity between each candidate path point and the candidate path points within a first preset range is determined by a first preset algorithm. The step of determining the starting path point closest to the current global position and the target path point closest to the target global position from among the multiple candidate path points includes: Using a second preset algorithm, the starting path point closest to the current global position and the target path point closest to the target global position are determined from among the multiple candidate path points; The step of determining intermediate path points from among the candidate path points to form a feasible path from the starting path point to the target path point based on the connectivity relationship includes: Using the second preset algorithm, based on the connectivity relationship, intermediate path points are determined from among the multiple candidate path points to form a feasible path from the starting path point to the target path point.

4. The control method according to claim 2, wherein, The step of determining multiple candidate path points in the global grid map includes: The global raster map is divided into multiple raster regions; In each of the grid regions, idle grid cells are randomly sampled as candidate path points to obtain multiple candidate path points. The idle grid cells are grid cells in the global grid map that are in an idle state.

5. The control method according to claim 1, wherein, After sequentially searching for locally feasible paths from the current local location to the corresponding unreachable path point in order of increasing distance from the current local location, the method further includes: When no local feasible path is found from the current local location to each of the unreachable path points, obtain infeasibility reason information. The infeasibility reason information describes that there are unreachable path points in an occupied state and / or there are grids in an occupied state within a second preset range around the unreachable path points. Based on the infeasibility information, update the global grid map and return to the step of determining a globally feasible path from the current global location of the mobile device to the target global location in the global grid map.

6. The control method according to claim 5, wherein, When no locally feasible path is found from the current local location to each of the unreachable path points, the infeasibility reason information is obtained, including: When no local feasible path is found from the current local location to each of the unreachable path points, a local feasible path is searched from the current local location to the unreachable path point closest to the current local location after a preset time period. If no local feasible path is found from the current local location to the nearest unreachable path point, then obtain the reason information for infeasibility.

7. The control method according to claim 1, wherein, The step of returning to determine the current local position of the mobile device in the real-time local grid map where the mobile device is located includes: When the mobile device moves to the unreached path point that is furthest from the current local location, the process returns to the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

8. The control method according to claim 1, wherein, The step of returning to determine the current local position of the mobile device in the real-time local grid map where the mobile device is located includes: When the mobile device moves to a point in the real-time local grid map where the mobile device is located and there is a new unreached path point, return to the step of determining the current local position of the mobile device in the real-time local grid map where the mobile device is located.

9. The control method according to claim 1, wherein, The step of sequentially searching for locally feasible paths from the current local location to the corresponding unreachable path point, in order of increasing distance from each unreachable path point to the current local location, includes: Search for a local feasible path from the current local location to the unreachable path point that is furthest from the current local location; When no local feasible path is found from the current local location to the unreachable path point that is furthest from the current local location, the remaining unreachable path points are sequentially determined to be unreachable path points that the mobile device has not reached, in order of their distance from the current local location from farthest to closest. When the remaining unreachable path point is an unreachable path point that the mobile device has not reached, search for a locally feasible path from the current local location to the remaining unreachable path point.

10. The control method according to claim 1, wherein, The control of the mobile device's movement based on the locally feasible path includes: Obtain the movement speed corresponding to the locally feasible path; If the moving speed is less than 0, then obtain the collision status of the mobile device rotating in place; If the collision condition describes that rotating the mobile device in place will not cause a collision, then control the mobile device to rotate in place.

11. The control method according to claim 10, wherein, After obtaining the collision status of the mobile device rotating in place, the method further includes: If the collision condition describes a situation where rotating the mobile device in place would result in a collision, then the mobile device is controlled to move backward.

12. The control method according to claim 11, wherein, The step of obtaining the collision status of the mobile device rotating in place includes: When it is determined that if the mobile device is controlled to rotate in place, all the grids that the mobile device passes through are grids that are in an idle state in the real-time local grid map where the mobile device is located, it is determined that the collision situation describes that the mobile device will not cause a collision when rotating in place. When it is determined that if the mobile device is controlled to rotate in place, the grids that the mobile device passes through are not all grids that are in an idle state in the real-time local grid map where the mobile device is located, it is determined that the collision condition describes that the mobile device will cause a collision when it rotates in place.

13. The control method according to claim 10, wherein, After obtaining the movement speed corresponding to the locally feasible path, the method further includes: If the moving speed is greater than 0, then based on the moving speed, the mobile device is controlled to move towards an unreached path point that the mobile device has not yet reached.

14. The control method according to claim 1, wherein, The method further includes: During the process of controlling the movement of the mobile device based on the local feasible path, positive obstacles and negative obstacles in the environment where the mobile device is located are identified. The positive obstacles and the mobile device are both located on the first side of the plane where the mobile device is located, and the negative obstacles are located on the second side of the plane. Determine the relative display positions of the positive obstacle, the negative obstacle, and the locally feasible path relative to the mobile device; Obtain the display modes corresponding to the positive obstacles and the negative obstacles respectively; Based on the relative display position and the display method, the mobile device, the positive obstacle, the negative obstacle, and the path that the mobile device has not traversed in the locally feasible path are displayed.

15. An information display method, comprising: Identify positive and negative obstacles in the environment where the mobile device is located. The positive obstacles and the mobile device are both located on a first side of the plane where the mobile device is located, and the negative obstacles are located on a second side of the plane. Determine the first relative display positions of the positive obstacle and the negative obstacle relative to the mobile device; Obtain the display modes corresponding to the positive obstacles and the negative obstacles respectively; Based on the first relative display position and the display method, the mobile device, the positive obstacle, and the negative obstacle are displayed.

16. The information display method according to claim 15, wherein, The positive obstacle is displayed as a solid line, and the negative obstacle is displayed as a dashed line.

17. The information display method according to claim 15, wherein, Before displaying the mobile device, the positive obstacle, and the negative obstacle according to the first relative display position and the display mode, the method further includes: Obtain the moving speed of the mobile device; Based on the moving speed, determine the moving trajectory of the mobile device relative to the second relative display position of the mobile device; The step of displaying the mobile device, the positive obstacle, and the negative obstacle according to the first relative display position and the display mode includes: Based on the first relative display position, the second relative display position, and the display mode, the mobile device, the positive obstacle, the negative obstacle, and the movement trajectory are displayed.

18. The information display method according to claim 17, wherein, Before determining the movement trajectory of the mobile device relative to the second relative display position of the mobile device based on the movement speed, the method further includes: The display position of the mobile device is determined based on the moving speed; Determining the movement trajectory of the mobile device relative to the second relative display position of the mobile device based on the movement speed includes: Based on the moving speed and the display position, determine the moving trajectory of the mobile device relative to the second relative display position of the mobile device; The step of displaying the mobile device, the positive obstacle, the negative obstacle, and the movement trajectory according to the first relative display position, the second relative display position, and the display mode includes: Based on the display position, the first relative display position, the second relative display position, and the display mode, the mobile device, the positive obstacle, the negative obstacle, and the movement trajectory are displayed.

19. The information display method according to claim 17, wherein, The display width of the mobile device and the movement trajectory is the same in the first display direction.

20. The information display method according to claim 15, wherein, After displaying the mobile device, the positive obstacle, and the negative obstacle according to the first relative display position and the display mode, the method further includes: Receive input for in-situ rotation operation; In response to the in-place rotation operation, a prompt animation is displayed, which is used to prompt the in-place rotation of the mobile device; Control the mobile device to rotate in place, and during the rotation of the mobile device, return to the step of identifying positive and negative obstacles in the environment in which the mobile device is located.

21. The information display method according to claim 20, wherein, The prompt animation includes a fan-shaped animation, wherein the fan-shaped animation includes a fan that dynamically expands from 0 radians to a target radian, the target radian being determined according to the rotation angle indicated by the in-place rotation operation, the display position of the vertex of the fan being determined according to the rotation center of the mobile device in-place rotation, and the radius of the fan being determined according to the display position and the display width of the mobile device in the second display direction.

22. A control device, comprising: The path determination module is used to determine a globally feasible path from the current global location of the mobile device to the target global location in the global grid map, wherein the globally feasible path includes multiple initial path points; The location determination module is used to determine the current local location of the mobile device in the real-time local grid map where the mobile device is located; A waypoint determination module is used to determine, from among a plurality of initial waypoints, an unreachable waypoint in the real-time local grid map, wherein the unreachable waypoint is a waypoint that the mobile device has not reached. The path search module is used to sequentially search for local feasible paths from the current local location to the corresponding unreachable path point in order of distance from the current local location to the current local location, from farthest to closest. The device control module is used to control the mobile device to move based on the local feasible path when a local feasible path from the current local location to the corresponding unreachable path point is found for the first time, and return to execute the step of determining the current local location of the mobile device in the real-time local grid map where the mobile device is located.

23. An information display device, comprising: An obstacle recognition module is used to identify positive and negative obstacles in the environment where the mobile device is located. The positive obstacles and the mobile device are both located on the first side of the plane where the mobile device is located, and the negative obstacles are located on the second side of the plane. The position determination module is used to determine the first relative display of the positive obstacle and the negative obstacle relative to the mobile device. Location; The mode acquisition module is used to acquire the display modes corresponding to the positive obstacles and the negative obstacles, respectively. The information display module is used to display the mobile device, the positive obstacle, and the negative obstacle according to the first relative display position and the display mode.

24. A mobile device comprising a memory and a processor; the memory storing a computer program, and the processor executing the computer program in the memory to perform the control method of claim 1.

25. A mobile device comprising a memory and a processor; the memory storing a computer program, and the processor executing the computer program in the memory to perform the information display method of claim 15.

Images