Map construction method, control system and self-moving device

Abstract

The embodiment of the invention discloses a map construction method, a control system and a self-moving device, which are applied to the self-moving device carrying a sensor. In the process of controlling the self-moving device to move from a mapping starting point to a mapping terminal point, a point cloud map is generated by using point cloud data obtained by scanning of a sensor carried by the self-moving device, the point cloud data comprises point cloud data obtained by scanning of a cargo container transportation tool, and then a map for planning is obtained. And planning a path in the cargo container transportation tool. According to the mode, the map for planning is created on the self-moving equipment, so that a basis is provided for path planning of the self-moving equipment. Even if the parking positions and the parking postures of the cargo container transportation tools have certain differences, accurate mapping can be carried out on the cargo container transportation tools. A sensor carried by self-moving equipment is used for scanning and mapping, and compared with a traditional mode, construction difficulty, calibration cost and maintenance cost are reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to a map construction method, a control system and a self-moving device. Background Art

[0002] Systems that use self-moving devices such as AGV (automated guided vehicle) have the advantages of being highly unmanned, automated, and intelligent, which improves production efficiency and operational levels for industries such as warehousing, manufacturing, and logistics. As one of the more typical scenarios, self-moving devices are often responsible for the handling tasks of cargo container transportation tools such as container trucks and container ships, mainly including loading / unloading, such as moving cargo in the temporary storage area to the container truck compartment, or moving cargo in the container truck compartment to the temporary storage area.

[0003] Due to the limited space of cargo container transport vehicles, in order to avoid collisions, there are high requirements for the positioning accuracy of self-moving equipment. However, given that the docking position and docking posture of cargo container transport vehicles are different each time, in order to realize the path planning for self-moving equipment, how to build a planning map has become an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present application provides a map construction method and device to facilitate the construction of a planning map, thereby providing a basis for path planning for self-mobile equipment.

[0005] This application provides the following solutions:

[0006] In a first aspect, a map construction method is provided, which is applied to a self-moving device equipped with a sensor, and the method is executed by a controller provided in the self-moving device, comprising:

[0007] In response to a mapping task instruction, controlling the mobile device to move from a mapping starting point to a mapping end point, wherein the mapping end point is located in a loading space of a cargo container transportation vehicle;

[0008] During the movement, a point cloud map is generated using point cloud data scanned by the sensor, wherein the point cloud data includes point cloud data scanned for the cargo container transport vehicle;

[0009] A planning map is obtained based on the point cloud map, and the planning map is used to plan a path within the cargo container transportation vehicle.

[0010] Optionally, obtaining a planning map according to the point cloud map includes:

[0011] Extracting point cloud data within the operating height range of the cargo container transportation vehicle from the point cloud map;

[0012] The intercepted point cloud data is projected onto the ground plane to obtain the planning map.

[0013] Optionally, projecting the intercepted point cloud data onto a ground plane to obtain the planning map includes:

[0014] Projecting the intercepted point cloud data onto the ground plane and performing binarization processing to obtain an initial map;

[0015] The initial map is rasterized to obtain the planning map, wherein the rasterization includes: dividing the initial map into a plurality of grids, and determining the state of each grid according to the projection relationship between the point cloud data and each grid, wherein the state includes being occupied or idle.

[0016] Optionally, the mapping task instruction includes: a static mapping path and a dynamic mapping path;

[0017] In response to the mapping task instruction, controlling the mobile device to move from the mapping starting point to the mapping end point includes:

[0018] According to the static mapping path, controlling the self-moving device to move from the mapping starting point to a target position, wherein the target position is within a preset range from the cargo container transportation vehicle;

[0019] According to the dynamic mapping path, the self-moving device is controlled to move from the target position to the mapping end point.

[0020] Optionally, during the moving process, generating a point cloud map using point cloud data scanned by the sensor includes:

[0021] In the process of controlling the mobile device to move from the mapping starting point to the target location, acquiring the point cloud data scanned by the sensor and generating a first point cloud map using a SLAM module;

[0022] In the process of controlling the mobile device to move from the target position to the mapping end point, the point cloud data scanned by the sensor is obtained and a second point cloud map is generated using a SLAM module.

[0023] Optionally, acquiring the point cloud data scanned by the sensor and generating a first point cloud map using a SLAM module includes:

[0024] The SLAM module is used to splice the key frames in the point cloud data scanned by the sensor with the static point cloud map loaded when the mobile device is initialized to obtain the first point cloud map.

[0025] Optionally, the method further comprises:

[0026] The first point cloud map and the second point cloud map are spliced ​​to obtain a third point cloud map, and the third point cloud map is used to determine the position and posture of the self-moving device when the self-moving device performs a transport task.

[0027] Optionally, the method further comprises:

[0028] According to the point cloud map and the cargo size information, the storage location posture information in the loading space of the cargo container transportation vehicle is determined.

[0029] Optionally, the method further comprises:

[0030] The planning map and the storage location position information are sent to a control system so that the control system plans a path within the cargo container transportation vehicle according to the planning map and the storage location position information.

[0031] Optionally, in response to the mapping task instruction, the step of controlling the mobile device to move from the mapping starting point to the mapping end point is performed by a system module;

[0032] The step of generating a point cloud map by using the point cloud data obtained by scanning the cargo container transportation vehicle with the sensor is performed by a map service module;

[0033] The step of obtaining a planning map according to the point cloud map is performed by the map service module;

[0034] Wherein, the system module and the graph service module both run on the controller.

[0035] Optionally, the controller also runs a perception service module;

[0036] After receiving the mapping task instruction, the system module sends a mapping start instruction to the map service module via the perception service module to execute the step of controlling the mobile device to move from the mapping starting point to the mapping end point;

[0037] In response to the map building start instruction, the map service module executes the step of generating a point cloud map by using the sensor to obtain the point cloud data from scanning the cargo container transportation vehicle.

[0038] Optionally, the method further comprises:

[0039] In response to the mobile device moving to the mapping end point, the system module sends a mapping end instruction to the mapping service module via the perception service module;

[0040] In response to the map creation end instruction, the map service module stops generating the point cloud map and executes the step of obtaining a planning map based on the point cloud map.

[0041] Optionally, the method further comprises:

[0042] In response to the mapping end instruction, the map service module provides the point cloud map to the perception service module;

[0043] The perception service module determines the storage location posture information in the loading space according to the point cloud map, and returns the storage location posture information in the loading space to the system module in response to the request of the system module.

[0044] Optionally, the method further comprises:

[0045] After the map service module stops generating the point cloud map, it sends a successful stop map status to the system module via the perception service module.

[0046] Optionally, the method further comprises:

[0047] In response to receiving the stop map building success status, the system module requests a planning map from the map service module;

[0048] In response to the request, the map service module returns the planning map to the system module.

[0049] Optionally, the method further comprises:

[0050] In response to receiving the storage location posture information, the system module sends a task idle instruction to the perception service module;

[0051] In response to receiving the task idle instruction, the perception service module clears the intermediate data generated by determining the storage location posture information, and notifies the map service module to stop providing the point cloud map.

[0052] Optionally, in response to the request, the map service module returns the planning map to the system module, including:

[0053] In response to the request, if the map service module has completed the generation of the planning map, the planning map is returned to the system module;

[0054] If the map service module has not completed the generation of the planning map, it returns an incomplete status to the system module. After receiving the incomplete status, the system module waits for a preset period of time and then requests the planning map from the map service module again.

[0055] Optionally, the map service module providing the point cloud map to the perception service module comprises:

[0056] The map service module publishes the generated point cloud map through the ECAL publishing method, and provides the published subject information to the perception service module for subscription;

[0057] The perception service module obtains the point cloud map according to the subject information.

[0058] In a second aspect, a control system is provided, including a controller and a memory, the memory being used to store program instructions, and the controller executing the program instructions to implement the steps of any one of the methods described in the first aspect.

[0059] In a third aspect, a self-moving device is provided, comprising a controller and a memory, wherein the memory is used to store program instructions, and the controller executes the program instructions to implement the steps of any one of the methods described in the first aspect.

[0060] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0061] 1) In this application, in the process of controlling the self-mobile device to move from the mapping starting point to the mapping end point, the point cloud data obtained by scanning the sensor carried by the self-mobile device (including the point cloud data obtained by scanning the cargo container transportation vehicle) is used to generate a point cloud map, and then a planning map is obtained to plan the path within the cargo container transportation vehicle. This method realizes the creation of a planning map on the self-mobile device, thereby providing a basis for path planning for the self-mobile device.

[0062] This method enables the mobile device to intelligently, efficiently and accurately obtain the external and internal environmental information of the cargo container transport vehicle, and accurately and clearly express the internal storage location information of the cargo container transport vehicle in the planning map. Even if the docking position and docking posture of the cargo container transport vehicle are different each time, they can be accurately mapped, thus providing a basis for the efficient and accurate execution of subsequent handling tasks.

[0063] 2) Since the sensors carried by the self-moving device are used for scanning and mapping, compared with the traditional method of placing external sensors at the bridge, the construction difficulty and maintenance cost are reduced, and the calibration of the relationship between the external sensor and the two coordinate systems in the self-moving device is avoided. Therefore, the implementation is more convenient and the cost is lower.

[0064] 3) After intercepting the preset spatial range of the point cloud data and projecting it onto the ground plane, the application performs binarization processing to obtain an initial map, and further performs rasterization processing on the initial map to obtain a two-dimensional planning map. The planning map can clearly determine the location of obstacles and environmental information, thereby making the path planning of subsequent handling tasks more convenient and accurate.

[0065] 4) This application realizes scanning and mapping using sensors carried by self-mobile devices through specific interactions and processing between system modules, perception modules and map service modules running on self-mobile devices, and generates two-dimensional planning maps using the generated point cloud maps, thereby realizing the creation of planning maps and providing a basis for path planning for self-mobile devices.

[0066] 5) In this application, the first point cloud map formed in the process from the starting position to the target position and the second point cloud map formed in the process from the target position to the mapping end point are spliced ​​to obtain a complete third point cloud map. The complete third point cloud map can determine the position and posture of the self-mobile equipment during the execution of the transportation task, thereby ensuring that the self-mobile equipment accurately and efficiently performs the transportation task.

[0067] 6) In the present application, point cloud data can be used to identify the position information of the storage location in the loading space of the cargo container transport vehicle and provide it to the system module, so that the mobile device can accurately locate the storage location in the loading space during the subsequent execution of the handling task, and assist the control system in planning the path for the handling task.

[0068] 7) This application realizes scanning and mapping using sensors carried by self-mobile devices through specific interactions and processing between system modules, perception modules and map service modules running on self-mobile devices, and generates two-dimensional planning maps using the generated point cloud maps, thereby realizing the creation of planning maps and providing a basis for path planning for self-mobile devices.

[0069] 8) This application uses the ECAL mechanism as a method of inter-process communication between the perception service module and the graph service module, and particularly uses ECAL to publish the generated point cloud map, and provides the published subject information to the perception service module for subscription. This method can significantly reduce data exchange delays and bandwidth usage, and is particularly suitable for the transmission of large files such as point cloud maps between processes in the embodiments of this application, thereby improving the performance of self-mobile equipment.

[0070] 9) In this application, after the graph service module stops generating the point cloud map, it sends the map building success status to the system module via the perception service module. The system module then requests the planning map from the graph service module, thereby effectively ensuring the integrity of the planning map.

[0071] 10) In this application, after receiving the storage location posture information, the system module promptly notifies the perception service module, so that the perception service module can clear the intermediate data and notify the graph service module to stop providing point cloud data, thereby effectively avoiding waste of resources and improving the performance of the self-moving device.

[0072] Of course, any invention of the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1 is a system architecture diagram applicable to the embodiments of the present application;

[0075] Figure 2 A flowchart of a map construction method provided in an embodiment of the present application;

[0076] Figure 3 A schematic diagram of a path for controlling movement of a mobile device provided in an embodiment of the present application;

[0077] Figure 4 A detailed flow chart of the interactions between the modules provided in the embodiments of the present application;

[0078] Figure 5 A schematic block diagram of a self-moving device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0080] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0081] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0082] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0083] Since the space of cargo container transport vehicles is limited, in order to avoid collisions, there are high requirements for the positioning accuracy of self-moving equipment. However, since the docking position and docking posture of cargo container transport vehicles are different each time, it is impossible to pre-establish a map for cargo container transport vehicles. Among the existing implementation methods, some require the installation of external sensors near the cargo container transport vehicles, and scan the cargo container transport vehicles to establish a map every time the cargo container transport vehicles arrive. However, this method has at least the following disadvantages:

[0084] 1) To ensure the accuracy of mapping, the external sensor needs to fully scan the cargo container transport vehicle, especially the internal environment of the cargo container transport vehicle, such as the interior of the carriage. Therefore, it is usually placed at the bridge between the edge of the outer platform and the cargo container transport vehicle. The bridge is also the location where mobile equipment enters and exits the cargo container transport vehicle, and passage space needs to be left. This makes it difficult to install the external sensor and the construction requirements are extremely high.

[0085] 2) Since the map scanned by the external sensor needs to be spliced ​​with the static map of the self-moving device, it is necessary to calibrate the coordinate transformation relationship between the two maps, and the calibration accuracy must be very high, otherwise it will cause the self-moving device to collide inside the cargo container transportation vehicle, such as colliding with the wall of the container truck.

[0086] 3) External sensors require additional power cables and network cables to connect, and are exposed to a semi-open outdoor environment. In order to avoid the influence of wind, rain and sun, external sensors need to be protected and maintained over a long period of time, which brings additional maintenance costs.

[0087] In view of this, the present application provides a new idea. In order to facilitate the understanding of the present application, the system architecture on which the present application is based is first described. Figure 1An exemplary system architecture to which the embodiments of the present application can be applied is shown. Figure 1 As shown in , the system architecture may include: a self-moving device and a self-moving device control system (RCS).

[0088] Among them, the self-moving device refers to a tool equipped with an electromagnetic or optical automatic guidance device, controlled by a computer, with its own power or power conversion device and capable of automatically moving along a specified path. In the embodiment of the present application, the self-moving device can be able to move and undertake certain tasks, such as transportation, handling (including picking up and unloading goods), etc. For example, the automatic moving device can be an AGV (Automated Guided Vehicle), a logistics robot, an unmanned forklift, etc.

[0089] In an embodiment of the present application, a self-mobile device is installed with an on-board software system, which has a SLAM (Simultaneous Localization and Mapping) function, a map generation function for generating a planning map based on a mapping result, a motion control function for controlling the movement and operation of the self-mobile device, etc.

[0090] The mobile device control system (RCS) can be set up on the server side and is responsible for scheduling each mobile device, including task allocation, route planning, and instruction issuance.

[0091] The above self-equipment device control system can be set on an independent server, a server group, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in a cloud computing service system to solve the defects of difficult management and weak service scalability in traditional physical hosts and virtual private servers (VPS) services. In addition, the above self-equipment device control system can also be set on a computer terminal with strong computing power.

[0092] It should be understood that Figure 1 The number of self-propelled devices and self-propelled device control systems (RCS) in the embodiment is only illustrative. Any number of self-propelled devices and control systems may be provided according to implementation requirements.

[0093] Figure 2 A flowchart of a map construction method provided in an embodiment of the present application. The method can be performed by Figure 1 The system is performed by a self-moving device equipped with a sensor. Figure 2 As shown in , the method can be executed by a controller set in a self-mobile device, and the method can include the following steps:

[0094] Step 201: In response to a mapping task instruction, control a mobile device to move from a mapping start point to a mapping end point, where the mapping end point is located in a loading space of a cargo container transportation vehicle.

[0095] Step 203: During the movement, point cloud data obtained by scanning with a sensor is used to generate a point cloud map, where the point cloud data includes point cloud data obtained by scanning a cargo container transport vehicle.

[0096] Step 205: A planning map is obtained based on the point cloud map, and the planning map is used to plan the path within the cargo container transportation vehicle.

[0097] It can be seen from the above process that in the process of controlling the self-moving device to move from the starting point to the end point of the mapping, the point cloud data obtained by scanning the sensor carried by the self-moving device (including the point cloud data obtained by scanning the cargo container transportation vehicle) is used to generate a point cloud map using the point cloud data, and then a planning map is obtained to plan the path within the cargo container transportation vehicle. This method enables the self-moving device to intelligently, efficiently and accurately obtain the external and internal environmental information of the cargo container transportation vehicle, realizes the creation of a planning map, and thus provides a basis for path planning for the self-moving device.

[0098] Since the sensors carried by the self-moving device are used for scanning and mapping, compared with the traditional method of placing external sensors at the bridge, the construction difficulty and maintenance cost are reduced, and the calibration of the relationship between the external sensor and the two coordinate systems in the self-moving device is avoided. Therefore, it is more convenient to implement and the cost is lower.

[0099] The following is a detailed description of each step in the above process and the effects that can be further produced in conjunction with the embodiments. It should be noted that the "first" and "second" limitations involved in this disclosure do not have limitations in terms of size, order, and quantity, and are only used to distinguish them in name. For example, "first point cloud map", "second point cloud map" and "third point cloud map" are only used to distinguish each point cloud map in name.

[0100] First, the above step 201, namely "in response to a mapping task instruction, controlling the mobile device to move from the mapping starting point to the mapping end point", is described in detail in conjunction with an embodiment.

[0101] When the cargo container transport vehicle stops at the designated location (usually the platform location), the staff can trigger the terminal device to send a task start instruction to the RCS. For example, when the staff determines that the cargo container transport vehicle stops at the platform location, the terminal device is triggered to send a task start instruction to the RCS by pressing a physical or virtual button on the terminal device or inputting a command. In addition to being triggered by the staff, other methods can also be used.

[0102] After receiving the task start instruction, RCS generates a mapping task. The process of generating a mapping task is mainly the process of path planning, that is, planning the path from the mapping start point to the mapping end point based on the location information of the cargo container transportation vehicle and the work map.

[0103] In the above path from the mapping start point to the mapping end point, the mapping start point is usually the starting position of the self-moving device. When the self-moving device is not performing work, it is usually located at its own parking position, which can be a location such as a warehouse or a carport. Therefore, the above starting position can be a parking position.

[0104] As one of the feasible ways, the mapping end point can be a certain position in the loading space of the cargo container transportation vehicle, which is referred to as the first position in the embodiment of the present application. This situation is often used in loading scenarios. In loading scenarios, the loading space of the cargo container transportation vehicle is usually relatively empty, and the self-mobile device can move into the loading space to scan the internal environment.

[0105] As another feasible method, the mapping end point can be a second position within the first distance from the cargo container transport vehicle, which is often used in unloading scenarios. In unloading scenarios, the loading space of the cargo container transport vehicle is usually full of cargo, and the self-mobile device cannot enter the loading space. Usually, scanning is performed at the boarding bridge position or a position between the boarding bridge position and the cargo container transport vehicle.

[0106] In an embodiment of the present application, a mapping task instruction may include a static mapping path and a dynamic mapping path. The static mapping path is the path from the starting position to the target position. The dynamic mapping path is the path from the target position to the mapping end point, wherein the target position is within the second distance of the container transport vehicle, wherein the first distance is less than or equal to the second distance. For example, the target position can be set to the position for boarding the container transport vehicle. For example, if the cargo container transport vehicle is a container truck, the target position may be the boarding bridge position. For another example, if the cargo container transport vehicle is a container ship, the target position may be the boarding bridge position. For another example, if the cargo container transport vehicle is a flying wing vehicle, the target position may be any position near the flying wing vehicle.

[0107] Static mapping paths are usually obtained based on static map planning to indicate the movement of a mobile device from a starting location to a boarding bridge location, such as Figure 3 The blue path L1 shown in the figure, the dotted circle in the figure represents the partial trajectory position of the self-moving device on the path. The dynamic mapping path is usually pre-set to indicate that the self-moving device enters the loading space of the cargo container transportation vehicle from a boarding bridge position, such as Figure 3The red path L2 is shown in FIG. Since there is no accurate map of the cargo container transport vehicle at the beginning, the dynamic mapping path can be pre-set with low accuracy requirements, and can be pre-set according to the approximate parking position and size of the cargo container transport vehicle, as long as it is ensured that there is no collision with the cargo container transport vehicle.

[0108] In the embodiment of the present application, the self-mobile device can be controlled to move from the mapping starting point to the target position according to the static mapping path; and the self-mobile device can be controlled to move from the target position to the mapping end point according to the dynamic mapping path.

[0109] The above step 203, namely "generating a point cloud map using point cloud data obtained by scanning with a sensor during movement" is described in detail below in conjunction with an embodiment.

[0110] In the embodiment of the present application, the sensor carried by the mobile device includes radars such as laser radar, millimeter wave radar, ToF (time of flight) radar, etc., and may further include visual sensors such as cameras.

[0111] In the process of controlling the self-mobile device to move from the starting point of the map to the target position, the point cloud data scanned by the sensor can be obtained and the first point cloud map can be generated by using the SLAM module. In the process of controlling the self-mobile device to move from the target position to the end point of the map, the point cloud data scanned by the sensor can be obtained and the second point cloud map can be generated by using the SLAM module.

[0112] Specifically, when controlling the movement of the mobile device according to the static mapping path contained in the mapping task instruction, the Local SLAM algorithm can be used for positioning. Among them, the Local SLAM algorithm may include: splicing the point cloud data of the latest N key frames collected in real time with the static point cloud map to obtain a first point cloud map, which can be regarded as a map obtained by aligning the latest N key frames collected on the static point cloud map. Then use an algorithm such as GICP (Generalized Iterative Closest Point) to calculate the relative pose of the point cloud of the current frame relative to the first point cloud map, so as to determine the current pose. In view of the fact that the Local SLAM algorithm is an existing algorithm, it will not be described in detail here.

[0113] When controlling the self-moving device to move along the dynamic mapping path included in the mapping task instruction, it moves from the target position and enters the loading space of the cargo container transportation vehicle. During the movement, the self-moving device uses the onboard sensor to scan. During the whole process, the outside and inside of the cargo container transportation vehicle can be scanned, thereby establishing a dynamic point cloud map of the cargo container transportation vehicle area, which is referred to as the second point cloud map in this embodiment of the application.

[0114] The first point cloud map and the second point cloud map are then spliced ​​together to obtain a third point cloud map. The third point cloud map, as a full point cloud map, can be used to determine the position and posture of the self-moving device during the self-moving device performs a handling task.

[0115] The above step 205, i.e., "obtaining a planning map based on the point cloud map, and using the planning map to determine the storage location information in the loading space" is described in detail below in conjunction with an embodiment.

[0116] In an embodiment of the present application, point cloud data within the operating height range of the cargo container transport vehicle can be intercepted from the point cloud map; the intercepted point cloud data is projected onto the ground plane to obtain a planning map.

[0117] The operating height range of the above-mentioned cargo container transport vehicle can be an empirical value, for example, a range of 0.2 meters to 2.5 meters according to the usual height of the cargo container transport vehicle. It can also be set according to actual scenarios.

[0118] The planning map is a two-dimensional map. After the intercepted point cloud data is projected onto the ground plane, it is further binarized to obtain an initial map, and then the initial map is rasterized to obtain a raster map as the planning map. The rasterization process may include dividing the initial map into multiple grids, and determining the state of each grid, including occupied state or idle state, based on the projection relationship between the point cloud data and each grid.

[0119] If the grid is occupied, it is determined that there is an obstacle in the grid and it is impassable; if the grid is not occupied, it is determined that there is no obstacle in the grid and it is passable. The state of the grid can be determined based on the number of points in the point cloud projected to the grid. For example, if the number of points in the point cloud projected to the grid exceeds a preset number threshold, the grid is determined to be occupied, otherwise it is determined to be idle. The preset number threshold can be determined based on experience and / or safety requirements.

[0120] In addition to raster maps, planning maps can also use other forms of maps, such as dot density maps.

[0121] Planning maps can be in image formats, such as PNG (Portable Network Graphics), BMP (Bitmap) and other lossless compression image formats.

[0122] Furthermore, based on the above point cloud map, the position information of the warehouse location in the loading space of the cargo container transportation vehicle can be determined, and the position information of the warehouse location is used to locate the warehouse location in the loading space during the transportation process of the self-mobile device. For example, the internal environment of the loading space of the cargo container transportation vehicle can be sensed from the point cloud map, and the position of the warehouse location in the loading space can be determined using the sensed internal environment of the loading space and information such as the cargo size.

[0123] In an embodiment of the present application, the planning map and the storage location position information may be sent to the RCS so that the RCS can plan a path within the cargo container transport vehicle based on the planning map and the storage location position information.

[0124] As one of the achievable ways, the above method flow can be executed by a module running on a controller. In the embodiment of the present application, the controller can be a processor or a control chip in a mobile device. Among them, the processor can be a general-purpose CPU, a microprocessor, one or more integrated circuits, etc.

[0125] The implementation of the above method is described below by taking the system module, perception service module and graph service module running on the controller as an example. In addition to the above module division method, other granular module division methods can also be used, as long as similar functions are completed, they are within the protection scope of this application.

[0126] Figure 4 A detailed flow chart of the interactions between the modules provided in the embodiment of the present application is shown in FIG. Figure 4 As shown in , the interaction process specifically includes the following steps:

[0127] Step 401: The system module receives a mapping task instruction from the RCS.

[0128] Step 402: The system module sends a map building start instruction to the map service module via the perception service module.

[0129] It should be noted that the system module, perception service module and graph service module involved in the embodiments of the present application can be presented in the form of processes when running in the self-mobile device, that is, the running system process, perception service process and graph service process. Then the system module, perception service module and graph service module essentially use inter-process communication to exchange instructions or data.

[0130] As one of the feasible ways, after receiving the mapping start instruction, the perception service module can send the mapping start instruction to the graph service module by means of ECAL (Enhanced Communication Abstraction Layer) RPC (Remote Procedure Call) to notify the graph service module to start the mapping task. Specifically, the perception service module can act as an ECAL Server (server) to send the mapping start instruction to the graph service module as an ECALClient (client).

[0131] In addition to ECAL RPC, other inter-process communication mechanisms or middleware interaction protocols may also be used, such as MQTT (Message Queuing Telemetry Transport).

[0132] Step 403: The graph service module returns status information indicating that normal graph construction has started to be started to the system module via the perception service module.

[0133] After receiving the map building start instruction, the map service module starts the map building task, such as starting the corresponding service process, and performs relevant verification. After the verification passes, it enters the state of starting normal map building and returns the state information of starting normal map building.

[0134] Step 404: After receiving the normal mapping state start information, the system module controls the mobile device to move from the mapping start point to the mapping end point.

[0135] At the same time, in step 405, the map service module obtains point cloud data scanned by the sensor carried by the mobile device, and generates a point cloud map using the point cloud data.

[0136] As described in the previous embodiment, the mapping task instruction may include two path information: a static mapping path from a starting position to a target position (e.g., a boarding bridge position) and a dynamic mapping path from the target position to a mapping end point. Accordingly, the system module controls the mobile device to move from the starting position to the target position, and then from the target position to the mapping end point.

[0137] In the process of controlling the movement of the self-mobile device according to the static mapping path, the map service module obtains the point cloud data scanned by the sensor and generates a first point cloud map using the SLAM module. In the process of controlling the movement of the self-mobile device according to the dynamic mapping path, the point cloud data scanned by the sensor (essentially point cloud image frames) is obtained and a second point cloud map is generated using the SLAM module.

[0138] Specifically, when the system module controls the movement of the mobile device according to the static mapping path contained in the mapping task instruction, it uses the Local SLAM algorithm for positioning. Among them, the Local SLAM algorithm may include: splicing the point cloud of the latest N key frames collected in real time and the static point cloud map to obtain a first point cloud map, which can be regarded as a map obtained by aligning the latest N key frames collected on the static point cloud map. Then use an algorithm such as GICP (Generalized Iterative Closest Point) to calculate the relative pose of the point cloud of the current frame relative to the first point cloud map, so as to determine the current pose. In view of the fact that the Local SLAM algorithm is an existing algorithm, it will not be described in detail here.

[0139] The system module controls the mobile device to move from the target position along the dynamic mapping path contained in the mapping task instruction and enter the loading space of the cargo container transportation vehicle. During the movement, the mobile device uses the onboard sensor to scan. During the whole process, the outside and inside of the cargo container transportation vehicle can be scanned. The map service module establishes a dynamic point cloud map of the cargo container transportation vehicle area, which is referred to as the second point cloud map in this embodiment of the application.

[0140] Step 406: In response to the mobile device moving to the mapping end point, the system module sends a mapping end instruction to the map service module via the perception service module.

[0141] As one of the feasible ways, after receiving the mapping end instruction, the perception service module can send the mapping end instruction to the graph service module by means of ECAL RPC to notify the graph service module to stop the mapping task. Specifically, the perception service module can act as an ECAL Server and send the mapping end instruction to the graph service module acting as an ECAL Client.

[0142] In addition to ECAL RPC, other inter-process communication mechanisms can also be used.

[0143] Step 407: In response to the mapping end instruction, the map service module stops generating the point cloud map, sends a successful stop mapping status to the perception service module, and provides the point cloud map to the perception service module.

[0144] As one of the feasible ways, the graph service module may provide the second point cloud map to the perception service module to generate a planning map.

[0145] Furthermore, the first point cloud map and the second point cloud map can be spliced ​​to obtain a third point cloud map. The third point cloud map can be used as a full point cloud map to determine the position and posture of the self-moving device during the self-moving device performing the transport task.

[0146] The static point cloud map is a point cloud map loaded by the mobile device when it is started, covering the working area of ​​the mobile device. In the embodiment of the present application, the above-mentioned working area mainly refers to areas such as warehouses, platforms, temporary storage areas, etc., and may include the location for boarding a cargo container transportation vehicle, but generally does not include areas outside the location for boarding a cargo container transportation vehicle (for example, the area for boarding a container transportation vehicle, the area of ​​the loading space of the container transportation vehicle, etc.).

[0147] As one of the feasible ways, when the graph service module provides the point cloud map (for example, the second point cloud map) to the perception service module, it can publish the generated point cloud map through ECAL PUB (publishing) and provide the published topic (topic information) to the perception service module for subscription. The perception service module obtains the point cloud map based on the topic.

[0148] Among them, ECAL PUB is the core component in the ECAL communication architecture, which provides an efficient communication mechanism, supports shared memory for local communication, and allows different processes to directly access the same physical memory area. In an embodiment of the present application, the perception service module acts as a subscriber, and the graph service module acts as a publisher. The publisher specifies a topic, and the subscriber receives data through shared memory based on this topic. This method can significantly reduce the delay and bandwidth occupancy of data exchange, and is particularly suitable for large files such as point cloud maps in the embodiment of the present application to be transmitted between processes, which improves the performance of self-mobile devices.

[0149] Step 408: The perception service module sends a successful stop mapping status to the system module.

[0150] Step 409: The perception service module uses the point cloud map to determine the storage location and posture information of the loading space of the cargo container transportation vehicle.

[0151] In an embodiment of the present application, the perception service module can perceive the internal environmental conditions of the loading space of the cargo container transport vehicle from the point cloud map, and determine the position of the loading space using the perceived internal environmental conditions of the loading space and information such as the cargo size.

[0152] Step 410: In response to receiving the stop mapping success status, the system module requests the storage location pose from the perception service module.

[0153] Step 411: The perception service module returns the storage location and posture information of the loading space of the cargo container transportation vehicle to the system module.

[0154] If the perception service module has not determined the warehouse position and posture information after receiving the request, it will return a non-complete status to the system module. After receiving the non-complete status, the system module can wait for a certain period of time and request the warehouse position and posture from the perception service module again until the perception service module determines the warehouse position and posture information and successfully returns the warehouse position and posture information to the system module.

[0155] In an embodiment of the present application, the storage position information of the loading space of the cargo container transport vehicle is stored locally on the self-moving device, so that the self-moving device can locate the storage position of the loading space of the cargo container transport vehicle during the execution of subsequent handling tasks.

[0156] Step 412: After stopping generating the point cloud, the map service module uses the point cloud map to obtain a planning map.

[0157] In an embodiment of the present application, the map service module can intercept point cloud data within the operating height range of the cargo container transportation vehicle from the second point cloud map, project the intercepted point cloud data onto the ground plane, and obtain a planning map.

[0158] Step 413: In response to receiving the status of successfully stopping map building, the system module requests a planning map from the map service module.

[0159] Step 414: In response to the request, the map service module returns the planning map to the system module.

[0160] If the map service module has not generated the planning map after receiving the request, it will return an incomplete status to the system module. After receiving the incomplete status, the system module can wait for a certain period of time and request the planning map from the map service module again until the map service module completes the generation of the planning map and successfully returns the planning map to the system module.

[0161] It should be noted here that the above-mentioned steps 409 to 411 on the sub-process of obtaining the storage location posture information and steps 412 to 414 on the sub-process of obtaining the planning map are executed in parallel, and therefore can be executed simultaneously or in any order. The order shown in the figure is only one of the ways.

[0162] Step 415: In response to receiving the storage location posture information, the system module sends a task idle instruction to the perception service module.

[0163] Step 416: In response to receiving the task idle instruction, the perception service module clears the intermediate data generated by identifying the storage location posture information, and notifies the map service module to stop providing the point cloud map.

[0164] Step 417: The map service module stops providing the point cloud map.

[0165] After receiving the notification, the map service module determines that the planning map has been generated and stops providing the point cloud map, for example, stops publishing the generated point cloud map through ECAL PUB.

[0166] In the embodiment of the present application, after the system module receives the planning map and the storage location posture information, the planning map and the storage location posture information can be sent to the RCS, so that the RCS can use the planning map and the storage location posture information to plan the path in the cargo container transportation vehicle to generate a handling task, and send a handling task instruction to the self-mobile device that completes the handling task. The handling task instruction contains the path information for executing the handling task obtained based on the planning map and the storage location posture information. It can be seen that the method for generating the planning map provided in the embodiment of the present application provides a basis for the self-mobile device to perform the handling task.

[0167] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0168] According to an embodiment of another aspect, a self-moving device is provided. Figure 5 A schematic diagram of the composition of a self-moving device provided in an embodiment of the present application, such as Figure 5 As shown in , the self-moving device includes: a self-moving device body 510, a sensor 520 mounted on the self-moving device body, a memory 530 and a controller 540.

[0169] The sensor 520 is used to scan the cargo container transport vehicle.

[0170] The memory 530 is used to store program instructions.

[0171] The controller 540 is coupled to the memory 530 and is used to read the program instructions stored in the memory 530 to perform the following processing:

[0172] In response to a mapping task instruction, control the mobile device to move from a mapping starting point to a mapping end point, where the mapping end point is located in a loading space of a cargo container transportation vehicle;

[0173] During the movement, point cloud data obtained by scanning with a sensor is used to generate a point cloud map, wherein the point cloud data includes point cloud data obtained by scanning a cargo container transport vehicle;

[0174] The point cloud map is used to obtain a planning map, and the planning map is used to plan the path within the cargo container transportation vehicle.

[0175] In some embodiments, when the controller 540 uses the point cloud map to obtain the planning map, it can specifically perform the following operations:

[0176] Extract point cloud data within the operating height range of the cargo container transportation vehicle from the point cloud map;

[0177] The intercepted point cloud data is projected onto the ground plane to obtain a planning map.

[0178] In some embodiments, when the controller 540 projects the intercepted point cloud data onto the ground plane to obtain a planning map, the controller 540 may specifically execute:

[0179] Project the intercepted point cloud data onto the ground plane and perform binarization processing to obtain the initial map;

[0180] The initial map is rasterized to obtain a planning map. The rasterization includes: dividing the initial map into multiple grids, and determining the status of each grid based on the projection relationship between the point cloud data and each grid. The status includes being occupied or idle.

[0181] In some embodiments, the mapping task instruction may include: a static mapping path and a dynamic mapping path; accordingly, when the controller 540 controls the mobile device to move from the mapping starting point to the mapping end point in response to the mapping task instruction, the controller 540 may specifically execute:

[0182] According to the static mapping path, the mobile device is controlled to move from the mapping starting point to the target location, and the target location is within the preset range of the cargo container transportation vehicle;

[0183] According to the dynamic mapping path, the mobile device is controlled to move from the target position to the mapping end point.

[0184] In some embodiments, when the controller 540 generates a point cloud map using point cloud data scanned by the sensor during movement, the controller 540 may specifically perform the following steps:

[0185] In the process of controlling the mobile device to move from the mapping starting point to the target position, the point cloud data scanned by the sensor is obtained and a first point cloud map is generated using the SLAM module;

[0186] In the process of controlling the mobile device to move from the target position to the mapping end point, the point cloud data scanned by the sensor is obtained and the second point cloud map is generated using the SLAM module.

[0187] In some embodiments, when the controller 540 acquires the point cloud data scanned by the sensor and generates the first point cloud map using the SLAM module, the following steps may be specifically performed:

[0188] By using the SLAM module, the key frames in the point cloud data scanned by the sensor are spliced ​​with the static point cloud map loaded when the mobile device is initialized to obtain the first point cloud map.

[0189] In some embodiments, the controller 540 may further execute: splicing the first point cloud map and the second point cloud map to obtain a third point cloud map, and the third point cloud map is used to determine the posture of the self-moving device during the self-moving device performs a transport task.

[0190] In some embodiments, the controller 540 may further execute: determining the storage location posture information in the loading space of the cargo container transportation vehicle according to the point cloud map and the cargo size information.

[0191] In some embodiments, the controller 540 may further execute: sending the planning map and the storage location position information to the control system, so that the control system plans the path within the cargo container transportation vehicle according to the planning map and the storage location position information.

[0192] In some embodiments, the sensor 520 may include radars such as lidar, millimeter wave radar, ToF (time of flight) radar, etc., and may further include visual sensors such as cameras.

[0193] In some embodiments, the controller 540 may be one or more, and may be implemented by a general-purpose CPU, a microcontroller, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing the above program instructions to implement the technical solution provided in this application.

[0194] The memory 530 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc.

[0195] It should be noted that, although the self-mobile device only shows the self-mobile device body 510, the sensor 520, the memory 530 and the controller 540, etc., in the specific implementation process, the self-mobile device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the self-mobile device may also only include the components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0196] According to another embodiment, a control system is provided, which is arranged in a self-moving device, and the self-moving device is equipped with a sensor. The control system may include:

[0197] The memory is configured to store point cloud data scanned by the sensor, point cloud maps generated by the controller, and planning maps.

[0198] The controller is coupled to the memory and is configured to execute the steps described in the above method embodiment. For example: in response to a mapping task instruction, the mobile device is controlled to move from a mapping starting point to a mapping end point, and the mapping end point is located in the loading space of the cargo container transportation vehicle; during the movement, the point cloud data obtained by scanning the cargo container transportation vehicle by a sensor is used to generate a point cloud map; the point cloud map is used to obtain a planning map, and the planning map is used to determine the storage location information in the loading space.

[0199] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0200] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0201] It can be seen from the above description of the implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can essentially be embodied in the form of a computer program product, which can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc.

[0202] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A map construction method, applied to a self-moving device equipped with a sensor, characterized in that: The method is executed by a controller disposed in the mobile device, and includes: In response to a mapping task instruction, controlling the mobile device to move from a mapping starting point to a mapping end point, wherein the mapping end point is located in a loading space of a cargo container transportation vehicle; During the movement, a point cloud map is generated using point cloud data scanned by the sensor, wherein the point cloud data includes point cloud data scanned for the cargo container transport vehicle; A planning map is obtained based on the point cloud map, and the planning map is used to plan a path within the cargo container transportation vehicle.

2. The method according to claim 1, characterized in that The obtaining of a planning map according to the point cloud map comprises: Extracting point cloud data within the operating height range of the cargo container transportation vehicle from the point cloud map; The intercepted point cloud data is projected onto the ground plane to obtain the planning map.

3. The method according to claim 2, characterized in that The step of projecting the intercepted point cloud data onto the ground plane to obtain the planning map comprises: Projecting the intercepted point cloud data onto the ground plane and performing binarization processing to obtain an initial map; The initial map is rasterized to obtain the planning map, wherein the rasterization includes: dividing the initial map into a plurality of grids, and determining the state of each grid according to the projection relationship between the point cloud data and each grid, wherein the state includes being occupied or idle.

4. The method according to claim 1, characterized in that The mapping task instructions include: a static mapping path and a dynamic mapping path; In response to the mapping task instruction, controlling the mobile device to move from the mapping starting point to the mapping end point includes: According to the static mapping path, controlling the self-moving device to move from the mapping starting point to a target position, wherein the target position is within a preset range from the cargo container transportation vehicle; According to the dynamic mapping path, the self-moving device is controlled to move from the target position to the mapping end point.

5. The method according to claim 4, characterized in that In the process of the movement, generating a point cloud map using the point cloud data scanned by the sensor includes: In the process of controlling the mobile device to move from the mapping starting point to the target location, acquiring the point cloud data scanned by the sensor and generating a first point cloud map using a SLAM module; In the process of controlling the mobile device to move from the target position to the mapping end point, the point cloud data scanned by the sensor is obtained and a second point cloud map is generated using a SLAM module.

6. The method according to claim 5, characterized in that Acquiring the point cloud data scanned by the sensor and generating a first point cloud map using a SLAM module includes: The SLAM module is used to splice the key frames in the point cloud data scanned by the sensor with the static point cloud map loaded when the mobile device is initialized to obtain the first point cloud map.

7. The method according to claim 5 or 6, characterized in that: The method further comprises: The first point cloud map and the second point cloud map are spliced ​​to obtain a third point cloud map, and the third point cloud map is used to determine the position and posture of the self-moving device when the self-moving device performs a transport task.

8. The method according to claim 1, characterized in that The method further comprises: According to the point cloud map and the cargo size information, the storage location posture information in the loading space of the cargo container transportation vehicle is determined.

9. The method according to claim 8, characterized in that The method further comprises: The planning map and the storage location position information are sent to a control system so that the control system plans a path within the cargo container transportation vehicle according to the planning map and the storage location position information.

10. The method according to claim 1, characterized in that The step of controlling the mobile device to move from the mapping starting point to the mapping end point in response to the mapping task instruction is performed by the system module; The step of generating a point cloud map by using the point cloud data obtained by scanning the cargo container transportation vehicle with the sensor is performed by a map service module; The step of obtaining a planning map according to the point cloud map is performed by the map service module; Wherein, the system module and the graph service module both run on the controller.

11. The method according to claim 10, characterized in that The controller also runs a perception service module; After receiving the mapping task instruction, the system module sends a mapping start instruction to the map service module via the perception service module to execute the step of controlling the mobile device to move from the mapping starting point to the mapping end point; In response to the map building start instruction, the map service module executes the step of generating a point cloud map by using the sensor to obtain the point cloud data from scanning the cargo container transportation vehicle.

12. The method according to claim 11, characterized in that The method further comprises: In response to the mobile device moving to the mapping end point, the system module sends a mapping end instruction to the mapping service module via the perception service module; In response to the map creation end instruction, the map service module stops generating the point cloud map and executes the step of obtaining a planning map based on the point cloud map.

13. The method according to claim 12, characterized in that The method further comprises: In response to the mapping end instruction, the map service module provides the point cloud map to the perception service module; The perception service module determines the storage location posture information in the loading space according to the point cloud map, and returns the storage location posture information in the loading space to the system module in response to the request of the system module.

14. The method according to claim 12, characterized in that The method further comprises: After the map service module stops generating the point cloud map, it sends a successful stop map status to the system module via the perception service module.

15. The method according to claim 14, characterized in that The method further comprises: In response to receiving the stop map building success status, the system module requests a planning map from the map service module; In response to the request, the map service module returns the planning map to the system module.

16. The method according to claim 13, characterized in that The method further comprises: In response to receiving the storage location posture information, the system module sends a task idle instruction to the perception service module; In response to receiving the task idle instruction, the perception service module clears the intermediate data generated by determining the storage location posture information, and notifies the map service module to stop providing the point cloud map.

17. The method according to claim 15, characterized in that In response to the request, the map service module returns the planning map to the system module, including: In response to the request, if the map service module has completed the generation of the planning map, the planning map is returned to the system module; If the map service module has not completed the generation of the planning map, it returns an incomplete status to the system module. After receiving the incomplete status, the system module waits for a preset period of time and then requests the planning map from the map service module again.

18. The method according to claim 13, characterized in that The map service module provides the point cloud map to the perception service module, including: The map service module publishes the generated point cloud map through the ECAL publishing method, and provides the published subject information to the perception service module for subscription; The perception service module obtains the point cloud map according to the subject information.

19. A control system, characterized in that: The invention comprises a controller and a memory, wherein the memory is used to store program instructions, and the controller executes the program instructions to implement the method according to any one of claims 1 to 18.

20. A self-propelled device, characterized in that: The invention comprises a controller and a memory, wherein the memory is used to store program instructions, and the controller executes the program instructions to implement the method according to any one of claims 1 to 18.

Images