Model processing method, storage medium and self-moving device

Abstract

The embodiment of the invention provides a model processing method, a storage medium and self-moving equipment. The method comprises the steps that scanning data of at least one image acquisition element on a target scene is acquired, and the at least one image acquisition element is arranged on a first self-moving device and has a set pose relation; determining pose data of at least one image acquisition element; creating a three-dimensional map model corresponding to the target scene according to the scanning data and the pose data; and converting the three-dimensional map model into a positioning map model corresponding to a set image acquisition element on the second self-moving equipment. According to the method and the device, the three-dimensional map model created by the first self-moving equipment can be repeatedly utilized, and the second self-moving equipment does not need to repeatedly establish a map for the target scene, so that the working efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a model processing method, a storage medium, and a self-moving device. Background Art

[0002] Three-dimensional reconstruction technology is an important component of mobile robot autonomous navigation, location environment model reconstruction, large-scale digital monitoring, etc.

[0003] In the related art, a single RGBD camera is generally used to perform three-dimensional reconstruction of the indoor environment. However, the single RGBD camera has a limited viewing angle and cannot guarantee the accuracy of the created three-dimensional model. Moreover, after the first mobile robot creates the three-dimensional model based on the RGBD camera, if the positioning device of the second mobile robot is different from the RGBD camera of the first mobile robot, for example, the positioning device on the second mobile robot is a laser radar, then in the same indoor environment, the second mobile robot cannot use the three-dimensional model that the first mobile robot has created, and needs to recreate the three-dimensional model of the indoor environment by itself, which is inefficient. Summary of the invention

[0004] Multiple aspects of the present application provide a model processing method, a storage medium and a self-mobile device, which are used to achieve the reuse of a three-dimensional map model created by a first self-mobile device, avoid repeated mapping of a target scene by a second self-mobile device, and improve work efficiency.

[0005] The present application provides a model processing method, the method comprising:

[0006] Acquire scanning data of a target scene by at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship;

[0007] Determining position and posture data of the at least one image acquisition element;

[0008] Creating a three-dimensional map model corresponding to the target scene according to the scan data and the posture data;

[0009] The three-dimensional map model is converted into a positioning map model corresponding to a set image acquisition component on the second mobile device.

[0010] In an optional embodiment, determining the position and posture data of the at least one image acquisition element includes:

[0011] Determine the computing power requirements of each image acquisition component;

[0012] Based on the computing power requirement, determine the posture data of the at least one image acquisition element.

[0013] In an optional embodiment, the at least one image acquisition element includes a first image acquisition element and a second image acquisition element, and determining the position and posture data of the at least one image acquisition element based on the computing power requirement includes:

[0014] If the sum of the computing power requirements of the first image acquisition component and the second image acquisition component is greater than a set threshold, obtaining the position and posture data of the first image acquisition component in the world coordinate system;

[0015] Based on the position and posture data of the first image acquisition element in the world coordinate system and the position and posture relationship, the position and posture data of the second image acquisition element in the world coordinate system is determined.

[0016] In an optional embodiment, determining the position data of the at least one image acquisition element based on the computing power requirement includes:

[0017] If the sum of the computing power requirements of the first image acquisition component and the second image acquisition component is less than or equal to a set threshold, the position and posture data of the first image acquisition component and the second image acquisition component in the world coordinate system are respectively obtained.

[0018] In an optional embodiment, creating a three-dimensional map model corresponding to the target scene according to the scan data and the posture data includes:

[0019] Creating a first three-dimensional map model based on the scan data and the position and posture data of the first image acquisition element;

[0020] Creating a second three-dimensional map model based on the scan data and the position and posture data of the second image acquisition element;

[0021] The three-dimensional map model is determined according to the first three-dimensional map model and the second three-dimensional map model.

[0022] In an optional embodiment, converting the three-dimensional map model into a positioning map model corresponding to the component information of the set image acquisition component on the second mobile device includes:

[0023] Determining a corresponding model conversion method according to a set image acquisition component on the second mobile device;

[0024] The three-dimensional map model is model-converted based on the model conversion method to obtain a positioning map model corresponding to the set image acquisition component on the second self-mobile device.

[0025] In an optional embodiment, the method further includes:

[0026] An obstacle avoidance map model is created according to the three-dimensional map model and the device information of the second self-moving device, and the obstacle avoidance map model is used to enable the second self-moving device to avoid obstacles in the target scene.

[0027] In an optional embodiment, the scan data includes: depth data and color data of multiple frames corresponding to the target scene.

[0028] The embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors are caused to perform at least the following actions:

[0029] Acquire scanning data of a target scene by at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship;

[0030] Determining position and posture data of the at least one image acquisition element;

[0031] Creating a three-dimensional map model corresponding to the target scene according to the scan data and the posture data;

[0032] The three-dimensional map model is converted into a positioning map model corresponding to a set image acquisition component on the second mobile device.

[0033] The embodiment of the present application also provides a self-moving device, the self-moving device comprising: a device body, the device body being provided with a memory, a processor and at least one image acquisition element;

[0034] The memory is used to store computer programs;

[0035] The processor is coupled to the memory and is used to execute the computer program to perform the above-mentioned model processing method.

[0036] In an embodiment of the present application, by acquiring the scanning data of the target scene by at least one image acquisition element, the richness and integrity of the acquired image data are ensured. By determining the posture data of at least one image acquisition element, and based on the scanning data and posture data, a three-dimensional map model corresponding to the target scene can be created, thereby ensuring the accuracy of the three-dimensional map model. After the three-dimensional map model is created, by converting the three-dimensional map model into a positioning map model corresponding to the set image acquisition element on the second self-mobile device, the second self-mobile device can be moved according to the positioning map model, thereby realizing the reuse of the three-dimensional map model created by the first self-mobile device, eliminating the need for the second self-mobile device to repeatedly map the target scene, thereby improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0038] Figure 1 A flowchart of a model processing method provided for an exemplary embodiment of the present application;

[0039] Figure 2 A schematic diagram of a process for determining the position and posture data of at least one image acquisition element provided by an exemplary embodiment of the present application;

[0040] Figure 3 A schematic diagram of a model conversion process of a three-dimensional map model provided by an exemplary embodiment of the present application;

[0041] Figure 4 Another flowchart of a model processing method provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0043] Three-dimensional reconstruction technology is an important component of mobile robot autonomous navigation, location environment model reconstruction, large-scale digital monitoring, etc. At present, a single RGBD camera is generally used to perform three-dimensional reconstruction of the indoor environment. However, the viewing angle of a single RGBD camera is limited, and the accuracy of the created three-dimensional model cannot be guaranteed. In addition, after the first mobile robot creates the three-dimensional model based on the RGBD camera, if the positioning device of the second mobile robot is different from the RGBD camera of the first mobile robot, for example, the positioning device on the second mobile robot is a laser radar, then in the same indoor environment, the second mobile robot cannot use the three-dimensional model that the first mobile robot has created, and needs to recreate the three-dimensional model of the indoor environment by itself, which has low work efficiency. In view of this, an embodiment of the present application provides a model processing method.

[0044] Figure 1 A flow chart of a model processing method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0045] Step 101: Acquire scanning data of a target scene from at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship.

[0046] The first self-moving device may be any mechanical device that can move autonomously in its environment, for example, a robot, a purifier, an unmanned vehicle, etc. The robot may include a sweeping robot, a glass cleaning robot, a home care robot, a welcoming robot, an autonomous service robot, etc. The target scene may be a classroom scene, a room scene, a factory scene, a shopping mall scene, etc., which are not listed here one by one.

[0047] The image acquisition element may be an RGBD camera, and the number may be one or more, but considering the cost and the use effect, the present application preferably has two. In practical applications, if the number of RGBD cameras is two, the positions of the two RGBD cameras on the first self-mobile device are generally pre-set, that is, the positions and postures of the two RGBD cameras on the first self-mobile device are known, that is, the relative position relationship (i.e., posture relationship) between the two is also known. For the positions of the two RGBD cameras, for example, one RGBD camera may be set at the bottom of the first self-mobile device, and the other RGBD camera may be set at the top of the first self-mobile device, and the two may cooperate with each other to fully collect the image data of the target scene. The scanning data of the target scene by the two RGBD cameras includes: multiple frames of depth data and color data corresponding to the target scene.

[0048] Step 102: Determine the position and posture data of at least one image acquisition element.

[0049] It should be noted that, at this time, the position and posture data of at least one image acquisition element is determined as the position and posture data of at least one image acquisition element in the world coordinate system.

[0050] Step 103: Create a three-dimensional map model corresponding to the target scene based on the scan data and the posture data.

[0051] In this embodiment, assuming that the at least one image acquisition element includes a first image acquisition element and a second image acquisition element, the steps of creating a three-dimensional map model corresponding to the target scene are as follows:

[0052] Creating a first three-dimensional map model based on the scan data and the position and posture data of the first image acquisition component;

[0053] Creating a second three-dimensional map model based on the scan data and the position and posture data of the second image acquisition component;

[0054] A three-dimensional map model is determined according to the first three-dimensional map model and the second three-dimensional map model.

[0055] In specific implementation, after acquiring the scanning data of the target scene through the first image acquisition element and determining the posture data of the first image acquisition element in the world coordinate system, a first three-dimensional map model can be created based on the scanning data and posture data of the first image acquisition element based on the TSDF (Truncated Signed Distance Function) algorithm.

[0056] Similarly, after acquiring the scanning data of the target scene through the second image acquisition element and determining the posture data of the second image acquisition element in the world coordinate system, a second three-dimensional map model can be created based on the TSDF algorithm according to the scanning data and posture data of the second image acquisition element.

[0057] After obtaining the first three-dimensional map model and the second three-dimensional map model, the first three-dimensional map model and the second three-dimensional map model can be spliced ​​by the PSDF algorithm based on the external parameter data (known data) of the first image acquisition element and the second image acquisition element to obtain the final three-dimensional map model. Among them, PSDF (Probabilistic Signed Distance Function) is a probability-based signed distance function representation method used for three-dimensional reconstruction and surface representation. Similar to TSDF (Truncated Signed Distance Function), PSDF also divides the three-dimensional space into a regular voxel grid and assigns a signed distance value to each voxel. The main difference between PSDF and TSDF is that it assigns a probability value to each voxel, indicating the uncertainty of the distance from the voxel to the surface of the object. In specific implementation, the scanning data and posture data of the first image acquisition element and the scanning data and posture data of the second image acquisition element are input into PSDF to complete the data fusion of the first three-dimensional map model and the second three-dimensional map model, and then the final three-dimensional map model is generated based on the external parameter data (known data) of the first image acquisition element and the second image acquisition element. Compared with TSDF, PSDF can handle noise and uncertainty better, thus achieving higher quality 3D reconstruction.

[0058] Step 104: convert the three-dimensional map model into a positioning map model corresponding to the set image acquisition component on the second mobile device.

[0059] In the specific implementation, for ease of understanding, the solution of the present application is described below with the number of image acquisition components being two:

[0060] Assume that the first self-mobile device is moving in the target scene. During this process, the target scene is scanned by two image acquisition components on the first self-mobile device to obtain two color images and two depth images of the target scene. In order to improve the quality and accuracy of the image, the two color images and the two depth images can be denoised.

[0061] Afterwards, the visual SLAM algorithm is used to determine the position data (including position data and attitude data) of the two image acquisition components in the world coordinate system. Since the first self-moving device is in motion, the position data of the two image acquisition components in the world coordinate system obtained at this time can be understood as the trajectory data of the two image acquisition components.

[0062] After obtaining the color image, depth image and pose data of the target scene, 3D reconstruction can be performed based on the TSDF algorithm to create a 3D map model corresponding to the target scene. It should be understood that after obtaining the final 3D map model, in order to make the 3D map model more realistic, the 3D map model can be smoothed and refined, and 3D texture mapping and rendering can be performed on the 3D map model.

[0063] It should be understood that at this time, the first self-mobile device has already created a three-dimensional map model of the target scene based on two RGBD cameras. However, if the second self-mobile device is also arranged to work in the target scene at this time, and the image acquisition unit on the second self-mobile device is a laser radar (different from the RGBD camera on the first self-mobile device), then the second self-mobile device cannot directly use the three-dimensional map model that has been created by the first self-mobile device. It needs to recreate it by itself, which is time-consuming and laborious. In this regard, the present application will convert the three-dimensional map model into a positioning map model corresponding to the set image acquisition element (i.e., the above-mentioned laser radar) on the second self-mobile device, so that the second self-mobile device moves according to the positioning map model. The specific conversion can be seen in the subsequent embodiments and will not be described in detail for the time being.

[0064] Based on the above, the model processing method provided in the embodiment of the present application ensures the richness and completeness of the collected image data by obtaining the scanning data of the target scene by multiple image acquisition components. By determining the posture data of at least one image acquisition component, and based on the scanning data and posture data, a three-dimensional map model corresponding to the target scene can be created, thereby ensuring the accuracy of the three-dimensional map model. After the three-dimensional map model is created, by converting the three-dimensional map model into a positioning map model corresponding to the set image acquisition component on the second self-mobile device, the second self-mobile device can be moved according to the positioning map model, thereby realizing the reuse of the three-dimensional map model created by the first self-mobile device, eliminating the need for the second self-mobile device to repeatedly build a map for the target scene, thereby improving work efficiency.

[0065] Figure 2 A schematic diagram of a process for determining the position and posture data of at least one image acquisition element, such as Figure 2 As shown, determining the position data of at least one image acquisition element includes:

[0066] Step 201: Determine the computing power requirement of each image acquisition component.

[0067] Step 202: Determine the position and posture data of at least one image acquisition element based on computing power requirements.

[0068] In actual applications, the processor computing power of different self-moving devices may also be different. Some self-moving devices have stronger processor computing power, while others have weaker processor computing power. Therefore, when determining the position data of at least one image acquisition component, the present application first needs to determine the computing power requirement of each image acquisition component, and then determine whether the processor computing power of the self-moving device corresponding to the image acquisition component is sufficient before performing subsequent operations.

[0069] Specifically, assuming that at least one image acquisition element includes a first image acquisition element and a second image acquisition element, if the sum of the computing power requirements of the first image acquisition element and the second image acquisition element is greater than a set threshold, it means that the computing power of the processor of the self-mobile device is relatively weak, and the pose data of the first image acquisition element and the second image acquisition element in the world coordinate system cannot be determined simultaneously. The pose data of the first image acquisition element in the world coordinate system is directly obtained through the visual SLAM algorithm, and then the pose data of the second image acquisition element in the world coordinate system is determined based on the pose data of the first image acquisition element in the world coordinate system and the pose relationship between the first image acquisition element and the second image acquisition element. Among them, the pose relationship between the first image acquisition element and the second image acquisition element can be determined by calibration using a camera calibration method after determining the positions and postures of the two on the first self-mobile device.

[0070] If the sum of the computing power requirements of the first image acquisition element and the second image acquisition element is less than or equal to the set threshold, it means that the processor computing power of the self-mobile device is strong, and the position data of the first image acquisition element and the second image acquisition element in the world coordinate system can be determined separately through the visual SLAM algorithm.

[0071] Based on the above, by determining the computing power requirements of each image acquisition element, and then determining the posture data of at least one image acquisition element based on the computing power requirements, computing power can be allocated according to the actual situation of the self-mobile device, thereby avoiding data determination errors due to shortage of processor computing power in the self-mobile device, or abnormal operation of the self-mobile device.

[0072] Following the above, after the final three-dimensional map model is created, the three-dimensional map model can be filtered to eliminate discrete points therein so that the final three-dimensional map model has a relatively flat ground or wall. After the filtering process is completed, the three-dimensional map model can be converted into a positioning map model corresponding to the component information of the set image acquisition component on the second self-mobile device. Figure 3 The following is a schematic diagram of the model conversion process of the 3D map model. The specific steps are as follows:

[0073] Step 301: Determine a corresponding model conversion method according to a set image acquisition component on a second mobile device.

[0074] Step 302: Perform model conversion on the three-dimensional map model based on the model conversion method to obtain a positioning map model corresponding to the set image acquisition component on the second mobile device.

[0075] In practical applications, when the second self-mobile device needs to work in the target scene, it can communicate with the first self-mobile device through Bluetooth, WIFI, etc. The first self-mobile device can obtain the device information of the second self-mobile device (such as the model of the second self-mobile device, the component information and posture data of the image acquisition component on the second self-mobile device, etc.) based on the communication connection with the second self-mobile device.

[0076] In specific implementation, for example, assume that the set image acquisition element on the second self-mobile device is a laser radar, which is installed at a height of 10 cm on the second self-mobile device. Then, after obtaining the above information, the first self-mobile device can directly intercept the three-dimensional map with a height of 10 cm on the three-dimensional map model it created, convert it into a corresponding two-dimensional map (that is, the positioning map model corresponding to the set image acquisition element on the second self-mobile device), and send the two-dimensional map to the second self-mobile device. After receiving the two-dimensional map, the second self-mobile device can be positioned according to the two-dimensional map. For a specific positioning method, for example, the image data actually collected by the second self-mobile device can be matched with the two-dimensional map for feature points. When the number of feature point matches is greater than the set threshold, the current position of the second self-mobile device in the two-dimensional map can be determined.

[0077] For another example, suppose that the image acquisition component set on the second self-mobile device is a monocular camera, and it is difficult to directly reuse monocular cameras with different perspectives for maps. Therefore, after the first self-mobile device obtains the image acquisition component set as a monocular camera and the perspective of the monocular camera, it can simulate the camera in the three-dimensional map model to capture an image with the same perspective as the monocular camera, thereby completing the conversion of the perspective of the three-dimensional map model. It should be understood that since the three-dimensional map model has depth information, after the perspective is converted, the corresponding monocular perspective relationship transformation can be completed, thereby enabling the second self-mobile device to complete the repositioning.

[0078] It can be seen that by determining the corresponding model conversion method according to the set image acquisition element on the second self-mobile device, and converting the three-dimensional map model based on the model conversion method, a positioning map model corresponding to the set image acquisition element on the second self-mobile device can be obtained, thereby realizing the reuse of the three-dimensional map model created by the first self-mobile device, avoiding the second self-mobile device from repeatedly mapping the target scene, and improving work efficiency.

[0079] Further, Figure 4 Another flow chart of the model processing method provided in the embodiment of the present application is as follows: Figure 4 As shown, the method also includes:

[0080] Step 401: Acquire scanning data of a target scene from at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship.

[0081] Step 402: Determine the position and posture data of at least one image acquisition element.

[0082] Step 403: Create a three-dimensional map model corresponding to the target scene based on the scan data and the posture data.

[0083] Step 404: convert the three-dimensional map model into a positioning map model corresponding to the set image acquisition component on the second self-moving device, so that the second self-moving device moves according to the positioning map model.

[0084] Step 405: Create an obstacle avoidance map model based on the three-dimensional map model and the device information of the second self-moving device, where the obstacle avoidance map model is used to enable the second self-moving device to avoid obstacles in the target scene.

[0085] In practical applications, through steps 401 to 404, the repositioning of the second self-moving device can be completed, and according to the positioning map model corresponding to the set image acquisition element on the second self-moving device, the second self-moving device can be moved according to the positioning map model. However, the second self-moving device is likely to hit obstacles such as thresholds and steps during the movement. If these obstacles cannot be avoided in time, the second self-moving device is likely to be damaged.

[0086] In view of this, after creating a three-dimensional map model corresponding to the target scene, the present application can create an obstacle avoidance map model based on the three-dimensional map model and the device information of the second self-mobile device, so that the second self-mobile device can avoid obstacles in the target scene.

[0087] In specific implementation, since the device information of the second self-moving device (such as length, width, height and other size data) is known data, for example, assuming that the height of the second self-moving device is 30cm, then the first self-moving device will create a 0-30cm obstacle avoidance map model based on the three-dimensional map model and the device information of the second self-moving device (the height is 30cm), and mark the specific information of the obstacle in the obstacle avoidance map. When the second self-moving device moves to a set position according to the positioning map model, the obstacle avoidance map model can be used to determine whether there are obstacles near the set position (because the obstacle avoidance map model is actually also a two-dimensional map, but it marks on it which positions within the height range of 0-30cm have obstacles that the second self-moving device cannot pass through).

[0088] Based on the above, by creating an obstacle avoidance map model according to the three-dimensional map model and the device information of the second self-moving device, the second self-moving device can avoid obstacles in the target scene, especially avoid obstacles that cannot be perceived by some image acquisition elements (ie, sensors), thereby completing clear and safe trajectory planning for the second self-moving device in the target scene.

[0089] The following scenario embodiments are provided below to illustrate the present application.

[0090] Scenario Example 1:

[0091] Assume that the target scene is a classroom, the first self-moving device is a sweeping robot, the image acquisition components of which are two RGBD cameras, and the second self-moving device is a welcoming robot, the image acquisition component of which is a laser radar installed at a height of 10 cm on the welcoming robot. The specific implementation steps are as follows:

[0092] S1. Obtain scan data (including depth images and color images) in the classroom through two RGBD cameras.

[0093] S2. Determine the computing power requirements of the two RGBD cameras. If the sum of the computing power requirements of the two RGBD cameras is greater than the set threshold, it means that the processor computing power of the sweeping robot is weak and cannot simultaneously determine the pose data of the two RGBD cameras in the world coordinate system. At this time, directly obtain the pose data of one RGBD camera in the world coordinate system through the visual SLAM algorithm, and then determine the pose data of the other RGBD camera in the world coordinate system based on the pose relationship of the two RGBD cameras. Among them, the pose relationship of the two RGBD cameras can be determined by calibration using the camera calibration method after determining the position and posture of the two on the sweeping robot.

[0094] S3, creating a first three-dimensional map model based on the scanning data and pose data of the first RGBD camera;

[0095] Creating a second three-dimensional map model based on the scanning data and pose data of the second RGBD camera;

[0096] After obtaining the first three-dimensional map model and the second three-dimensional map model, the first three-dimensional map model and the second three-dimensional map model are spliced ​​by a PSDF algorithm based on the external parameter data of two RGBD cameras to obtain a final three-dimensional map model.

[0097] S4, the sweeping robot communicates with the welcoming robot via Bluetooth, WIFI, etc. The sweeping robot can obtain the device information of the welcoming robot (such as the model of the welcoming robot, the laser radar information and posture data of the welcoming robot, etc.) based on the communication connection with the welcoming robot.

[0098] The sweeping robot intercepts a 3D map with a height of 10 cm in the 3D map model it creates, converts it into a corresponding 2D map, and sends the 2D map to the welcoming robot. After receiving the 2D map, the welcoming robot can locate and move according to the 2D map.

[0099] S5. Create an obstacle avoidance map model based on the three-dimensional map model and the device information of the welcoming robot. When the welcoming robot moves to a certain set position according to the positioning map model, the obstacle avoidance map model can be used to determine whether there are obstacles near the set position, and then perform an obstacle avoidance operation. At the same time, the welcoming robot can also plan a clear and safe operation trajectory based on the obstacle avoidance map model.

[0100] Scenario Example 2:

[0101] Assume that the first self-moving device is a sweeping robot, the image acquisition components of which are two RGBD cameras, and the second self-moving device is a cleaning robot, the image acquisition component of which is a laser radar installed at a height of 10 cm on the cleaning robot. The specific implementation steps are as follows:

[0102] S1. Obtain scan data (including depth images and color images) in the classroom through two RGBD cameras.

[0103] S2. Determine the computing power requirements of the two RGBD cameras. If the sum of the computing power requirements of the two RGBD cameras is less than or equal to the set threshold, it means that the processor computing power of the sweeping robot is strong and the pose data of the two RGBD cameras in the world coordinate system can be determined simultaneously. At this time, the pose data of the two RGBD cameras in the world coordinate system is directly obtained through the visual SLAM algorithm. Among them, the pose relationship of the two RGBD cameras can be calibrated and determined using the camera calibration method after determining the position and posture of the two on the sweeping robot.

[0104] S3, creating a first three-dimensional map model based on the scanning data and pose data of the first RGBD camera;

[0105] Creating a second three-dimensional map model based on the scanning data and pose data of the second RGBD camera;

[0106] After obtaining the first three-dimensional map model and the second three-dimensional map model, the first three-dimensional map model and the second three-dimensional map model are spliced ​​by a PSDF algorithm based on the external parameter data of two RGBD cameras to obtain a final three-dimensional map model.

[0107] S4, the sweeping robot communicates with the cleaning robot via Bluetooth, WIFI, etc. The sweeping robot can obtain device information of the cleaning robot (such as the model of the cleaning robot, laser radar information and posture data on the cleaning robot, etc.) based on the communication connection with the cleaning robot.

[0108] The sweeping robot intercepts a 3D map with a height of 10 cm in the 3D map model it creates, converts it into a corresponding 2D map, and sends the 2D map to the cleaning robot. After receiving the 2D map, the cleaning robot can locate and move according to the 2D map.

[0109] S5. Create an obstacle avoidance map model based on the three-dimensional map model and the equipment information of the cleaning robot. When the cleaning robot moves to a certain set position according to the positioning map model, the obstacle avoidance map model can be used to determine whether there are obstacles near the set position, and then perform an obstacle avoidance operation. At the same time, the cleaning robot can also plan a clear and safe operation trajectory based on the obstacle avoidance map model.

[0110] Scenario Example 3:

[0111] Assume that the first self-moving device is a sweeping robot, the image acquisition components provided on the sweeping robot are two RGBD cameras, and the second self-moving device is a welcoming robot, the image acquisition component provided on the welcoming robot is a monocular camera. The specific implementation steps are as follows:

[0112] S1. Obtain scan data (including depth images and color images) in the classroom through two RGBD cameras.

[0113] S2. Determine the computing power requirements of the two RGBD cameras. If the sum of the computing power requirements of the two RGBD cameras is greater than the set threshold, it means that the processor computing power of the sweeping robot is weak and cannot simultaneously determine the pose data of the two RGBD cameras in the world coordinate system. At this time, directly obtain the pose data of one RGBD camera in the world coordinate system through the visual SLAM algorithm, and then determine the pose data of the other RGBD camera in the world coordinate system based on the pose relationship of the two RGBD cameras. Among them, the pose relationship of the two RGBD cameras can be determined by calibration using the camera calibration method after determining the position and posture of the two on the sweeping robot.

[0114] S3, creating a first three-dimensional map model based on the scanning data and pose data of the first RGBD camera;

[0115] Creating a second three-dimensional map model based on the scanning data and pose data of the second RGBD camera;

[0116] After obtaining the first three-dimensional map model and the second three-dimensional map model, the first three-dimensional map model and the second three-dimensional map model are spliced ​​by a PSDF algorithm based on the external parameter data of two RGBD cameras to obtain a final three-dimensional map model.

[0117] S4, the sweeping robot communicates with the welcoming robot via Bluetooth, WIFI, etc. The sweeping robot can obtain the device information of the welcoming robot (such as the model of the welcoming robot, the laser radar information and posture data of the welcoming robot, etc.) based on the communication connection with the welcoming robot.

[0118] The sweeping robot simulates a camera in the three-dimensional map model, captures images with the same perspective as the monocular camera, and completes the conversion of the perspective of the three-dimensional map model. It should be understood that since the three-dimensional map model has depth information, after the perspective is converted, the corresponding monocular perspective relationship can be converted, thereby enabling the second self-moving device to complete the repositioning.

[0119] S5. Create an obstacle avoidance map model based on the three-dimensional map model and the device information of the welcoming robot. When the welcoming robot moves to a certain set position according to the positioning map model, the obstacle avoidance map model can be used to determine whether there are obstacles near the set position, and then perform an obstacle avoidance operation. At the same time, the welcoming robot can also plan a clear and safe operation trajectory based on the obstacle avoidance map model.

[0120] Scenario Example 4:

[0121] Assume that the first self-moving device is a sweeping robot, the image acquisition components provided on the sweeping robot are two RGBD cameras, and the second self-moving device is a welcoming robot, the image acquisition component provided on the welcoming robot is a monocular camera. The specific implementation steps are as follows:

[0122] S1. Obtain scan data (including depth images and color images) in the classroom through two RGBD cameras.

[0123] S2. Determine the computing power requirements of the two RGBD cameras. If the sum of the computing power requirements of the two RGBD cameras is less than or equal to the set threshold, it means that the processor computing power of the sweeping robot is strong and the pose data of the two RGBD cameras in the world coordinate system can be determined simultaneously. At this time, the pose data of the two RGBD cameras in the world coordinate system is directly obtained through the visual SLAM algorithm. Among them, the pose relationship of the two RGBD cameras can be calibrated and determined using the camera calibration method after determining the position and posture of the two on the sweeping robot.

[0124] S3, creating a first three-dimensional map model based on the scanning data and pose data of the first RGBD camera;

[0125] Creating a second three-dimensional map model based on the scanning data and pose data of the second RGBD camera;

[0126] After obtaining the first three-dimensional map model and the second three-dimensional map model, the first three-dimensional map model and the second three-dimensional map model are spliced ​​by a PSDF algorithm based on the external parameter data of two RGBD cameras to obtain a final three-dimensional map model.

[0127] S4, the sweeping robot communicates with the welcoming robot via Bluetooth, WIFI, etc. The sweeping robot can obtain the device information of the welcoming robot (such as the model of the welcoming robot, the laser radar information and posture data of the welcoming robot, etc.) based on the communication connection with the welcoming robot.

[0128] The sweeping robot simulates a camera in the three-dimensional map model, captures images with the same perspective as the monocular camera, and completes the conversion of the perspective of the three-dimensional map model. It should be understood that since the three-dimensional map model has depth information, after the perspective is converted, the corresponding monocular perspective relationship can be converted, thereby enabling the second self-moving device to complete the repositioning.

[0129] S5. Create an obstacle avoidance map model based on the three-dimensional map model and the device information of the welcoming robot. When the welcoming robot moves to a certain set position according to the positioning map model, the obstacle avoidance map model can be used to determine whether there are obstacles near the set position, and then perform an obstacle avoidance operation. At the same time, the welcoming robot can also plan a clear and safe operation trajectory based on the obstacle avoidance map model.

[0130] In summary, the present application ensures the richness and integrity of the collected image data by acquiring the scanning data of the target scene by at least one image acquisition element. By determining the posture data of at least one image acquisition element, and based on the scanning data and posture data, a three-dimensional map model corresponding to the target scene can be created, thereby ensuring the accuracy of the three-dimensional map model. After the three-dimensional map model is created, by converting the three-dimensional map model into a positioning map model corresponding to the set image acquisition element on the second self-mobile device, the second self-mobile device can be moved according to the positioning map model, thereby realizing the reuse of the three-dimensional map model created by the first self-mobile device, avoiding the second self-mobile device from repeatedly mapping the target scene, and improving work efficiency.

[0131] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors are caused to perform at least the following actions:

[0132] Acquire scanning data of a target scene by at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship;

[0133] Determining position data of at least one image acquisition element;

[0134] Create a three-dimensional map model corresponding to the target scene based on the scanning data and posture data;

[0135] The three-dimensional map model is converted into a positioning map model corresponding to the set image acquisition component on the second mobile device.

[0136] In an optional embodiment, the processor is specifically used to: determine the computing power requirement of each image acquisition element; and determine the posture data of at least one image acquisition element based on the computing power requirement.

[0137] In an optional embodiment, the at least one image acquisition element includes a first image acquisition element and a second image acquisition element, and the processor is specifically used to: if the sum of the computing power requirements of the first image acquisition element and the second image acquisition element is greater than a set threshold, obtain the posture data of the first image acquisition element in the world coordinate system; based on the posture data of the first image acquisition element in the world coordinate system and the posture relationship, determine the posture data of the second image acquisition element in the world coordinate system.

[0138] In an optional embodiment, the processor is specifically used to: create a first three-dimensional map model based on the scanning data and posture data of the first image acquisition element; create a second three-dimensional map model based on the scanning data and posture data of the second image acquisition element; and determine the three-dimensional map model based on the first three-dimensional map model and the second three-dimensional map model.

[0139] In an optional embodiment, the processor is specifically used to: determine the corresponding model conversion method according to the set image acquisition element on the second self-mobile device; perform model conversion on the three-dimensional map model based on the model conversion method to obtain a positioning map model corresponding to the set image acquisition element on the second self-mobile device.

[0140] In an optional embodiment, the processor is also used to: create an obstacle avoidance map model based on the three-dimensional map model and device information of the second self-mobile device, and the obstacle avoidance map model is used to enable the second self-mobile device to avoid obstacles in the target scene.

[0141] Accordingly, an embodiment of the present application further provides a self-moving device, the self-moving device comprising: a device body, on which a memory, a processor and at least one image acquisition element are disposed;

[0142] Memory for storing computer programs;

[0143] The processor is coupled to the memory and is configured to execute a computer program for performing the following actions:

[0144] Acquire scanning data of a target scene by at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship;

[0145] Determining position data of at least one image acquisition element;

[0146] Create a three-dimensional map model corresponding to the target scene based on the scanning data and posture data;

[0147] The three-dimensional map model is converted into a positioning map model corresponding to the set image acquisition component on the second mobile device.

[0148] In an optional embodiment, the processor is specifically used to: determine the computing power requirement of each image acquisition element; and determine the posture data of at least one image acquisition element based on the computing power requirement.

[0149] In an optional embodiment, the at least one image acquisition element includes a first image acquisition element and a second image acquisition element, and the processor is specifically used to: if the sum of the computing power requirements of the first image acquisition element and the second image acquisition element is greater than a set threshold, then obtain the posture data of the first image acquisition element in the world coordinate system; based on the posture data of the first image acquisition element in the world coordinate system and the posture relationship, determine the posture data of the second image acquisition element in the world coordinate system. If the sum of the computing power requirements of the first image acquisition element and the second image acquisition element is less than or equal to the set threshold, then respectively obtain the posture data of the first image acquisition element and the second image acquisition element in the world coordinate system.

[0150] In an optional embodiment, the processor is specifically used to: create a first three-dimensional map model based on the scanning data and posture data of the first image acquisition element; create a second three-dimensional map model based on the scanning data and posture data of the second image acquisition element; and determine the three-dimensional map model based on the first three-dimensional map model and the second three-dimensional map model.

[0151] In an optional embodiment, the processor is specifically used to: determine the corresponding model conversion method according to the set image acquisition element on the second self-mobile device; perform model conversion on the three-dimensional map model based on the model conversion method to obtain a positioning map model corresponding to the set image acquisition element on the second self-mobile device.

[0152] In an optional embodiment, the processor is also used to: create an obstacle avoidance map model based on the three-dimensional map model and device information of the second self-mobile device, and the obstacle avoidance map model is used to enable the second self-mobile device to avoid obstacles in the target scene.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0154] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0159] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0161] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A model processing method, characterized in that: include: Acquire scanning data of a target scene by at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship; Determining position data of the at least one image acquisition element; Creating a three-dimensional map model corresponding to the target scene according to the scan data and the posture data; The three-dimensional map model is converted into a positioning map model corresponding to a set image acquisition component on the second mobile device.

2. The method according to claim 1, characterized in that The determining of the position and posture data of the at least one image acquisition element comprises: Determine the computing power requirements of each image acquisition component; Based on the computing power requirement, determine the posture data of the at least one image acquisition element.

3. The method according to claim 2, characterized in that The at least one image acquisition element includes a first image acquisition element and a second image acquisition element, and determining the position and posture data of the at least one image acquisition element based on the computing power requirement includes: If the sum of the computing power requirements of the first image acquisition component and the second image acquisition component is greater than a set threshold, obtaining the position and posture data of the first image acquisition component in the world coordinate system; Based on the position and posture data of the first image acquisition element in the world coordinate system and the position and posture relationship, the position and posture data of the second image acquisition element in the world coordinate system is determined.

4. The method according to claim 3, characterized in that The determining, based on the computing power requirement, the position and posture data of the at least one image acquisition element comprises: If the sum of the computing power requirements of the first image acquisition component and the second image acquisition component is less than or equal to a set threshold, the position data of the first image acquisition component and the second image acquisition component in the world coordinate system are respectively obtained.

5. The method according to claim 4, characterized in that The step of creating a three-dimensional map model corresponding to the target scene according to the scan data and the posture data comprises: Creating a first three-dimensional map model based on the scan data and the position and posture data of the first image acquisition element; Creating a second three-dimensional map model based on the scan data and the position and posture data of the second image acquisition element; The three-dimensional map model is determined according to the first three-dimensional map model and the second three-dimensional map model.

6. The method according to claim 1, characterized in that Converting the three-dimensional map model into a positioning map model corresponding to the component information of the set image acquisition component on the second mobile device includes: Determining a corresponding model conversion method according to a set image acquisition component on the second mobile device; The three-dimensional map model is model-converted based on the model conversion method to obtain a positioning map model corresponding to the set image acquisition component on the second self-mobile device.

7. The method according to claim 1, characterized in that Also includes: An obstacle avoidance map model is created according to the three-dimensional map model and the device information of the second self-moving device, and the obstacle avoidance map model is used to enable the second self-moving device to avoid obstacles in the target scene.

8. The method according to any one of claims 1 to 7, characterized in that: The scan data includes: depth data and color data of multiple frames corresponding to the target scene.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the one or more processors are caused to perform at least the following actions: Acquire scanning data of a target scene by at least one image acquisition element, wherein the at least one image acquisition element is disposed on a first self-moving device and has a set posture relationship; Determining position data of the at least one image acquisition element; Creating a three-dimensional map model corresponding to the target scene according to the scan data and the posture data; The three-dimensional map model is converted into a positioning map model corresponding to a set image acquisition component on the second mobile device.

10. A self-propelled device, characterized in that: include: A device body, wherein a memory, a processor and at least one image acquisition element are disposed on the device body; The memory is used to store computer programs; The processor is coupled to the memory and is configured to execute the computer program for performing the method of any one of claims 1 to 8.

Images