Positioning method and device, mobile robot and storage medium

Abstract

The invention relates to a positioning method and device, a mobile robot and a storage medium. The method comprises the steps of collecting a target image; identifying a target object in the target image and first position information of the target object, wherein the target image is an image obtained by shooting the environment where the mobile robot is located; based on a target sensor, a target distance is determined, the target distance is the distance between the target sensor and the target object, and the target sensor is a sensor which is installed in the mobile robot and has a distance measuring function; and based on the target distance, correcting the first position information to obtain second position information of the target object. According to the method, on the basis of the target distance, the first position information is corrected, the more accurate second position information is obtained, the positioning accuracy of the target object is improved, and therefore under the condition that positioning is accurate, the mobile robot can be prevented from colliding with the target object when moving, and the coverage rate of the mobile robot can be increased.

Description

Technical Field

[0001] The present disclosure relates to the field of robot technology, and particularly to a positioning method, apparatus, mobile robot, and storage medium. Background Art

[0002] Robots are equipped with various sensors, such as lidar, infrared sensors, cameras and vision sensors, line structured light, etc. These sensors are used to sense the surrounding environment to prevent the robot from colliding with surrounding objects during movement. The object can be an obstacle, landmark, furniture, or other object.

[0003] In related technologies, a mobile robot captures images through a camera and uses image processing technology to identify the positions of objects in the environment. However, due to the influence of the external environment during shooting and the low accuracy of the camera itself, the positioning of objects in the environment is not accurate. If the positioned area is smaller than the actual area occupied by the object, the mobile robot may collide with the object during movement. If the positioned area is larger than the actual area occupied by the object, the movement path of the mobile robot cannot accurately cover the area not occupied by the object. Summary of the Invention

[0004] To overcome the problems in related technologies, the present disclosure provides a positioning method, apparatus, mobile robot, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a positioning method is provided.

[0006] Capture a target image, where the target image is an image obtained by a mobile robot capturing the environment it is in.

[0007] Identify a target object in the target image and first position information of the target object.

[0008] Based on a target sensor, determine a target distance, where the target distance is the distance between the target sensor and the target object, and the target sensor is a sensor installed in the mobile robot and having a ranging function.

[0009] Based on the target distance, correct the first position information to obtain second position information of the target object.

[0010] In some embodiments, before determining the target distance based on the target sensor, the method further includes:

[0011] Select the target sensor that matches the target object from multiple sensors, where the multiple sensors at least include a laser direct structuring (LDS) sensor, an obstacle avoidance sensor, and an edge following sensor.

[0012] In some embodiments, selecting the target sensor that matches the target object from the multiple sensors includes:

[0013] Determining the object type to which the target object belongs;

[0014] Selecting the target sensor that matches the object type from the multiple sensors.

[0015] In some embodiments, selecting the target sensor that matches the target object from the multiple sensors includes:

[0016] Determining the height of the target object;

[0017] Based on the installation position of each sensor, selecting the target sensor whose installation position matches the height from the multiple sensors.

[0018] In some embodiments, based on the installation position of each sensor, selecting the target sensor whose installation position matches the height from the multiple sensors includes:

[0019] For each sensor, when the height is higher than the installation position of the sensor, determining the sensor as the target sensor.

[0020] In some embodiments, the number of target sensors is multiple, and the number of target distances is multiple; based on the target distances, correcting the first position information to obtain the second position information of the target object includes:

[0021] Selecting any one of the multiple target distances, and based on the selected target distance, correcting the first position information to obtain the corrected position information;

[0022] Selecting any one of the unselected target distances, and based on the selected target distance, correcting the corrected position information until all the multiple target distances are selected, and determining the position information corrected based on the last selected target distance as the second position information.

[0023] In some embodiments, after correcting the first position information based on the target distances to obtain the second position information of the target object, the method further includes:

[0024] When the target object belongs to a preset object type, adjusting the second position information, wherein the area represented by the adjusted second position information is larger than the area represented by the second position information before adjustment.

[0025] In some embodiments, collecting the target image includes:

[0026] Based on a monocular camera, acquire the target image.

[0027] In some embodiments, the method further includes:

[0028] Mark the second position information in the grid map.

[0029] According to a second aspect of the embodiments of the present disclosure, a positioning device is provided.

[0030] An image acquisition module, configured to acquire a target image, where the target image is an image obtained by a mobile robot photographing the environment where it is located;

[0031] An identification module, configured to identify a target object in the target image and the first position information of the target object;

[0032] A distance determination module, configured to determine a target distance based on a target sensor, where the target distance is the distance between the target sensor and the target object, and the target sensor is a sensor installed in the mobile robot and having a ranging function;

[0033] A position correction module, configured to correct the first position information based on the target distance to obtain the second position information of the target object.

[0034] In some embodiments, the device further includes:

[0035] A sensor selection module, configured to select the target sensor that matches the target object from multiple sensors, and the multiple sensors at least include a Laser Direct Structuring (LDS) sensor, an obstacle avoidance sensor, and an edge following sensor.

[0036] In some embodiments, the sensor selection module is configured to:

[0037] Determine the object type to which the target object belongs;

[0038] Select the target sensor that matches the object type from the multiple sensors.

[0039] In some embodiments, the sensor selection module is configured to:

[0040] Determine the height of the target object;

[0041] Based on the installation position of each sensor, select the target sensor whose installation position matches the height from the multiple sensors.

[0042] In some embodiments, the sensor selection module is configured to, for each sensor, determine the sensor as the target sensor when the height is higher than the installation position of the sensor.

[0043] In some embodiments, the number of the target sensors is multiple, and the number of the target distances is multiple; the position correction module is configured to:

[0044] Select any one of the multiple target distances, and correct the first position information based on the selected target distance to obtain corrected position information;

[0045] Select any one of the unselected target distances, and correct the corrected position information based on the selected target distance until all the multiple target distances are selected, and determine the position information corrected based on the last selected target distance as the second position information.

[0046] In some embodiments, the apparatus further includes:

[0047] A position adjustment module, configured to adjust the second position information when the target object belongs to a preset object type, wherein the area represented by the adjusted second position information is larger than the area represented by the second position information before adjustment.

[0048] In some embodiments, the image acquisition module is configured to acquire the target image based on a monocular camera.

[0049] In some embodiments, the apparatus further includes:

[0050] A map construction module, configured to mark the second position information in a grid map.

[0051] According to a third aspect of the embodiments of the present disclosure, there is provided a mobile robot, including:

[0052] A processor;

[0053] A memory for storing instructions executable by the processor;

[0054] Wherein, the processor is configured to execute the method described in the first aspect of the embodiments of the present disclosure.

[0055] According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile robot, enabling the mobile robot to execute the method described in the first aspect of the embodiments of the present disclosure.

[0056] Adopting the above method of the present disclosure, the following beneficial effects are achieved:

[0057] After the method provided by the embodiments of the present disclosure acquires a target image and identifies the first position information of the target object in the target image, it will also determine the target distance between the target sensor and the target object, and then correct the first position information based on the target distance to obtain more accurate second position information, improving the accuracy of the positioning of the target object. Therefore, when the positioning is accurate, it is possible to avoid the mobile robot colliding with the target object when moving and improve the coverage rate of the mobile robot.

[0058] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0060] Figure 1 is a schematic diagram of a grid map shown according to an exemplary embodiment;

[0061] Figure 2 is a flowchart of a positioning method shown according to an exemplary embodiment;

[0062] Figure 3 is a flowchart of a positioning method shown according to an exemplary embodiment;

[0063] Figure 4 is a schematic diagram of a grid map in the related art shown according to an exemplary embodiment;

[0064] Figure 5 is a schematic diagram of a grid map shown according to an exemplary embodiment;

[0065] Figure 6 is a schematic diagram of a grid map in the related art shown according to an exemplary embodiment;

[0066] Figure 7 is a schematic diagram of a grid map shown according to an exemplary embodiment;

[0067] Figure 8 is a schematic diagram of the comparison of the minimum passing distance shown according to an exemplary embodiment;

[0068] Figure 9 is a flowchart of a positioning method shown according to an exemplary embodiment;

[0069] Figure 10It is a block diagram of a positioning device shown according to an exemplary embodiment;

[0070] Figure 11 It is a block diagram of a mobile robot shown according to an exemplary embodiment. Detailed implementation manners

[0071] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0072] The grid map includes a plurality of grids, and each grid represents an area. These grids can be square, rectangular or other shapes. Each grid can be marked. For example, different colors can be used for marking, and different colors indicate the state of the area represented by the grid. For example, referring to Figure 1 the schematic diagram of the grid map shown, Figure 1 in which the black grids indicate that there are obstacles in the area represented by the grid, and the white grids indicate that the area represented by the grid is idle, and the robot can move in the area represented by the white grids.

[0073] In the related art, a monocular camera is used to collect images, and the collected images are transmitted to a computing unit or a processor. Through the computing unit or the processor, the images are preprocessed, such as denoising, image enhancement, color correction, etc. Then, an AI (Artificial Intelligence) algorithm is used to identify the processed images to obtain the objects in the images. At the same time, the category to which the object belongs and the position of the object (i.e., the distance between the monocular camera and the object) can also be determined. Finally, a grid map can be constructed based on the identified information.

[0074] However, using a monocular camera for ranging has limitations. For example, when collecting images, the change of lighting conditions has a great impact on monocular camera ranging, and the accuracy will decrease when the lighting is low; monocular ranging usually has certain limitations on the measured depth range, especially for long-distance or short-distance measurements; monocular ranging usually also requires additional information. For example, camera parameters are needed to improve the accuracy of depth estimation; compared with other depth sensors (such as lidar or binocular cameras), the accuracy of depth estimation by monocular ranging is relatively low.

[0075] In the embodiments of the present disclosure, in view of the limitations of monocular ranging accuracy, a solution is proposed to obtain position information through monocular ranging and further correct the position information to obtain more accurate position information.

[0076] The embodiments of the present disclosure can be applied to various scenarios where a mobile robot needs to locate objects in the surrounding environment. For example, in a floor cleaning scenario, when the mobile robot is a floor cleaning robot, the floor cleaning robot can adopt the method provided by the embodiments of the present disclosure to determine the positions of various obstacles in the environment. Then, when the floor cleaning robot is working, it can accurately avoid each obstacle and cover as comprehensively as possible the area without obstacles, so as to better complete the work of cleaning.

[0077] The method provided by the embodiments of the present disclosure is executed by a mobile robot, which can be various types of robots such as a floor cleaning robot and a delivery robot.

[0078] Figure 2 is a flowchart of a positioning method shown according to an exemplary embodiment, which is executed by a mobile robot. Refer to Figure 2 The method includes the following steps:

[0079] Step S201, collect a target image, which is an image obtained by the mobile robot taking a picture of the surrounding environment.

[0080] Among them, the environment where the mobile robot is located can be indoor or outdoor. A camera can be installed in the mobile robot, and the target image is collected through the camera. The target image can include at least one object, and the object can be an obstacle, a landmark, furniture or other objects in the environment where the mobile robot is located.

[0081] Step S202, identify the target object in the target image and the first position information of the target object.

[0082] After obtaining the target image, the target image is identified to determine the target object included in the target image and the first position information of the target object. Among them, the first position information is used to represent the area where the target object is located, and the first position information can be the coordinates of the area where the target object is located, the distance between the target object and the mobile robot, or other information.

[0083] It should be noted that the target image can include multiple objects. In the embodiments of the present disclosure, only the target object is taken as an example to illustrate the positioning process of the target object, and other objects can also be positioned in the same way.

[0084] Step S203: Based on the target sensor, determine the target distance, which is the distance between the target sensor and the target object. The target sensor is a sensor with a ranging function installed in the mobile robot.

[0085] In the embodiments of the present disclosure, considering that the first position information may not be accurate enough, in order to obtain more accurate position information, the mobile robot determines the target distance based on the target sensor. The target sensor is a sensor with a ranging tool installed in the mobile robot. For example, the target sensor can be an LDS (Laser Direct Structuring) sensor, an obstacle avoidance sensor, an edge following sensor, etc. Among them, different target sensors determine the target distance in different ways, but the determined target distance is the distance between the target sensor and the target object.

[0086] Step S204: Based on the target distance, correct the first position information to obtain the second position information of the target object.

[0087] Since the target distance is used to represent the distance between the target sensor and the target object, and the target distance is relatively accurate, the first position information is corrected based on the target distance, that is, the first position information is adjusted to obtain more accurate second position information.

[0088] The method provided by the embodiments of the present disclosure, after collecting the target image and identifying the first position information of the target object in the target image, will also determine the target distance between the target sensor and the target object, and then correct the first position information based on the target distance to obtain more accurate second position information, improving the accuracy of the positioning of the target object. Thus, when the positioning is accurate, it can not only avoid the mobile robot colliding with the target object when moving, but also improve the coverage rate of the mobile robot.

[0089] Figure 3 is a flowchart of a positioning method shown according to an exemplary embodiment, executed by a mobile robot. Refer to Figure 3 and the method includes the following steps:

[0090] Step S301: Based on a monocular camera, collect a target image.

[0091] Among them, the monocular camera is installed in the mobile robot. For example, the monocular camera is installed at the front or top of the mobile robot. When the mobile robot is a mobile robot, it can scan the surrounding environment through the monocular camera, and the target image is an image captured based on the monocular camera.

[0092] In the embodiments of the present disclosure, the monocular camera has a lower cost compared to a binocular camera or other cameras.

[0093] Step S302: Identify the target object in the target image and the first position information of the target object.

[0094] In some embodiments, before identifying the target image, the target image is preprocessed to obtain a target image with better quality. For example, denoising processing, image enhancement processing, color correction processing, etc. are performed on the target image.

[0095] In some embodiments, a trained image recognition model is called to identify the target image to identify the target object in the target image. Then, after the target object is identified, the first position information of the target object is determined. Optionally, a preset ranging algorithm is used to determine the first position information, or other AI algorithms are used to determine the first position information. The embodiments of the present disclosure do not limit the determination method of the first position information.

[0096] In some embodiments, the height of the target object is identified.

[0097] Step S303: Select a target sensor that matches the target object from multiple sensors.

[0098] In the embodiments of the present disclosure, considering that the first position information may not be accurate enough, in order to obtain more accurate position information, a suitable target sensor is selected to adjust the first position information of the target object. Among them, the multiple sensors may be sensors such as LDS sensors, obstacle avoidance sensors, and edge sensors. The target sensor matching the target object means that the target sensor is more suitable for measuring the distance from the target object than other sensors.

[0099] In some embodiments, the object type to which the target object belongs is determined; a target sensor that matches the object type is selected from multiple sensors. Considering that the measurement methods of different sensors are different, for different object types, in order to make the measurement more accurate, a target sensor that matches the object type is selected. Optionally, a correspondence relationship between the object type and the sensor is set. After the object type to which the target object belongs is determined, based on this correspondence relationship, the target sensor corresponding to the object type is selected.

[0100] Optionally, the object type may refer to the object shape. For example, when the object type is conical (such as a bar chair), the LDS sensor and the edge sensor cannot be selected as the target sensor.

[0101] In some embodiments, the height of the target object is determined; based on the installation position of each sensor, a target sensor whose installation position matches the height is selected from multiple sensors. Since the installation position of the sensor on the mobile robot is fixed, the range that the sensor can measure during measurement is also fixed. For example, some sensors cannot measure objects whose height is lower than the installation position. Therefore, to avoid meaningless measurements by the sensor, a target sensor whose installation position matches the height is selected.

[0102] Optionally, for each sensor, when the height is higher than the installation position of the sensor, the sensor is determined as the target sensor.

[0103] In some embodiments, based on the first position information, a target sensor is selected from multiple sensors. Considering that some sensors are more accurate in measuring short distances, some sensors are more accurate in measuring long distances, and some sensors are more accurate in measuring both short and long distances, therefore, in order to obtain a more accurate measurement distance, based on the first position information, that is, based on the initially recognized distance between the mobile robot and the target object, according to the distance, a suitable target sensor is selected.

[0104] It should be noted that by adopting the above implementation manner, the selected target sensor can be one or more.

[0105] Step S304, based on the target sensor, determine the target distance.

[0106] In some embodiments, when the number of target sensors is multiple, the target distance corresponding to each target sensor can be determined respectively based on each target sensor.

[0107] In some embodiments, considering that some mobile robots may be equipped with target sensors and some mobile robots may not be equipped with target sensors, therefore, first determine whether the mobile robot is equipped with a target sensor. If it is equipped with a target sensor, then based on the target sensor, determine the target distance. If it is not equipped with a target sensor, then end the process.

[0108] It should be noted that for target sensors, different types of target sensors measure the target distance in different ways. For example, the target sensor can measure the distance by emitting point laser, emitting line laser or using other methods. Therefore, in the embodiments of the present disclosure, the method for the target sensor to measure the target distance is not limited.

[0109] Step S305, based on the target distance, correct the first position information to obtain the second position information of the target object.

[0110] Since the target distance is used to represent the distance between the target sensor and the target object, and the target distance is relatively accurate, the first position information is corrected based on the target distance to obtain more accurate second position information. Here, correcting the first position information means: adjusting the distance between the target object represented by the first position information and the mobile robot according to the measured target distance, so as to obtain the corrected second position information.

[0111] In some embodiments, when the number of target sensors is multiple and the number of target distances is multiple, any one of the multiple target distances is selected, and the first position information is corrected based on the selected target distance to obtain the corrected position information; any one of the unselected target distances is selected, and the corrected position information is corrected based on the selected target distance until all the multiple target distances are selected, and the position information corrected based on the last selected target distance is determined as the second position information. That is to say, first correct the first position information based on one target distance, and then correct the position information that has been corrected once based on the next target distance until the position information corrected last time is corrected based on the last target distance to obtain the second position information.

[0112] In some embodiments, when the target object belongs to a preset object type, the second position information is adjusted, where the area represented by the adjusted second position information is larger than the area represented by the second position information before adjustment. The preset object type is a pre-set object type, and the object belonging to the preset object type is an object that is easily dragged by the mobile robot during movement and affects the movement of the mobile robot. The area represented by the second position information is the area occupied by the target object. For example, the preset object type is wire type, sock type, etc.

[0113] Optionally, adjusting the second position information includes: reducing the distance between the target object represented by the second position information and the mobile robot, so as to increase the area occupied by the target object.

[0114] For example, when the mobile robot is a sweeping robot and the sweeping robot is equipped with side brushes, in order to prevent the side brushes from rolling up the target object belonging to the preset object type, each side of the area occupied by the target object is increased, and the increased distance is not less than the length of the side brush.

[0115] Step S306, mark the second position information in the grid map.

[0116] Mark the second position information in the grid map, so that the mobile robot can move according to the positions of the various objects marked in the grid map, and the user can also view the grid map to understand the distribution of the various objects.

[0117] It should be noted that for larger objects, the first position information of the target object may not be fully determined based on only one target image. In this case, the mobile robot can move around the target object, shoot from multiple angles, and obtain multiple target images, thereby obtaining the first position information of the target object by identifying multiple target images.

[0118] For example, see Figure 4 and Figure 5 A schematic diagram of a grid map is shown, where Figure 4 It is a grid map determined by the positioning solution in the relevant technology. Figure 5 is a grid map determined by the positioning solution provided by the embodiment of the present disclosure, from Figure 4 and Figure 5 It can be seen that the mobile robot takes target images at four locations around the obstacle to determine the location information of the obstacle. Figure 4 The black and gray grids are the grids where the obstacles are located. Figure 5 The black grid is where the obstacles are located. Figure 5 The grid determined in Figure 4 The grid determined in the algorithm is more accurate, which reduces the expansion of obstacles, thereby providing more areas for the mobile robot to move and improving the coverage of the mobile robot.

[0119] For example, see Figure 6 and Figure 7 A schematic diagram of a grid map is shown, where Figure 6 It is a grid map determined by the positioning solution in the relevant technology. Figure 7 is a grid map determined by using the positioning solution provided by the embodiment of the present disclosure, Figure 6 The black and gray grids are the grids where the obstacles are located. Figure 7 The black grid in the middle is the grid where the obstacle is located. When the mobile robot moves, it moves in the grid where there is no obstacle according to the grid where the obstacle is marked in the grid. Figure 6 and Figure 7 It can be seen that Figure 6 Due to the expansion of the obstacle, the mobile robot thinks that the obstacle occupies more grids, and there is no area between the two obstacles that the mobile robot can pass through. Therefore, the mobile robot cannot pass between the two obstacles. Figure 7 The grid occupied by the obstacles is more accurate, and there is an area between two obstacles that the mobile robot can pass through. Therefore, the mobile robot can pass between the two obstacles, which improves the coverage rate of the mobile robot.

[0120] In one example, the positioning method provided by the embodiments of the present disclosure is compared with other positioning methods in the related art. Refer to Figure 8 the schematic diagram of the minimum passing distance comparison shown in Figure 8 . For scenarios where the target objects are socks, wires, slippers, children's shoes, sports shoes, weighing scales, and bar stools, in each scenario, different positioning methods are used to determine the minimum passing distance of the mobile robot. Figure 8 The last data in each scenario in Figure 8 is the minimum passing distance determined by using the method provided by the embodiments of the present disclosure. In the scenarios of socks, wires, slippers, children's shoes, sports shoes, weighing scales, and bar stools, the determined minimum passing distances are 55, 55, 50, 50, 45, 50, and 45 respectively. From Figure 8 it can be seen that compared with the minimum passing distances determined in the related art, the minimum passing distances determined by the method provided by the embodiments of the present disclosure are the smallest in each scenario, and other related technologies cannot cover all scenarios. The minimum passing distances determined by the embodiments of the present disclosure are more accurate, and the mobile robot has a higher coverage rate.

[0121] For the method provided by the embodiments of the present disclosure, after collecting the target image and identifying the first position information of the target object in the target image, the target distance between the target sensor and the target object will also be determined, and then the first position information will be corrected based on the target distance to obtain more accurate second position information, improving the accuracy of positioning the target object. And, it reduces the expansion of the target object in the grid map, obtains a more accurate grid map, and when the grid map is displayed through the application program, enables the user to see a more accurate grid map. And, while improving the accuracy of positioning the target object, it also improves the coverage rate of the mobile robot. And, in order to prevent the target object of the preset object type from being rolled by the mobile robot during the movement of the mobile robot, affecting the movement of the mobile robot, the area representing the target object of the preset object type will be increased.

[0122] In one example, taking the target sensor as an LDS sensor, a line laser obstacle avoidance sensor, and an edge following sensor as examples, refer to Figure 9 the flowchart of the positioning method shown in Figure 9 , which is executed by the mobile robot and includes the following steps:

[0123] Step 1, construct a grid map.

[0124] Step 2, scan through the monocular camera to detect obstacles.

[0125] Step 3, move around the obstacle, use the monocular ranging algorithm to estimate the position information and height of the obstacle, and mark the position information of the obstacle in the grid map.

[0126] Step 4, determine whether there is an LDS sensor. If there is an LDS sensor, execute Step 5. If there is no LDS sensor, end the process.

[0127] Step 5, move around the obstacle, scan the obstacle based on the LDS sensor, and determine the first distance between the LDS sensor and the obstacle.

[0128] Step 6, if the height of the obstacle is higher than the installation height of the LDS sensor, correct the position information in the grid map according to the first distance.

[0129] Step 7, determine whether there is a line laser obstacle avoidance sensor. If there is a line laser obstacle avoidance sensor, execute Step 8. If there is no line laser obstacle avoidance sensor, end the process.

[0130] Step 8, move around the obstacle, scan the obstacle based on the line laser obstacle avoidance sensor, and determine the second distance between the line laser obstacle avoidance sensor and the obstacle.

[0131] Step 9, if the line laser can scan the obstacle, correct the position information in the grid map again according to the second distance.

[0132] Step 10, determine whether there is an edge following sensor. If there is an edge following sensor, execute Step 11. If there is no edge following sensor, end the process.

[0133] Step 11, move around the obstacle, scan the obstacle based on the edge following sensor, and determine the third distance between the edge following sensor and the obstacle.

[0134] Step 12, if the height of the obstacle is higher than the installation height of the edge following sensor, correct the position information in the grid map again according to the third distance.

[0135] Figure 10 is a block diagram of a positioning device shown according to an exemplary embodiment, configured in a mobile robot. Refer to Figure 10 , the device includes:

[0136] An image acquisition module 1001, configured to acquire a target image, where the target image is an image obtained by the mobile robot taking a picture of the environment it is in;

[0137] An identification module 1002, configured to identify a target object in the target image and the first position information of the target object;

[0138] A distance determination module 1003, configured to determine a target distance based on a target sensor, where the target distance is the distance between the target sensor and the target object, and the target sensor is a sensor installed in the mobile robot and having a ranging function;

[0139] A position correction module 1004, configured to correct the first position information based on a target distance to obtain second position information of a target object.

[0140] In some embodiments, the apparatus further includes:

[0141] A sensor selection module, configured to select a target sensor that matches the target object from a plurality of sensors, where the plurality of sensors at least includes a laser direct structuring (LDS) sensor, an obstacle avoidance sensor, and an edge following sensor.

[0142] In some embodiments, the sensor selection module is configured to:

[0143] Determine the object type to which the target object belongs;

[0144] Select a target sensor that matches the object type from the plurality of sensors.

[0145] In some embodiments, the sensor selection module is configured to:

[0146] Determine the height of the target object;

[0147] Based on the installation position of each sensor, select a target sensor whose installation position matches the height from the plurality of sensors.

[0148] In some embodiments, for each sensor, the sensor selection module is configured to determine the sensor as a target sensor when the height is higher than the installation position of the sensor.

[0149] In some embodiments, the number of target sensors is multiple, and the number of target distances is multiple; the position correction module 1004 is configured to:

[0150] Select any one of the multiple target distances, correct the first position information based on the selected target distance to obtain corrected position information;

[0151] Select any one of the unselected target distances, correct the corrected position information based on the selected target distance until all the multiple target distances are selected, and determine the position information corrected based on the last selected target distance as the second position information.

[0152] In some embodiments, the apparatus further includes:

[0153] A position adjustment module, configured to adjust the second position information when the target object belongs to a preset object type, where the area represented by the adjusted second position information is larger than the area represented by the second position information before adjustment.

[0154] In some embodiments, the image acquisition module 1001 is configured to acquire a target image based on a monocular camera.

[0155] In some embodiments, the apparatus further includes:

[0156] a map construction module configured to mark second position information in a grid map.

[0157] Regarding the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0158] An embodiment of the present disclosure further provides a mobile robot, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the positioning method in the above embodiments.

[0159] Figure 11 is a block diagram of a mobile robot 1100 shown according to an exemplary embodiment.

[0160] Referring to Figure 11 , the mobile robot 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power supply component 1106, a multimedia component 1108, an audio component 1110, an input / output (I / O) interface 1112, a sensor component 1114, and a communication component 1116.

[0161] The processing component 1102 generally controls the overall operation of the mobile robot 1100, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 1102 may include one or more modules to facilitate the interaction between the processing component 1102 and other components. For example, the processing component 1102 may include a multimedia module to facilitate the interaction between the multimedia component 1108 and the processing component 1102.

[0162] The memory 1104 is configured to store various types of data to support the operation of the mobile robot 1100. Examples of such data include instructions for any application or method operating on the mobile robot 1100, contact data, phone book data, messages, pictures, videos, and the like. The memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0163] The power supply component 1106 provides power to various components of the mobile robot 1100. The power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the mobile robot 1100.

[0164] The multimedia component 1108 includes a screen that provides an output interface between the mobile robot 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1108 includes a front camera and / or a rear camera. When the mobile robot 1100 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0165] The audio component 1110 is configured to output and / or input audio signals. For example, the audio component 1110 includes a microphone (MIC) that is configured to receive external audio signals when the mobile robot 1100 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 1104 or transmitted via the communication component 1116. In some embodiments, the audio component 1110 further includes a speaker for outputting audio signals.

[0166] The I / O interface 1112 provides an interface between the processing component 1102 and a peripheral interface module, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: home button, volume button, power button, and lock button.

[0167] The sensor component 1114 includes one or more sensors for providing status assessment of various aspects of the mobile robot 1100. For example, the sensor component 1114 can detect the on / off state of the mobile robot 1100, the relative positioning of components, such as the display and keypad of the mobile robot 1100. The sensor component 1114 can also detect a change in the position of the mobile robot 1100 or a component of the mobile robot 1100, the presence or absence of user contact with the mobile robot 1100, the orientation or acceleration / deceleration of the mobile robot 1100, and the temperature change of the mobile robot 1100. The sensor component 1114 can include proximity sensors configured to detect the presence of nearby objects without any physical contact. The sensor component 1114 can also include light sensors, such as CMOS or CCD image sensors, for use in imaging applications. In some embodiments, the sensor component 1114 can also include acceleration sensors, gyroscope sensors, magnetic sensors, pressure sensors, or temperature sensors.

[0168] The communication component 1116 is configured to facilitate communication between the mobile robot 1100 and other devices in a wired or wireless manner. The mobile robot 1100 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1116 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1116 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0169] In an exemplary embodiment, the mobile robot 1100 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0170] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1104 including instructions, and the above instructions can be executed by a processor 1120 of the mobile robot 1100 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0171] An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a mobile robot, the mobile robot can execute the positioning method in the above embodiment.

[0172] Those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0173] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A positioning method, characterized in that, The method includes: Collecting a target image, where the target image is an image obtained by a mobile robot capturing the environment it is in; Identifying a target object in the target image and first position information of the target object; Determining a target distance based on a target sensor, where the target distance is the distance between the target sensor and the target object, and the target sensor is a sensor installed in the mobile robot and having a ranging function; Based on the target distance, correcting the first position information to obtain second position information of the target object.

2. The method according to claim 1, wherein Before determining the target distance based on the target sensor, the method further includes: Selecting the target sensor that matches the target object from multiple sensors, where the multiple sensors at least include a Laser Direct Structuring (LDS) sensor, an obstacle avoidance sensor, and an edge following sensor.

3. The method according to claim 2, wherein The selecting the target sensor that matches the target object from multiple sensors includes: Determining the object type to which the target object belongs; Selecting the target sensor that matches the object type from the multiple sensors.

4. The method according to claim 2, wherein The selecting the target sensor that matches the target object from multiple sensors includes: Determining the height of the target object; Based on the installation position of each sensor, selecting the target sensor whose installation position matches the height from the multiple sensors.

5. The method according to claim 4, wherein The based on the installation position of each sensor, selecting the target sensor whose installation position matches the height from the multiple sensors includes: For each sensor, when the height is higher than the installation position of the sensor, determining the sensor as the target sensor.

6. The method according to claim 1, wherein The number of the target sensors is multiple, and the number of the target distances is multiple; the based on the target distance, correcting the first position information to obtain second position information of the target object includes: Selecting any one of the multiple target distances, and based on the selected target distance, correcting the first position information to obtain corrected position information; Selecting any one of the unselected target distances, and based on the selected target distance, correcting the corrected position information until all the multiple target distances are selected, and determining the position information corrected based on the last selected target distance as the second position information.

7. The method according to claim 1, wherein After the based on the target distance, correcting the first position information to obtain second position information of the target object, the method further includes: When the target object belongs to a preset object type, adjusting the second position information, where the area represented by the adjusted second position information is larger than the area represented by the second position information before adjustment.

8. The method according to claim 1, wherein The collecting the target image includes: Collecting the target image based on a monocular camera.

9. The method according to claim 1, characterized in that, The method further includes: Marking the second position information in a grid map.

10. A positioning device, characterized in that, The device includes: An image acquisition module configured to collect a target image, where the target image is an image obtained by a mobile robot capturing the environment it is in; An identification module, configured to identify a target object in the target image and first position information of the target object; A distance determination module, configured to determine a target distance based on a target sensor, where the target distance is the distance between the target sensor and the target object, and the target sensor is a sensor installed in the mobile robot and having a ranging function; A position correction module, configured to correct the first position information based on the target distance to obtain second position information of the target object.

11. A mobile robot, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the mobile robot, the mobile robot is enabled to execute the method according to any one of claims 1-9.

Images