Positioning method and device for self-moving subject, self-moving subject and storage medium

Abstract

The embodiment of the present invention discloses a positioning method, device, self-moving subject and storage medium of a self-moving subject, the method comprising: determining a laser key frame according to laser data information, the laser data information comprising multiple frames of laser data; determining the first pose information of the self-moving subject at the current moment according to the laser key frame; determining a visual key frame according to image information, the image information comprising multiple frames of images; performing visual loopback matching according to the visual key frame to determine the second pose information of the self-moving subject at the current moment; and correcting the first pose information according to the error between the first pose information and the second pose information. The embodiment of the present invention realizes real-time positioning of the self-moving subject, and reduces the cumulative error of traditional laser positioning through visual loopback matching, thereby improving the accuracy of positioning of the self-moving subject.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of automatic positioning technology, and in particular to a positioning method and device for a self-moving subject, a self-moving subject and a storage medium. Background Art

[0002] An Automated Guided Vehicle (AGV) is a mobile robot that integrates perception, planning, decision-making and mobility functions. It can move along prescribed guided routes and is a key carrier device in the modern logistics system. It provides extremely important support for intelligent, integrated and efficient modern industrial production.

[0003] Common AGV equipment uses a laser sensor positioning solution, which uses a laser sensor to obtain distance information on the horizontal plane, and processes the environmental information scanned at different times, extracts similar features for matching to calculate the possible posture information of the AGV, and calculates the posture information with the highest probability as the real-time posture information of the AGV. When scanning environmental information, this positioning method can only obtain the distance information of obstacles on the same horizontal plane as the laser sensor. When the obstacle information on the horizontal plane is highly similar, the AGV will have a large error in constructing and positioning the map. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a positioning method and device for a self-moving subject, a self-moving subject, and a storage medium to improve the positioning accuracy of the self-moving subject when constructing a map and the closure of the map.

[0005] In a first aspect, an embodiment of the present invention provides a positioning method for a self-moving object, comprising:

[0006] Determine a laser key frame according to laser data information, wherein the laser data information includes multiple frames of laser data;

[0007] Determine the first position information of the self-moving subject at the current moment according to the laser key frame;

[0008] Determine a visual key frame according to image information, wherein the image information includes multiple frames of images;

[0009] Performing visual loop matching according to the visual key frame to determine the second position information of the self-moving subject at the current moment;

[0010] The first posture information is corrected according to an error between the first posture information and the second posture information.

[0011] Further, determining the laser key frame according to the laser data information includes:

[0012] Determine whether the laser data of the current frame of the laser data information meets the first preset condition of the laser key frame;

[0013] If the current frame laser data meets the first preset condition of the laser key frame, the current frame laser data is used as the laser key frame;

[0014] If the current frame laser data does not meet the first preset condition of the laser key frame, the next frame laser data is used as the current frame laser data, and the process returns to the step of determining whether the current frame laser data of the laser data information meets the first preset condition of the laser key frame.

[0015] Furthermore, determining the visual key frame according to the image information includes:

[0016] If the current frame image in the image information meets the second preset condition of the visual key frame, the current frame image is used as the visual key frame;

[0017] Correspondingly, performing visual loop matching according to the visual key frame to determine the second posture information of the self-moving subject at the current moment includes:

[0018] The current frame image is visually loop matched with the visual key frame to determine the second position information of the self-moving subject at the current moment.

[0019] Furthermore, after determining the first pose information of the self-moving subject at the current moment according to the laser key frame, the method also includes: if the visual key frame cannot be determined according to the image information, laser loopback matching is performed according to the laser key frame to determine the second pose information of the self-moving subject at the current moment.

[0020] Further, the determining the first position information of the self-moving subject at the current moment according to the laser key frame includes:

[0021] The current frame laser data is compared with the previous frame laser key frame by scanning matching to determine the first position information of the self-moving subject at the current moment.

[0022] Furthermore, after determining the laser key frame according to the laser data information, the method further includes:

[0023] A map is constructed based on the laser keyframes.

[0024] Further, after performing visual loop matching according to the visual key frame to determine the second posture information of the self-moving subject at the current moment, the method further includes:

[0025] The map is corrected according to an error between the first posture information and the second posture information.

[0026] In a second aspect, an embodiment of the present invention provides a positioning device for a self-moving body, comprising:

[0027] A laser key frame determination module, used to determine a laser key frame according to laser data information, wherein the laser data information includes multiple frames of laser data;

[0028] A posture information determination module, used to determine the first posture information of the self-moving subject at the current moment according to the laser key frame;

[0029] A visual key frame determination module, used to determine a visual key frame according to image information, wherein the image information includes multiple frames of images;

[0030] A visual loop closure matching module, used for performing visual loop closure matching according to the visual key frame to determine the second position information of the self-moving subject at the current moment;

[0031] A correction module is used to correct the first posture information according to the error between the first posture information and the second posture information.

[0032] In a third aspect, an embodiment of the present invention provides a self-moving body, including:

[0033] one or more processors;

[0034] A storage device for storing one or more programs;

[0035] Laser sensor, used to obtain laser data information;

[0036] An image sensor, used to obtain image information;

[0037] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for positioning a self-moving subject provided by any embodiment of the present invention.

[0038] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for positioning a self-moving subject provided by any embodiment of the present invention.

[0039] The method for positioning a self-moving subject provided by an embodiment of the present invention determines a laser key frame according to laser data information, the laser data information includes multiple frames of laser data; determines the first pose information of the self-moving subject at the current moment according to the laser key frame; determines a visual key frame according to image information, the image information includes multiple frames of images; performs visual loopback matching according to the visual key frame to determine the second pose information of the self-moving subject at the current moment; and corrects the first pose information according to the error between the first pose information and the second pose information. Real-time positioning of the self-moving subject is achieved, and the cumulative error of traditional laser positioning is reduced through visual loopback matching, thereby improving the accuracy of positioning of the self-moving subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of a method for positioning a self-moving subject provided in Embodiment 1 of the present invention;

[0041] Figure 2 A schematic diagram of a flow chart of a method for positioning a self-moving subject provided in Embodiment 2 of the present invention;

[0042] Figure 3 A schematic diagram of the structure of a positioning device for a self-moving body provided in Embodiment 3 of the present invention;

[0043] Figure 4 A structural schematic diagram of a self-moving body provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0045] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0046] In addition, the terms "first", "second", etc. may be used in this article to describe various directions, actions, steps or elements, but these directions, actions, steps or elements are not limited by these terms. These terms are only used to distinguish a first direction, action, step or element from another direction, action, step or element. The terms "first", "second", etc. should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" and "batch" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0047] Embodiment 1

[0048] Figure 1 This is a flow chart of a method for positioning a self-moving subject provided in Embodiment 1 of the present invention. This embodiment can be applied to real-time positioning and map construction of a self-moving subject. A self-moving subject refers to an object capable of autonomous movement, such as a robot, an autonomous driving vehicle, a sweeper, etc. Figure 1 As shown, the positioning method of a self-moving subject provided by the first embodiment of the present invention includes:

[0049] S110, determining a laser key frame according to laser data information, wherein the laser data information includes multiple frames of laser data.

[0050] Specifically, laser data information is acquired by a laser sensor or laser radar device installed on a self-moving subject. The self-moving subject can be any object that can move autonomously, such as an AGV (Automated Guided Vehicle). A frame of data is a unit of data transmission. Laser data information usually includes multiple frames of laser data. Laser key frames are usually key points where new things appear in laser data information. Adjacent laser key frames contain both common points and differences. Usually, the first frame of laser data obtained at the starting position of the self-moving subject is used as the initial laser key frame.

[0051] S120, determining the first position information of the self-moving subject at the current moment according to the laser key frame.

[0052] Specifically, after determining the laser key frame, the actual posture information of the self-moving subject at the current moment can be determined by comparing the laser data of the current frame with the laser key frame data of the previous frame, which is called the first posture information. The posture information usually includes displacement and rotation angle. The laser data of the current frame is compared with the laser key frame, that is, the relative displacement and relative rotation angle between the laser data of the current frame and the previous frame of laser key frame are determined. Then, based on the posture information of the previous frame of laser key frame where the self-moving subject is located, according to the relative displacement and relative rotation angle, the first posture information of the self-moving subject at the current moment can be determined.

[0053] S130: Determine a visual key frame according to image information, where the image information includes multiple frames of images.

[0054] Specifically, the image information is acquired by an image sensor disposed on the self-moving subject, and the image sensor may be a monocular camera, a binocular camera, or a wide-angle camera. The image information acquired by the image sensor is continuous, and a frame of image is a still image in the image information, which is the smallest unit in the image information, so the image information includes multiple frames of images. The visual key frame is usually the key point where new things appear in the image information. Adjacent visual key frames contain both commonalities and differences. Usually, the first frame of visual image obtained at the starting position of the self-moving subject is used as the initial visual key frame.

[0055] It should be noted that the acquisition of laser data information and the acquisition of image information are performed synchronously, and the processing of laser data information and the processing of image information are also performed synchronously. The steps in the embodiment of the present invention are not limited to the order of execution.

[0056] S140, performing visual loop matching according to the visual key frame to determine second posture information of the self-moving subject at a current moment.

[0057] Specifically, visual loop matching is performed by comparing the currently obtained visual keyframe with all previously stored visual keyframes to determine whether the currently obtained visual keyframe is one of all previously stored visual keyframes, that is, to determine whether the self-moving subject has reached the current posture before. Since the first posture information of the self-moving subject at the current moment is obtained based on the relative posture information between the current frame laser data and the previous frame laser keyframe, the longer the path of the self-moving subject, the greater the cumulative error. If it is determined through visual loop matching that the self-moving subject has reached the current posture, the historical keyframe obtained when the self-moving subject has reached the posture is used as the currently obtained visual keyframe for calculation, which can reduce this cumulative error. The second posture information of the self-moving subject at the current moment refers to the posture information of the self-moving subject at the current moment obtained by comparing the current frame image with the currently obtained visual keyframe.

[0058] S150. Correct the first posture information according to an error between the first posture information and the second posture information.

[0059] Specifically, the first pose information is obtained based on the previous laser key frame, and the second pose information is the pose information with reduced cumulative error obtained through visual loop matching. The error between the second pose information and the first pose information is the cumulative error. The first pose information is corrected according to the cumulative error, and the actual pose information of the self-moving subject at the current moment is also corrected, thereby making the positioning of the self-moving subject more accurate.

[0060] The method for positioning a self-moving subject provided in the first embodiment of the present invention determines a laser key frame according to laser data information, wherein the laser data information includes multiple frames of laser data; determines the first pose information of the self-moving subject at the current moment according to the laser key frame; determines a visual key frame according to image information, wherein the image information includes multiple frames of images; performs visual loopback matching according to the visual key frame to determine the second pose information of the self-moving subject at the current moment; and corrects the first pose information according to the error between the first pose information and the second pose information. The method realizes real-time positioning of the self-moving subject, reduces the cumulative error of traditional laser positioning through visual loopback matching, and improves the accuracy of positioning the self-moving subject.

[0061] Embodiment 2

[0062] Figure 2 This is a flow chart of a method for positioning a self-moving subject provided by Embodiment 2 of the present invention. This embodiment is a further refinement of the above embodiment. Figure 2 As shown, the positioning method of a self-moving subject provided by the second embodiment of the present invention includes:

[0063] S210: Determine whether the laser data of the current frame of the laser data information meets the first preset condition of the laser key frame.

[0064] Specifically, the laser data information is acquired by a laser sensor or a laser radar device installed on a self-moving subject. The self-moving subject can be any object that can move autonomously, for example, an AGV (Automated Guided Vehicle). A frame of data is a unit of data transmission. Laser data information usually includes multiple frames of laser data. Laser key frames are usually the key points where new things appear in laser data information. Adjacent laser key frames contain both commonalities and differences. The first frame of laser data obtained at the starting position of the self-moving subject is usually used as the initial laser key frame. Whether the current frame laser data meets the first preset condition of the laser key frame refers to whether the current frame laser data has a certain proportion of similarity with the previous frame laser key frame, such as a similarity of 60% to 70%.

[0065] S220: If the current frame laser data meets the first preset condition of the laser key frame, use the current frame laser data as the laser key frame.

[0066] Specifically, the first preset condition that the current frame laser data meets the laser key frame means that if the current frame laser data has a certain ratio of similarity with the previous frame laser key frame, the current frame laser data can be used as the laser key frame. The certain ratio mentioned here includes a ratio range. For example, if the current frame laser data has a 60% to 70% similarity with the previous frame laser key frame, the current frame laser data can be used as the laser key frame.

[0067] S230: If the current frame laser data does not meet the first preset condition of the laser key frame, the next frame laser data is used as the current frame laser data, and the process returns to step S210.

[0068] Specifically, there are two situations in which the current frame laser data does not meet the first preset condition of the laser key frame: the similarity between the current frame laser data and the previous frame laser key frame is too high, and the similarity between the current frame laser data and the previous frame laser key frame is too low. When the similarity between the current frame laser data and the previous frame laser key frame is too high, it means that the self-moving subject may be in a stationary state; when the similarity between the current frame laser data and the previous frame laser key frame is too low, it means that the laser data obtained from the self-moving subject may be wrong. At this time, the similarity between the next frame laser data and the previous frame laser key frame should continue to be compared, that is, the next frame laser data is used as the current frame laser data, and the step S210 is returned to extract the laser key frame from the subsequent multiple frames of laser data.

[0069] S240, comparing the laser data of the current frame with the laser key frame of the previous frame by means of scan matching to determine the first position information of the self-moving subject at the current moment.

[0070] Specifically, after determining the laser key frame, the actual position information of the self-moving subject at the current moment can be determined by comparing the current frame laser data with the previous frame laser key frame, which is called the first position information. The position information usually includes displacement and rotation angle. By scanning and matching, the relative displacement and relative rotation angle between the current frame laser data and the previous frame laser key frame are obtained. Then, based on the position information of the previous frame laser key frame where the self-moving subject is located, the first position information of the self-moving subject at the current moment can be determined according to the relative displacement and relative rotation angle.

[0071] S250: construct a map according to the laser key frames.

[0072] Specifically, the positioning of a self-moving subject usually involves determining the position of the self-moving subject in its environment, so a map of the environment in which the self-moving subject is located needs to be constructed during positioning.

[0073] S260: If the current frame image in the image information meets the second preset condition of the visual key frame, use the current frame image as the visual key frame.

[0074] Specifically, the image information is acquired by an image sensor disposed on the self-moving subject, and the image sensor may be a monocular camera, a binocular camera, or a wide-angle camera. The image information acquired by the image sensor is continuous, and a frame of image is a still image in the image information, which is the smallest unit in the image information, so the image information includes multiple frames of images. The visual key frame is usually the key point where new things appear in the image information. Adjacent visual key frames contain both commonalities and differences. Usually, the first frame of visual image obtained at the starting position of the self-moving subject is used as the initial visual key frame.

[0075] Furthermore, the second preset condition of the visual key frame may be that the current frame image has a certain ratio range of similarity with the previous frame visual key frame, or that the current frame image has a certain number of identical image features with the previous frame visual key frame. For example, if the current frame image has a similarity of 60% to 70% with the previous frame visual key frame, the current frame image is used as the visual key frame. If the similarity between the current image and the previous frame visual key frame is too high, it means that the self-moving subject may be in a stationary state; if the similarity between the current frame image and the previous frame visual key frame is too low, it means that the image information obtained from the self-moving subject may be wrong. At this time, the next frame image can be used as the current frame image and determine whether it meets the preset conditions of the visual key frame, so as to extract the visual key frame from the subsequent multiple frames.

[0076] It should be noted that the acquisition of laser data information and the acquisition of image information are performed synchronously, and the processing of laser data information and the processing of image information are also performed synchronously. The steps in the embodiment of the present invention are not limited to the order of execution.

[0077] S270, performing visual loop matching on the current frame image and the visual key frame to determine the second posture information of the self-moving subject at the current moment.

[0078] Specifically, visual loop matching is to compare the currently obtained visual keyframe with all the previously stored visual keyframes to determine whether there is a historical visual keyframe with a high similarity to the currently obtained visual keyframe among all the previously stored historical visual keyframes, that is, to determine whether the self-moving subject has reached the current posture before. If the currently obtained visual keyframe has a high similarity with one of the historical visual keyframes among all the previously stored visual keyframes, it means that a loop is detected. When performing positioning calculations, the historical visual keyframe and the currently obtained visual keyframe are calculated separately, thereby reducing the calculation error. If the currently obtained visual keyframe is different from any of the previously stored visual keyframes, it means that the self-moving subject may have moved to a new posture, and the currently obtained visual keyframe is stored.

[0079] Since the first pose information of the self-moving subject at the current moment is obtained based on the relative pose information between the current frame laser data and the previous frame laser key frame, the longer the path of the self-moving subject, the greater the cumulative error. If it is determined through visual loop matching that the self-moving subject has reached the current pose, the historical key frames obtained when the self-moving subject has reached the pose and the currently obtained visual key frames are calculated separately, and the calculated pose information is corrected according to the error between the two, which can reduce this cumulative error.

[0080] The second position and posture information of the self-moving subject at the current moment refers to the position and posture information of the self-moving subject at the current moment obtained by comparing the current frame image with the currently obtained visual key frame.

[0081] In an alternative embodiment, steps S260 to S270 may also be: if the visual key frame cannot be determined based on the image information, laser loopback matching is performed based on the laser key frame to determine the second posture information of the self-moving subject at the current moment.

[0082] Specifically, the inability to determine the visual keyframe based on the image information means that the image information acquisition fails, or the current frame image does not have a certain ratio of similarity with the previous frame visual keyframe. For example, if the image sensor on the self-moving subject is damaged, the visual keyframe cannot be determined, and visual loop matching cannot be performed. At this time, laser loop matching can be performed through laser keyframes to determine the second posture information of the self-moving subject at the current moment. The principle of laser loop matching is the same as that of visual loop matching, which will not be repeated here. The difference between the two lies in the different data processed. Laser loop matching uses laser keyframes for loop matching, and visual loop matching uses visual keyframes for loop matching.

[0083] S280. Correct the first posture information according to an error between the first posture information and the second posture information.

[0084] Specifically, the first pose information is obtained based on the previous laser key frame, and the second pose information is the pose information with reduced cumulative error obtained through visual loop matching. The error between the second pose information and the first pose information is the cumulative error. The first pose information is corrected according to the cumulative error, and the actual pose information of the self-moving subject at the current moment is also corrected, thereby making the positioning of the self-moving subject more accurate.

[0085] S290: Correct the map according to an error between the first posture information and the second posture information.

[0086] Specifically, the constructed map is corrected by the accumulated error obtained by loop closure matching, so that the map data is more accurate. For example, the constructed map is rotated 5° as a whole and translated 0.5 cm along the forward direction of the self-moving subject according to the accumulated error, so that the orientation of the map is more accurate, and the orientation of the self-moving subject is also more accurate. For another example, if the current position is detected by loop closure matching as a position that the self-moving subject has been to before, then the point is marked so that the constructed map has good closure.

[0087] The method for positioning a self-moving subject provided in the second embodiment of the present invention realizes real-time positioning of the self-moving subject by combining laser data information and image data information, and reduces the cumulative error of traditional laser positioning through visual loop matching, thereby improving the accuracy of self-moving subject positioning and the closure of the map.

[0088] Embodiment 3

[0089] Figure 3This is a schematic diagram of the structure of the positioning device of the self-moving subject provided in the third embodiment of the present invention. This embodiment can be applied to the real-time positioning and map construction of the self-moving subject. The self-moving subject refers to an object that can move autonomously, such as a robot, an automatic driving vehicle, a sweeper, etc. The positioning device of the self-moving subject provided in the third embodiment of the present invention can implement the positioning method of the self-moving subject provided in any embodiment of the present invention, and has the corresponding functional structure and beneficial effects of the implementation method. For the contents not described in detail in this embodiment, reference can be made to the description of any method embodiment of the present invention.

[0090] like Figure 3 As shown, the positioning device for a self-moving subject provided in the third embodiment of the present invention includes: a laser key frame determination module 310, a posture information determination module 320, a visual key frame determination module 330, a visual loop matching module 340 and a correction module 350.

[0091] The laser key frame determination module 310 is used to determine the laser key frame according to the laser data information, wherein the laser data information includes multiple frames of laser data;

[0092] The posture information determination module 320 is used to determine the first posture information of the self-moving subject at the current moment according to the laser key frame;

[0093] The visual key frame determination module 330 is used to determine the visual key frame according to the image information, wherein the image information includes multiple frames of images;

[0094] The visual loop closure matching module 340 is used to perform visual loop closure matching according to the visual key frame to determine the second position information of the self-moving subject at the current moment;

[0095] The correction module 350 is used to correct the first posture information according to the error between the first posture information and the second posture information.

[0096] Furthermore, the laser key frame determination module 310 is specifically used for:

[0097] Determine whether the laser data of the current frame of the laser data information meets the first preset condition of the laser key frame;

[0098] If the current frame laser data meets the first preset condition of the laser key frame, the current frame laser data is used as the laser key frame;

[0099] If the current frame laser data does not meet the first preset condition of the laser key frame, the next frame laser data is used as the current frame laser data, and the process returns to the step of determining whether the current frame laser data of the laser data information meets the first preset condition of the laser key frame.

[0100] Furthermore, the visual key frame determination module 330 is specifically used to: if the current frame image in the image information meets the preset conditions of the visual key frame, then the current frame image is used as the visual key frame; correspondingly, the visual loop matching module 340 is specifically used to: perform visual loop matching on the current frame image and the visual key frame to determine the second posture information of the self-moving subject at the current moment.

[0101] Furthermore, it also includes: a laser loop matching module, which is used to perform laser loop matching based on the laser key frame to determine the second posture information of the self-moving subject at the current moment if the visual key frame cannot be determined based on the image information.

[0102] Furthermore, the posture information determination module 320 is specifically used to compare the current frame laser data with the previous frame laser key frame by scanning matching to determine the first posture information of the self-moving subject at the current moment.

[0103] Furthermore, it also includes: a map construction module, which is used to construct a map according to the laser key frame.

[0104] Furthermore, the correction module 350 is further used to correct the map according to the error between the first posture information and the second posture information.

[0105] The positioning device for a self-moving subject provided in Example 3 of the present invention realizes real-time positioning of the self-moving subject through a laser key frame determination module, a posture information determination module, a visual key frame determination module, a visual loop matching module and a correction module, and reduces the cumulative error of traditional laser positioning through visual loop matching, thereby improving the accuracy of the self-moving subject positioning and the closure of the map.

[0106] Embodiment 4

[0107] Figure 4 This is a schematic diagram of a self-moving subject provided in Embodiment 4 of the present invention. In this embodiment, the self-moving subject refers to an object that can move autonomously, such as a robot, an automatic driving vehicle, a sweeper, etc. Figure 4 As shown, the self-moving body includes a processor 410, a memory 420, an input device 430, an output device 440, a laser sensor 450 and an image sensor 460; the number of processors 410 in the device / terminal / server can be one or more. Figure 4 A processor 410 is taken as an example; the processor 410, the memory 420, the input device 430 and the output device 440 in the device / terminal / server can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0108] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the positioning method of the self-moving subject in the embodiment of the present invention (for example, the laser key frame determination module 310, the posture information determination module 320, the visual key frame determination module 330, the visual loop matching module 340 and the correction module 350 in the positioning device of the self-moving subject). The processor 410 executes various functional applications and data processing of the device / terminal / server by running the software programs, instructions and modules stored in the memory 420, that is, to implement the above-mentioned positioning method of the self-moving subject, which may include: determining a laser key frame according to laser data information, the laser data information including multiple frames of laser data; determining the first pose information of the self-moving subject at the current moment according to the laser key frame; determining a visual key frame according to image information, the image information including multiple frames of images; performing visual loop matching according to the visual key frame to determine the second pose information of the self-moving subject at the current moment; and correcting the first pose information according to the error between the first pose information and the second pose information.

[0109] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the device / terminal / server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0110] The input device 430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the self-mobile body. The output device 440 may include a display device such as a display screen.

[0111] The laser sensor 450 is used to obtain laser data information, and may be a common laser sensor, a laser radar sensor, etc. The image sensor 460 is used to obtain image information, and may be a monocular camera, a binocular camera, a wide-angle camera, etc.

[0112] Embodiment 5

[0113] Embodiment 5 of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a method for positioning a self-moving subject as provided in any embodiment of the present invention is implemented. The method may include: determining a laser key frame based on laser data information, wherein the laser data information includes multiple frames of laser data; determining a first pose information of the self-moving subject at a current moment based on the laser key frame; determining a visual key frame based on image information, wherein the image information includes multiple frames of images; performing visual loop matching based on the visual key frame to determine a second pose information of the self-moving subject at a current moment; and correcting the first pose information based on an error between the first pose information and the second pose information.

[0114] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0115] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0116] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0117] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0118] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for positioning a self-moving object, characterized in that: include: Determine a laser key frame according to laser data information, wherein the laser data information includes multiple frames of laser data; Determine the first position information of the self-moving subject at the current moment according to the laser key frame; Determine a visual key frame according to image information, wherein the image information includes multiple frames of images; Performing visual loop matching according to the visual key frame to determine the second position information of the self-moving subject at the current moment; Correcting the first posture information according to an error between the first posture information and the second posture information; After determining the first position information of the self-moving subject at the current moment according to the laser key frame, the method further includes: If the visual keyframe cannot be determined according to the image information, laser loopback matching is performed according to the laser keyframe to determine the second posture information of the self-moving subject at the current moment; The visual loop matching is to perform similarity matching on all pre-stored historical visual key frames with the currently determined visual key frame. If a loop exists, the historical visual key frame and the currently obtained visual key frame are calculated separately during the positioning calculation. If no loop exists, the currently obtained visual key frame is stored.

2. The method according to claim 1, characterized in that Determining the laser key frame according to the laser data information includes: Determine whether the laser data of the current frame of the laser data information meets the first preset condition of the laser key frame; If the current frame laser data meets the first preset condition of the laser key frame, the current frame laser data is used as the laser key frame; If the current frame laser data does not meet the first preset condition of the laser key frame, the next frame laser data is used as the current frame laser data, and the process returns to the step of determining whether the current frame laser data of the laser data information meets the first preset condition of the laser key frame.

3. The method according to claim 1, characterized in that Determining the visual key frame according to the image information comprises: If the current frame image in the image information meets the second preset condition of the visual key frame, the current frame image is used as the visual key frame; Correspondingly, performing visual loop matching according to the visual key frame to determine the second posture information of the self-moving subject at the current moment includes: The current frame image is visually loop matched with the visual key frame to determine the second position information of the self-moving subject at the current moment.

4. The method according to claim 1, characterized in that Determining the first position information of the self-moving subject at the current moment according to the laser key frame includes: The current frame laser data is compared with the previous frame laser key frame by scanning matching to determine the first position information of the self-moving subject at the current moment.

5. The method according to claim 1, characterized in that After determining the laser key frame according to the laser data information, the method further includes: A map is constructed based on the laser keyframes.

6. The method according to claim 5, characterized in that After performing visual loop matching according to the visual key frame to determine the second position information of the self-moving subject at the current moment, the method further includes: The map is corrected according to an error between the first posture information and the second posture information.

7. A positioning device for a self-moving body, characterized in that: include: A laser key frame determination module, used to determine a laser key frame according to laser data information, wherein the laser data information includes multiple frames of laser data; A posture information determination module, used to determine the first posture information of the self-moving subject at the current moment according to the laser key frame; A visual key frame determination module, used to determine a visual key frame according to image information, wherein the image information includes multiple frames of images; A visual loop closure matching module, used for performing visual loop closure matching according to the visual key frame to determine the second position information of the self-moving subject at the current moment; A correction module, used for correcting the first posture information according to an error between the first posture information and the second posture information; Also includes: A laser loopback matching module, used for performing laser loopback matching according to the laser keyframe to determine the second position information of the self-moving subject at the current moment if the visual keyframe cannot be determined according to the image information; The visual loop matching is to perform similarity matching on all pre-stored historical visual key frames with the currently determined visual key frame. If a loop exists, the historical visual key frame and the currently obtained visual key frame are calculated separately during the positioning calculation. If no loop exists, the currently obtained visual key frame is stored.

8. A self-moving body, characterized in that: include: one or more processors; A storage device for storing one or more programs; Laser sensor, used to obtain laser data information; An image sensor, used to obtain image information; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for positioning a self-moving subject as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for positioning a self-moving subject as described in any one of claims 1 to 6 is implemented.

Images