SLAM method, apparatus, intelligent device, and computer-readable storage medium

Abstract

The application provides a SLAM method, device, intelligent equipment and computer readable storage medium. The method comprises: when a first brightness difference value of n preset regions of a current image collected is greater than a first preset threshold, performing m times of SLAM operations, n is a positive integer greater than or equal to 2, and m is a positive integer greater than or equal to 1; wherein, performing the jth SLAM operation comprises: determining an ith exposure parameter corresponding to an ith preset region, wherein j is 1, …, m in turn, and i is 1, …, n in turn; acquiring an ith frame of image by using the ith exposure parameter; extracting an ith group of feature points corresponding to the ith preset region from the ith frame of image; and performing SLAM operation based on the extracted ith group of feature points. In this way, the situation that the collected image is underexposed or overexposed at the place where light and shade are interchanged can be avoided, so that SLAM operation based on the underexposed or overexposed image can be avoided.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a SLAM method, apparatus, smart device, and computer-readable storage medium. Background Technology

[0002] SLAM (Simultaneous Localization and Mapping) is primarily used to solve the problem of robot localization and map building when moving in unknown environments. This problem can be described as follows: A robot starts moving from an unknown location in an unknown environment, performs self-localization based on its current location and a map during its movement, and simultaneously builds an incremental map based on its self-localization to achieve autonomous localization and navigation.

[0003] SLAM can operate in both indoor and outdoor environments, but it is highly dependent on light and cannot function in dark areas or areas without texture. In particular, when the device moves through areas of alternating light and dark, the images captured by the camera on the device are often underexposed or overexposed, making it difficult to perform subsequent SLAM operations based on the images. Summary of the Invention

[0004] The purpose of this application is to provide a SLAM method, apparatus, smart device, and computer-readable storage medium that can perform SLAM operations using feature points corresponding to appropriately exposed area images, thereby avoiding underexposure or overexposure of images acquired in areas of alternating light and dark, and thus avoiding SLAM operations based on underexposure or overexposure images.

[0005] This invention is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a SLAM method, comprising: when the first brightness difference value of n preset regions of the acquired current image is greater than a first preset threshold, performing m SLAM operations, where n is a positive integer greater than or equal to 2 and m is a positive integer greater than or equal to 1; wherein performing the j-th SLAM operation comprises: determining the i-th exposure parameter corresponding to the i-th preset region, where j takes values ​​of 1, ..., m in sequence; i takes values ​​of 1, ..., n in sequence; acquiring the i-th frame image using the i-th exposure parameter; extracting the i-th set of feature points corresponding to the i-th preset region from the i-th frame image; and performing the SLAM operation based on the extracted i-th set of feature points.

[0007] In the embodiments of the present application, when the first luminance difference values of the n preset regions of the current image collected are greater than the first preset threshold, it indicates that the intelligent device enters an environment with alternating light and dark at this time, that is, the current collected image is an image at the alternating light and dark position; at this time, by determining the appropriate exposure parameters of different preset regions, using the exposure parameters for image collection, and extracting the feature points of the corresponding regions from the collected image, the SLAM operation using the feature points corresponding to the region image with appropriate exposure can be realized, thereby avoiding the situation of insufficient or excessive exposure of the image collected at the alternating light and dark position, and further avoiding the SLAM operation according to the image with insufficient or excessive exposure.

[0008] In combination with the technical solutions provided by the above first aspect, in some possible implementation manners, the determining the ith exposure parameter corresponding to the ith preset region comprises: determining the ith exposure parameter corresponding to the ith preset region according to the current image, where j is equal to 1; and / or determining the ith exposure parameter corresponding to the ith preset region according to the (i-1)th image, where when j is equal to 1 and i is equal to 1, the (i-1)th image is the current image; and when j is greater than 1 and i is equal to 1, the (i-1)th image is the n-th image in the j-1th SLAM operation.

[0009] In the embodiments of the present application, in the first SLAM operation, because the current image is the first obtained image, the exposure parameters corresponding to each preset region are determined according to the current image, which can improve the rate of obtaining the exposure parameters. In the first SLAM operation, the ith image is obtained using the current image and the (i-1)th image, which can improve the accuracy of the ith exposure parameter obtained. In addition, because the (i-1)th image is the previous image of the ith image, the ith exposure parameter corresponding to the ith image can be more accurately obtained by using the (i-1)th image to obtain the ith image.

[0010] In combination with the technical solutions provided by the above first aspect, in some possible implementation manners, when j is equal to 1, before the determining the ith exposure parameter corresponding to the ith preset region, the method further comprises: setting the frame rate to n times of the initial frame rate.

[0011] In the embodiments of the present application, when j is equal to 1, before the determining the ith exposure parameter corresponding to the ith preset region, the frame rate of the intelligent device is set to n times of the initial frame rate, which can enable n images to be collected in the time length of collecting one image in the previous process when collecting each image, that is, the n images are respectively the images corresponding to each preset region, thereby facilitating the SLAM to use the images according to the previous frame rate.

[0012] With reference to the technical scheme provided in the first aspect above, in some possible implementation manners, when j is 2, 3, …, m in turn, after the i-th frame image is acquired by using the i-th exposure parameter, the method further includes: determining whether a second luminance difference value of n preset regions of the i-th frame image is less than a second preset threshold; and if the second luminance difference value is less than the second preset threshold, setting the frame rate to the initial frame rate and adopting another SLAM operation.

[0013] In the embodiments of the present application, when j is 2, 3, …, m in turn, it indicates that the SLAM operation after the first SLAM operation is performed. At this time, after the i-th frame image is acquired by using the i-th exposure parameter, whether the current is still in the environment with alternating light and dark can be determined according to the second luminance difference value of n preset regions of the i-th frame image. If the second luminance difference value is less than the second preset threshold, it indicates that the smart device is not currently in the environment with alternating light and dark, and another SLAM operation can be adopted. In this way, the amount of computation of the processor can be reduced, and the power consumption of the device can be reduced.

[0014] With reference to the technical scheme provided in the first aspect above, in some possible implementation manners, when j is 2, 3, …, m in turn, after the n-th frame image is acquired by using the n-th exposure parameter, the method further includes: determining whether a second luminance difference value of n preset regions of the n-th frame image is less than a second preset threshold; and if the second luminance difference value is less than the second preset threshold, setting the frame rate to the initial frame rate and adopting another SLAM operation.

[0015] In the embodiments of the present application, when j is 2, 3, …, m in turn, it indicates that the SLAM operation after the first SLAM operation is performed. At this time, after the n-th frame image is acquired by using the n-th exposure parameter, whether the current is still in the environment with alternating light and dark can be determined according to the second luminance difference value of n preset regions of the n-th frame image. If the second luminance difference value is less than the second preset threshold, it indicates that the smart device is not currently in the environment with alternating light and dark, and another SLAM operation can be adopted. In this way, the amount of computation of the processor can be reduced, and the power consumption of the device can be reduced. In addition, by determining the second luminance difference value of the n-th frame image in the SLAM operation after the first SLAM operation, the feature points corresponding to the n preset regions are collected in each SLAM operation, which is beneficial to the subsequent SLAM operation.

[0016] With reference to the technical scheme provided in the first aspect above, in some possible implementation manners, the total number of the preset regions is positively correlated with the maximum frame rate of the device.

[0017] With the technical solution provided by the first aspect, in some possible implementation manners, the first luminance difference value is a luminance difference value of any two adjacent preset regions.

[0018] In the embodiments of the present application, through the setting, the light-dark transition determined can be more accurate, and it can be avoided that an image is determined as an image with a large luminance difference according to two regions far away from each other, so as to enter the subsequent operation for light-dark transition, and thus the power consumption of the intelligent device can be reduced.

[0019] In the second aspect, the embodiments of the present application provide a SLAM device, comprising: a processing module, configured to perform m times of SLAM operations when a first luminance difference value of n preset regions of a current image collected is greater than a first preset threshold, n is a positive integer greater than or equal to 2, and m is a positive integer greater than or equal to 1; wherein performing the jth SLAM operation comprises: determining an ith exposure parameter corresponding to an ith preset region, wherein i is 1, …, n in turn, and j is 1, …, m in turn; acquiring an ith frame of image by using the ith exposure parameter; extracting an ith group of feature points corresponding to the ith preset region from the ith frame of image; and performing the SLAM operation based on the ith group of feature points extracted.

[0020] In the third aspect, the embodiments of the present application provide an intelligent device, comprising: an image and video collection device, a processor and a memory, the processor is connected with the image and video collection device and the memory respectively; the memory is configured to store a program; and the processor is configured to call the program stored in the memory to execute the method provided by the first aspect and / or some possible implementation manners of the first aspect.

[0021] In the fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the method provided by the first aspect and / or some possible implementation manners of the first aspect when being run by a processor. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Figure 1 A step flowchart of a SLAM method provided by the embodiments of the present application.

[0024] Figure 2Another step flow chart of a SLAM method provided by an embodiment of the present application.

[0025] Figure 3 Another step flow chart of a SLAM method provided by an embodiment of the present application.

[0026] Figure 4 A module block diagram of a SLAM device provided by an embodiment of the present application.

[0027] Figure 5 A module block diagram of a smart device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0029] In view of the fact that when the device moves at the place where light and dark are interchanged, the image captured by the camera on the device usually has the case of underexposure or overexposure, so that the SLAM cannot accurately perform subsequent operations according to the image. The present inventors have conducted research and exploration, and propose the following embodiments to solve the above problems.

[0030] The following will be described in combination with Figure 1 The specific process and steps of a SLAM method are described. The SLAM method provided by the embodiments of the present application can be applied to a smart device, such as a robot, for example, a cleaning robot, a sweeping robot, a service robot, a mobile robot, etc.

[0031] It should be noted that the SLAM method provided by the embodiments of the present application is not limited to Figure 1 and the order shown below.

[0032] At present, when the smart device moves and collects image videos, it usually adopts a normal working mode. In the normal working mode, the overall brightness of the collected current image is usually processed to obtain the corresponding exposure parameter, and then the exposure parameter is used to adjust the image video collection device to collect the next frame of image; or a preset exposure parameter is used to collect each frame of image.

[0033] When the first brightness difference value of the n preset regions of the collected current image is greater than the first preset threshold, m times of SLAM operation is performed, n is a positive integer greater than or equal to 2, and m is a positive integer greater than or equal to 1.

[0034] The preset area can be a region divided in advance, for example, two regions on the left and right of the image are pre-set, and the areas of the two regions are equal; or two regions on the top and bottom of the image are pre-set, and the area of the upper region accounts for 60% of the entire image, and the area of the lower region accounts for 40% of the entire image; or three regions on the left, middle and right of the image are pre-set, and the areas of the three regions are equal. It can be understood that when the region is divided in advance, the size, shape and position of the region can be divided according to the actual situation, which is not limited here.

[0035] The preset area can also be a region segmented by the intelligent device according to the brightness of the image after the image is collected, for example, after the image is collected, the brightness difference between the upper left region, the upper right region and the lower region is detected to be large, and then the image can be segmented into three regions, namely the upper left region, the upper right region and the lower region. In addition, the total number of preset regions and / or the minimum area of each preset region can be pre-set, so that the image region is segmented according to the total number and the brightness of the image after the image is collected. The total number can be 2, or 3, or 4, or 5, which is not limited here.

[0036] The first brightness difference value can be the brightness difference value of any two preset regions, and correspondingly, the first brightness difference value greater than the first preset threshold value indicates that the brightness difference value of any two preset regions is greater than the first preset threshold value. Through this setting, when there are two regions with large brightness difference in the image, it is determined that the environment is in the bright and dark interlaced state at this moment, so that the subsequent operation for the bright and dark interlaced state is entered. In this way, it can avoid missing images with large region brightness difference, and avoid the situation that the collected image is underexposed or overexposed.

[0037] In addition, the first brightness difference value can also be the brightness difference value of any two adjacent preset regions, and correspondingly, the first brightness difference value greater than the first preset threshold value indicates that the brightness difference value of any two adjacent preset regions is greater than the first preset threshold value. Through this setting, the bright and dark interlaced position determined can be more accurate, and it can avoid judging the image as an image with large brightness difference according to two regions far apart, so that the subsequent operation for the bright and dark interlaced state is entered, and thus the power consumption of the intelligent device can be reduced.

[0038] It should be noted that the brightness of the preset region can be the average brightness of the preset region. When the value range of the pixel value of the image is in [0, 1], the first preset threshold value can be between 0.05-0.15, for example, the first preset threshold value can be 0.07, or 0.1, or 0.13. When the value range of the pixel value is in [0, 255], the first preset threshold value can be between 25-45, for example, the first preset threshold value can be 30, or 35, or 45.

[0039] Further, the jth SLAM operation includes:

[0040] Step S101: determining an i-th exposure parameter corresponding to an i-th preset region, where j is sequentially taken as 1, …, m; and i is sequentially taken as 1, …, n.

[0041] Step S102: acquiring an i-th frame image by using the i-th exposure parameter.

[0042] Step S103: extracting an i-th group of feature points corresponding to the i-th preset region from the i-th frame image.

[0043] Step S104: performing a SLAM operation based on the extracted i-th group of feature points.

[0044] In the embodiment of the present application, when the first luminance difference value of the n preset regions of the current image collected is greater than the first preset threshold, it indicates that the intelligent device enters an environment with alternating light and dark at this time, that is, the current collected image is an image at an alternating light and dark place; at this time, by determining the appropriate exposure parameter of different preset regions, using the exposure parameter for image acquisition, and extracting the feature points of the corresponding region from the collected image, the SLAM operation using the feature points corresponding to the region with appropriate exposure can be realized, thereby avoiding the situation of insufficient or excessive exposure of the image collected at the alternating light and dark place, and further avoiding the SLAM operation according to the image with insufficient or excessive exposure.

[0045] In addition, it also needs to be explained that when the first luminance difference value of the n preset regions of the current image collected is less than or equal to the first preset threshold, the exposure parameter of the next frame image can be acquired from the current image, or the preset exposure parameter can be configured in the next frame image. And when the next frame image is collected, and the luminance difference thereof is obtained according to the next frame image, the next frame image can be understood as a new current image.

[0046] The way of performing the j-th SLAM operation will be described in detail below.

[0047] Step S101: determining an i-th exposure parameter corresponding to an i-th preset region, where j is sequentially taken as 1, …, m; and i is sequentially taken as 1, …, n.

[0048] The exposure parameter can be an exposure duration and / or a gain. The specific principle of the exposure duration and the gain can refer to the principle in the prior art, and the description is avoided here.

[0049] When j is equal to 1, before determining the i-th exposure parameter corresponding to the i-th preset region, the above SLAM method can further include: setting the frame rate to n times of the initial frame rate.

[0050] The initial frame rate is a frame rate preset by the smart device during operation, which is usually between 15-60 frames per second, such as 35 frames, 40 frames, or 50 frames. The n times is related to the total number of preset regions, for example, if the total number of preset regions is 3, the frame rate is set to 3 times the initial frame rate.

[0051] It can be understood that the total number of preset regions is positively correlated with the maximum frame rate of the device, that is, the larger the maximum frame rate, the more preset regions can be set, for example, if the maximum frame rate of the device is 120 and the initial frame rate is 40, a maximum of 3 preset regions can be set; if the maximum frame rate of the device is 160 and the initial frame rate is 40, a maximum of 4 preset regions can be set.

[0052] In the embodiments of the present application, before determining the i-th exposure parameter corresponding to the i-th preset region, the frame rate of the smart device is set to n times the initial frame rate, so that n frames of images can be collected in the time length of collecting one frame of image before, that is, the n frames of images are respectively the images corresponding to each preset region, thereby facilitating the SLAM to use the images according to the previous frame rate.

[0053] As another optional implementation, when j is equal to 1, before determining the i-th exposure parameter corresponding to the i-th preset region, the SLAM method can further include: setting the frame rate to a preset multiple of the initial frame rate. The preset multiple can be 2 times or 3 times.

[0054] In the embodiments of the present application, by setting a preset multiple in advance, the speed of setting and adjusting the frame rate can be improved.

[0055] Further, determining the i-th exposure parameter corresponding to the i-th preset region can specifically include: when j is equal to 1, the i-th exposure parameter corresponding to the i-th preset region can be determined according to the current image; and / or, the i-th exposure parameter corresponding to the i-th preset region is determined according to the i-1th frame of image. When i is equal to 1, the i-1th frame of image is the current image.

[0056] For example, when j is equal to 1 and i is equal to 1, the first exposure parameter is acquired to acquire the first frame of image according to the first exposure parameter; at this time, the first exposure parameter can be acquired from the current image. When j is equal to 1 and i is equal to 2, the second exposure parameter is acquired to acquire the second frame of image according to the second exposure parameter; at this time, the second exposure parameter can be acquired from the current image, or from the first frame of image, or from the combination of the current image and the first frame of image.

[0057] Further, when j is greater than 1, the i-th preset area corresponding i-th exposure parameter can be determined according to the i-1-th frame image. When i is equal to 1, the i-1-th frame image is the n-th frame image in the j-1-th SLAM operation.

[0058] For example, when j is equal to 3 and i is equal to 1, it means that the first preset area corresponding first exposure parameter is acquired for the third time. At this time, the first exposure parameter can be acquired according to the first preset area in the n-th frame image in the second SLAM operation, i.e., the first exposure parameter can be acquired according to the last image acquired in the previous SLAM operation of the current SLAM operation. When j is equal to 2 and i is equal to 2, it means that the second preset area corresponding second exposure parameter is acquired for the second time. At this time, the second exposure parameter can be acquired according to the first preset area in the first frame image in the second SLAM operation.

[0059] It should be noted that the i-th exposure parameter acquired from the current image and / or the i-1-th frame image is the i-th exposure parameter obtained by processing the i-th preset area in the current image and / or the i-1-th frame image. The specific principle of acquiring the exposure parameter of the next frame image according to the current frame image can refer to the principle in the prior art, and details are omitted here.

[0060] In the embodiment of the present application, in the first SLAM operation, because the current image is the first obtained image, the exposure parameters corresponding to each preset area are determined according to the current image, which can improve the rate of acquiring the exposure parameters. In the first SLAM operation, the i-th frame image is acquired by using the current image and the i-1-th frame image, which can improve the accuracy of the i-th exposure parameter acquired. In addition, because the i-1-th frame image is the previous frame image of the i-th frame image, the i-th exposure parameter corresponding to the i-th frame image can be more accurately acquired by using the i-1-th frame image.

[0061] As another optional implementation, determining the i-th preset area corresponding i-th exposure parameter can specifically include: when j is greater than 1, the i-th preset area corresponding i-th exposure parameter in the j-th SLAM operation can be determined according to the i-th preset area in the i-th frame image in the j-1-th SLAM operation.

[0062] For example, when j is equal to 3 and i is equal to 1, it means that the first preset area corresponding first exposure parameter is acquired for the third time. At this time, the first exposure parameter can be acquired according to the first preset area in the first frame image obtained in the last SLAM operation (i.e., the second SLAM operation).

[0063] In the embodiment of the present application, since the i-th preset region in the i-th frame image in the j-1-th SLAM operation is a region image without underexposure or overexposure, the i-th exposure parameter corresponding to the i-th preset region is determined by using the region image, so that the accuracy of the determined i-th exposure parameter can be improved.

[0064] After the i-th exposure parameter is determined, the method can continue to step S102.

[0065] Step S102: acquiring the i-th frame image by using the i-th exposure parameter.

[0066] The processor in the intelligent device can send an instruction to the image video acquisition device to acquire the i-th frame image according to the i-th exposure parameter, so that the image video acquisition device sets its exposure parameter to the i-th exposure parameter and acquires the i-th frame image.

[0067] As a first optional implementation, please refer to Figure 2 When j is 2, …, m in turn, after the i-th frame image is acquired by using the i-th exposure parameter, the SLAM method can further include: judging whether a second luminance difference value of the n preset regions of the i-th frame image is less than a second preset threshold; if the second luminance difference value is less than the second preset threshold, setting the frame rate to the initial frame rate, and using another SLAM operation.

[0068] The second luminance difference value can be a luminance difference value of any two preset regions, or a luminance difference value of any two adjacent preset regions, which is not limited here.

[0069] The second preset threshold can be equal to the first preset threshold, or slightly less than the first preset threshold, for example: when the first preset threshold is 0.1, the second preset threshold can be 0.1 or 0.08. By setting the second preset threshold to a value slightly less than the first preset threshold, the intelligent device can only change to another SLAM operation when it is determined that the second luminance difference value is less than the second preset threshold, i.e., it is determined that it is not in an environment with alternating light and dark, so that the images collected by the intelligent device can be used by SLAM.

[0070] The other SLAM operation can be an operation that the intelligent device usually uses to collect images and use the collected images to perform SLAM operation, for example: collecting a current image, determining an exposure parameter of a next frame image to be collected according to the current image, or using a preset exposure parameter to acquire the next frame image by using the exposure parameter, i.e., this method does not divide the image into regions, but acquires the whole image and processes the whole image.

[0071] For example, there are three preset regions, and when j is equal to 2, after the second frame image is acquired by using the second exposure parameter, it is determined whether the second luminance difference value of the three preset regions of the second frame image is less than a second preset threshold value; if the second luminance difference value is less than the second preset threshold value, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0072] It can be understood that when it is determined that the second luminance difference value is less than the second preset threshold value, the frame rate can be immediately set to the initial frame rate, and another SLAM operation is adopted, that is, subsequent extraction of the second group of feature points and SLAM operation based on the second group of feature points (that is, steps S103 and S104) are not performed; the frame rate can also be set to the initial frame rate and another SLAM operation is adopted after subsequent extraction of the second group of feature points and SLAM operation based on the second group of feature points (that is, steps S103 and S104) are performed, which is not limited here.

[0073] In the embodiments of the present application, when j is 2, …, m in turn, it indicates that the SLAM operation after the first SLAM operation is performed, and after the i-th frame image is acquired by using the i-th exposure parameter, whether the current is still in the environment with alternating light and dark can be determined according to the second luminance difference value of the n preset regions of the i-th frame image; if the second luminance difference value is less than the second preset threshold value, it indicates that the intelligent device is not currently in the environment with alternating light and dark, and another SLAM operation can be adopted. In this way, the amount of calculation of the processor can be reduced, and the power consumption of the device can be reduced.

[0074] In addition, if the second luminance difference value is greater than or equal to the second preset threshold value, subsequent operations are continued, that is, the second group of feature points is extracted, and SLAM operation based on the second group of feature points (that is, steps S103 and S104) is performed. After completing the step S104 this time, the steps S101-S104 are continued for the next region.

[0075] As a second optional implementation, please refer to Figure 3 When j is 2, …, m in turn, after the n-th frame image is acquired by using the n-th exposure parameter, the method further comprises: determining whether the second luminance difference value of the n preset regions of the n-th frame image is less than a second preset threshold value; if the second luminance difference value is less than the second preset threshold value, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0076] Among them, the second luminance difference value, the second preset threshold value and the other SLAM operation in the embodiments of the present application can refer to the description of the second luminance difference value, the second preset threshold value and the other SLAM operation in the foregoing embodiments for description, and will not be repeated here.

[0077] It can be understood that in the embodiments of the present application, after the subsequent feature points are extracted and the SLAM operation is performed based on the feature points (i.e., steps S103 and S104), the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0078] In the embodiments of the present application, when j takes 2, …, m in turn, it indicates that the SLAM operation after the first SLAM operation is performed. At this time, after the nth frame of image is acquired by using the nth exposure parameter, whether the current is still in the environment with alternating light and dark can be determined according to the second brightness difference value of the n preset regions of the nth frame of image. If the second brightness difference value is less than the second preset threshold, it indicates that the intelligent device is not currently in the environment with alternating light and dark, and another SLAM operation can be adopted. In this way, the amount of calculation of the processor can be reduced, and the power consumption of the device can be reduced. In addition, by judging the second brightness difference value of the nth frame of image in the SLAM operation after the first SLAM operation, the feature points corresponding to the n preset regions are collected in each SLAM operation, which is beneficial to the subsequent SLAM operation.

[0079] As a third optional implementation, when j takes 2, …, m in turn, after the ith (i<=n) frame of image is acquired by using the ith exposure parameter, the above SLAM method can further include: judging whether the second brightness difference between the preset regions corresponding to each frame of image in the jth SLAM operation is less than the second preset threshold; if the second brightness difference value is less than the second preset threshold, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0080] Among them, the description of the second brightness difference value, the second preset threshold and the other SLAM operation in the embodiments of the present application can refer to the description of the second brightness difference value, the second preset threshold and the other SLAM operation in the foregoing embodiments, and will not be repeated here. The preset region corresponding to each frame of image refers to the ith preset region corresponding to the ith frame of image.

[0081] For example, there are three preset regions, when j is equal to 3, after the second frame of image is acquired, the first preset region can be extracted from the first frame of image, and the second preset region can be extracted from the second frame of image, and whether the second brightness difference of the extracted first preset region and the second preset region is less than the second preset threshold is judged. If the second brightness difference value is less than the second preset threshold, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0082] After the third frame of image is acquired, a first preset region can be extracted from the first frame of image, a second preset region can be extracted from the second frame of image, and a third preset region can be extracted from the third frame of image, and it is judged whether the second luminance difference of the extracted first preset region, second preset region and third preset region is less than a second preset threshold value; if the second luminance difference value is less than the second preset threshold value, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0083] In the embodiments of the present application, when j takes 2, …, m in turn, it indicates that the SLAM operation after the first time is performed, and at this time, by judging whether the second luminance difference between the preset regions corresponding to each frame of image in the jth time is less than the second preset threshold value, it can be determined whether the current is still in the environment with alternating light and dark, and if the second luminance difference value is less than the second preset threshold value, it indicates that the intelligent device is not currently in the environment with alternating light and dark, and another SLAM operation can be adopted. In this way, the amount of calculation of the processor can be reduced, and the power consumption of the device can be reduced.

[0084] As a fourth optional implementation, when j takes 1, …, m in turn, after the nth frame of image is acquired by using the nth exposure parameter, the above SLAM method can further include: judging whether the second luminance difference between the preset regions corresponding to each frame of image in the jth SLAM is less than the second preset threshold value; if the second luminance difference value is less than the second preset threshold value, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0085] Wherein, the description of the second luminance difference value, the second preset threshold value and the other SLAM operation in the embodiments of the present application can refer to the description of the second luminance difference value, the second preset threshold value and the other SLAM operation in the foregoing embodiments, and will not be repeated here. And the preset region corresponding to each frame of image refers to the n frames of image in the jth SLAM operation.

[0086] For example, when j is equal to 1 and there are four preset regions, after the fourth frame of image is acquired, the first preset region can be extracted from the first frame of image, the second preset region can be extracted from the second frame of image, the third preset region can be extracted from the third frame of image, and the fourth preset region can be extracted from the fourth frame of image, and it is judged whether the second luminance difference between the first preset region, the second preset region, the third preset region and the fourth preset region is less than the second preset threshold value; if the second luminance difference value is less than the second preset threshold value, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0087] In the embodiments of the present application, after the nth frame of image is acquired by using the nth exposure parameter, whether the current is still in the environment with alternating light and dark can be determined according to the second luminance difference between the preset regions corresponding to each frame of image in the jth SLAM. If the second luminance difference value is less than the second preset threshold, it indicates that the intelligent device is not currently in the environment with alternating light and dark, and another SLAM operation can be adopted. In this way, the amount of calculation of the processor can be reduced, and the power consumption of the device can be reduced. In addition, by performing the second luminance difference value judgment only after the nth frame of image is acquired, the feature points corresponding to the n preset regions are collected in each SLAM operation, which is beneficial to the subsequent SLAM operation.

[0088] It can be understood that in the foregoing embodiments, if the second luminance difference value is greater than or equal to the second preset threshold, subsequent operations can be continued, such as: after it is judged that the second luminance difference between the preset regions corresponding to the n frames of image in the jth SLAM is greater than or equal to the second preset threshold, the jth SLAM operation can be performed based on the n groups of feature points first; and after the jth SLAM operation is completed, the first exposure parameter for the j+1th SLAM operation is acquired, that is, the subsequent operation is continued.

[0089] In addition, in the foregoing embodiments, if the second luminance difference value is less than the second preset threshold, the SLAM operation of this time can be performed based on the feature points corresponding to the frames of image that have been obtained first, and after the SLAM operation is completed, the frame rate is set to the initial frame rate, and another SLAM operation is adopted, such as: after it is judged that the second luminance difference between the preset regions corresponding to the n frames of image in the jth SLAM is less than the second preset threshold, the jth SLAM operation can be performed based on the n groups of feature points first; and after the jth SLAM operation is completed, the frame rate is set to the initial frame rate, and another SLAM operation is adopted.

[0090] After the ith frame of image is acquired, the method can continue to step S103.

[0091] Step S103: Extracting the ith group of feature points corresponding to the ith preset region from the ith frame of image.

[0092] The above feature point extraction can be performed by using a SIFT (Scale-invariant feature transform) algorithm, an ORB (Oriented FAST and Rotated BRIEF) algorithm or a FAST (Features from accelerated segment test) algorithm. In addition, other algorithms capable of extracting feature points from images can also be used to extract the above feature points.

[0093] It is understandable that the specific principles of the SIFT, ORB, and FAST algorithms can be found in existing technologies, and will not be elaborated upon here to avoid further explanation.

[0094] After extracting the i-th set of feature points, this method can continue to step S104.

[0095] Step S104: Perform SLAM operation based on the extracted i-th group of feature points.

[0096] The SLAM operation includes map building and operations such as localization, navigation, and obstacle avoidance for smart devices. The specific principles of SLAM operations based on the extracted i-th set of feature points can be found in existing technologies and will not be elaborated upon here.

[0097] It is understandable that for each image to be acquired, steps S101-S104 can be followed. For example, if the second frame image is about to be acquired, the second exposure parameters corresponding to the second frame image can be acquired first; then the second frame image can be acquired based on the second exposure parameters; next, the second set of feature points can be extracted from the second frame image; finally, SLAM operation can be performed based on the second set of feature points.

[0098] Furthermore, in the first SLAM operation (i.e., the SLAM operation when j equals 1), the exposure parameters corresponding to all regions can be calculated based on the current image; then, each frame image can be acquired based on these exposure parameters; after acquiring each frame image, the feature points corresponding to each frame image can be extracted; finally, the SLAM operation can be performed based on the feature points of each frame image. Similarly, in SLAM operations other than the first SLAM operation (i.e., the SLAM operation when j equals 2, ..., m), after acquiring each frame image, the feature points corresponding to each frame image can be extracted; finally, the SLAM operation can be performed based on the feature points of each frame image.

[0099] Furthermore, in the j-th SLAM operation, for each image to be acquired, steps S101-S103 can be followed. Before entering the (j+1)-th SLAM operation, a SLAM operation can be performed based on the n sets of feature points from the j-th SLAM operation. For example, if there are three preset regions, when j equals 1, the first set of feature points, the second set of feature points, and the third set of feature points can be acquired sequentially according to steps S101-S103. Then, based on the acquired first set of feature points, the second set of feature points, and the third set of feature points, the first SLAM operation can be performed. After completing the first SLAM operation, the above operation is repeated.

[0100] Or, before the jth SLAM operation stops, the SLAM operation is performed based on the plurality of groups of feature points obtained in the jth SLAM operation, for example, there are four preset regions, in the third SLAM operation, after the third frame of image is collected, it is judged that the second luminance difference value of the four preset regions of the third frame of image is less than the second preset threshold value, that is, at this time, another SLAM operation can be used, and then, before the another SLAM operation is used, the first group of feature points, the second group of feature points and the third group of feature points obtained in the third SLAM operation can be used to perform the SLAM operation.

[0101] In the embodiment of the present application, by the above-mentioned manner, the SLAM operation can be performed using the feature points corresponding to the region image with appropriate exposure, so that the situation of insufficient or excessive exposure of the image collected at the place where light and dark are interchanged can be avoided, and then the SLAM operation based on the image with insufficient or excessive exposure can be avoided.

[0102] Please refer to Figure 4 Based on the same inventive concept, the embodiment of the present application also provides a SLAM device 100, which comprises a processing module 101.

[0103] The processing module 101 is configured to perform m times of SLAM operation when the first luminance difference value of the n preset regions of the collected current image is greater than the first preset threshold value, n is a positive integer greater than or equal to 2, and m is a positive integer greater than or equal to 1; wherein, the jth SLAM operation comprises: determining the ith exposure parameter corresponding to the ith preset region, wherein j is 1, …, m in turn; i is 1, …, n in turn; obtaining the ith frame of image by using the ith exposure parameter; extracting the ith group of feature points corresponding to the ith preset region from the ith frame of image; and performing the SLAM operation based on the extracted ith group of feature points.

[0104] Optionally, the processing module 101 is specifically configured to determine the ith exposure parameter corresponding to the ith preset region according to the current image, wherein j is equal to 1, and when i is equal to 1, the (i-1)th frame of image is the current image; and / or, determine the ith exposure parameter corresponding to the ith preset region according to the (i-1)th frame of image, wherein when j is equal to 1 and i is equal to 1, the (i-1)th frame of image is the current image; and when j is greater than 1 and i is equal to 1, the (i-1)th frame of image is the nth frame of image in the (j-1)th SLAM operation.

[0105] Optionally, before determining the ith exposure parameter corresponding to the ith preset region, the processing module 101 is further configured to set the frame rate to n times of the initial frame rate.

[0106] Optionally, the SLAM device 100 further comprises a judging module 102, when j takes 2, …, m in turn, after the i-th frame image is acquired by using the i-th exposure parameter, the judging module 102 is further used for judging whether the second brightness difference value of the n preset regions of the i-th frame image is less than a second preset threshold; if the second brightness difference value is less than the second preset threshold, the frame rate is set as the initial frame rate, and another SLAM operation is adopted.

[0107] Optionally, when j takes 2, …, m in turn, after the n-th frame image is acquired by using the n-th exposure parameter, the judging module 102 is further used for judging whether the second brightness difference value of the n preset regions of the n-th frame image is less than a second preset threshold; if the second brightness difference value is less than the second preset threshold, the frame rate is set as the initial frame rate, and another SLAM operation is adopted.

[0108] Please refer to Figure 5 , based on the same inventive concept, the embodiment of the present application provides a schematic structural block diagram of an intelligent device 200, which can be used to implement the above-mentioned SLAM method. In the embodiment of the present application, the intelligent device 200 can be a robot, such as a cleaning robot, a sweeping robot, a service robot, a mobile robot, etc. In structure, the intelligent device 200 can include an image video acquisition device 210, a processor 220 and a memory 230.

[0109] The processor 220 is directly or indirectly electrically connected with the image video acquisition device 210 and the memory 230, to realize the transmission or interaction of data, for example, these elements can be electrically connected with each other through one or more communication buses or signal lines. Among them, the processor 220 can be an integrated circuit chip with signal processing capability. The processor 220 can also be a general-purpose processor, for example, it can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a discrete gate or transistor logic device, a discrete hardware component, which can realize or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. In addition, the general-purpose processor can be a microprocessor or any conventional processor.

[0110] The memory 230 can be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), and an Electric Erasable Programmable Read-Only Memory (EEPROM). The memory 230 is used to store a program, which the processor 220 executes after receiving an execution instruction.

[0111] It should be understood that Figure 5 The structure shown is only schematic, the intelligent device 200 provided by the embodiments of the present application can also have fewer or more components, or have a different configuration from that shown. Figure 5 In addition, Figure 5 The components shown can be implemented by software, hardware, or a combination thereof. Figure 5 It should be noted that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0112] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program performs the method provided in the foregoing embodiments when being executed.

[0113] The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available medium sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a Solid State Disk (SSD)), etc.

[0114]

[0115] ​In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely exemplary, for example, the division of the units is only a logical function division, and there can be another division manner for the actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrically, mechanically or in other forms.

[0116] In addition, the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0117] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0118] In this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations.

[0119] The above are only embodiments of the present application, and are not used to limit the protection scope of the present application. For those skilled in the art, various modifications and changes can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A SLAM method, characterized in that, include: When the first brightness difference value of the n preset regions of the current image is greater than the first preset threshold, m SLAM operations are performed, where n is a positive integer greater than or equal to 2 and m is a positive integer greater than or equal to 1. The j-th SLAM operation includes: Determine the i-th exposure parameter corresponding to the i-th preset region, where j takes values ​​of 1, ..., m in sequence; and i takes values ​​of 1, ..., n in sequence; The i-th frame image is obtained using the i-th exposure parameter; Extract the i-th set of feature points corresponding to the i-th preset region from the i-th frame image; The SLAM operation is performed based on the extracted i-th set of feature points; When j equals 1, before determining the i-th exposure parameter corresponding to the i-th preset region, the method further includes: Set the frame rate to n times the initial frame rate; When j takes values ​​of 2, ..., m sequentially, after obtaining the i-th frame image using the i-th exposure parameter, the method further includes: Determine whether the second brightness difference value of the n preset regions of the i-th frame image is less than a second preset threshold, wherein the second preset threshold is equal to or slightly less than the first preset threshold; If the second brightness difference value is less than the second preset threshold, the frame rate is set to the initial frame rate, and another SLAM operation is performed.

2. The method according to claim 1, characterized in that, Determining the i-th exposure parameter corresponding to the i-th preset region includes: Based on the current image, determine the i-th exposure parameter corresponding to the i-th preset region, where j equals 1; and / or Based on the (i-1)th frame image, determine the i-th exposure parameter corresponding to the i-th preset region, wherein when j equals 1 and i equals 1, the (i-1)th frame image is the current image; when j is greater than 1 and i equals 1, the (i-1)th frame image is the n-th frame image in the (j-1)th SLAM operation.

3. The method according to claim 1, characterized in that, The total number of preset regions is positively correlated with the device's maximum frame rate.

4. The method according to claim 1, characterized in that, The first brightness difference value is the brightness difference between any two adjacent preset areas.

5. A SLAM device, characterized in that, include: The processing module is configured to perform m SLAM operations when the first brightness difference value of n preset regions in the acquired current image is greater than a first preset threshold, where n is a positive integer greater than or equal to 2 and m is a positive integer greater than or equal to 1; wherein performing the j-th SLAM operation includes: determining the i-th exposure parameter corresponding to the i-th preset region, where i takes values ​​of 1, ..., n, and j takes values ​​of 1, ..., m; acquiring the i-th frame image using the i-th exposure parameter; extracting the i-th set of feature points corresponding to the i-th preset region from the i-th frame image; and performing the SLAM operation based on the extracted i-th set of feature points; The processing module is also used to set the frame rate to n times the initial frame rate before determining the i-th exposure parameter corresponding to the i-th preset region when j equals 1. The judgment module is used to determine whether the second brightness difference value of the n preset regions of the i-th frame image is less than a second preset threshold when j takes the values ​​2, ..., m in sequence, after the i-th frame image is obtained using the i-th exposure parameter. The second preset threshold is equal to or slightly less than the first preset threshold. If the second brightness difference value is less than the second preset threshold, the frame rate is set to the initial frame rate and another SLAM operation is performed.

6. A smart device, characterized in that, include: An image and video acquisition device, a processor, and a memory, wherein the processor is connected to the image and video acquisition device and the memory, respectively. The memory is used to store programs; The processor is used to run a program stored in the memory and perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a computer, performs the method as described in any one of claims 1-4.

Images