A slam method in a dark light environment

Abstract

The application discloses a SLAM method in a dark light environment, the method comprises the following steps: acquiring inertial feature information based on a feature extraction system; acquiring an image in a dark light environment to acquire a dark light image; processing the dark light image to acquire an exposure image; acquiring feature points in the exposure image to acquire image information; acquiring line segment features in the image based on the image information; and positioning a device and constructing a spatial map based on the line segment features in the image and the inertial feature information. The SLAM method in the dark light environment in the prior art has the problems of excessive mapping and positioning errors and low data processing efficiency. Therefore, the mapping of the environment and the positioning accuracy of the device are comprehensively ensured, and the data processing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and robot positioning, and particularly relates to a SLAM method in a dark environment. BACKGROUND

[0002] SLAM technology has been widely applied in many robot fields due to its spatial mapping and device positioning capabilities. The core of this technology is to obtain images of the surrounding scene, and then perform mapping operations on the surrounding space based on accurate image information. However, in current technical applications, the main problems found include: 1. Insufficient feature matching rate. Based on the measured data, it is found that the feature matching degree of the traditional ORB-SLAM3 system decreases to 39.7% in a Lux<5 environment, resulting in a positioning error accumulation rate of ±2.38m / 10m for the device; 2. Limited dynamic range. In the extraction of environmental information in a dark environment, the image needs to be exposed. In the current single-exposure imaging scheme, 63.2% of the overexposure / underexposure area is generated in the alternating light and dark scene; 3. Real-time calculation bottleneck. Currently, various methods based on SLAM technology have been developed. For example, in the Mobile SLAM scheme, although a feature point screening strategy is used to obtain surrounding image information, the ARM Cortex-A53 platform can only achieve a processing speed of 12.7fps, which cannot meet the technical solution used in industrial scenarios. The above three problems result in high mapping error rate and positioning error rate of the SLAM method, which has gradually become unsuitable for high-precision scenarios.

[0003] In summary, how to establish a SLAM method in a dark environment to improve mapping and positioning accuracy is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] In order to improve the mapping accuracy and device positioning accuracy in a dark environment, the application proposes a SLAM method in a dark environment, which specifically includes:

[0005] A SLAM method in a dark environment, the method comprising:

[0006] Based on a feature extraction system, inertial feature information is obtained;

[0007] Obtain images in a dark environment to obtain dark images;

[0008] Process the dark images to obtain exposure images;

[0009] Obtain feature points in the exposure images to obtain image information;

[0010] Based on the image information, line segment features in the image are obtained;

[0011] Based on the line segment features in the image and the inertial feature information, positioning of the device and construction of a spatial map are performed.

[0012] Optionally, the feature extraction system obtains inertial feature information, including:

[0013] Based on a feature extraction system provided on the device, inertial feature information of the device is obtained.

[0014] The inertial feature information includes device speed, movement deflection angle, and acceleration.

[0015] Optionally, the image in the dark environment is obtained by obtaining a dark image, including:

[0016] A camera is provided on the device, and continuous images are obtained.

[0017] Based on the continuous images, an image in the dark environment is obtained, and a dark image is obtained.

[0018] Optionally, the dark image is processed to obtain an exposure image, including:

[0019] Based on the illumination matrix, an illumination map of the dark image is obtained.

[0020] Based on the pre-processed illumination map, a preset exposure image is obtained.

[0021] The preset exposure image is verified to obtain an exposure image.

[0022] Optionally, the illumination matrix is obtained based on the light matrix, and the illumination map of the dark image is obtained, including:

[0023] An illumination matrix of a space in which the device is located is obtained, and a determination equation of the illumination matrix is:

[0024] ;

[0025] Wherein, represents the illumination matrix; represents the input image; c represents the contrast; R , G , B represents three primary color components of pixel values in the input dark image;

[0026] Based on the illumination matrix, an illumination map of the dark image is obtained, and a determination equation of the illumination map of the dark image is:

[0027] ;

[0028] Wherein, represents an illumination map; represents a filter coefficient; represents a derivative of a local pixel in a direction dimension; d represents a derivative of a local pixel in a direction dimension; d represents a direction dimension; h and v represents a vertical direction and a horizontal direction; I x , y represents a pixel at any point on an image.

[0029] Optionally, the pre-processing illumination map is used to obtain a preset exposure image, including:

[0030] obtaining an operating parameter of a dark-light image acquisition device, obtaining a response model of the dark-light image acquisition device, and obtaining a BTF model;

[0031] The BTF model equation is:

[0032] ;

[0033] wherein, represents an exposure result; represents a basic exposure function; represents an original image; represents an exposure rate; β and γ is a parameter related to the exposure rate in the BTF model; k

[0034] wherein, The equation of is:

[0035] ;

[0036] wherein, a and b represents a fixed camera parameter.

[0037] Optionally, the preset exposure image is verified to obtain an exposure image, including:

[0038] obtaining a pixel average value of a region image in the preset exposure image, and the pixel average value of the region image equation is:

[0039] ;

[0040] wherein, f x and Mean x represents a pixel average value of a region image; x represents a pixel value of a pixel point of a region image.​​​​i and j respectively represent the horizontal and vertical indices of the region image; N respectively represent the horizontal and vertical indices of the region image; I i，j x represent the region image of the dark light image.

[0041] Based on the pixel average value, the global underexposure image in the dark light image is obtained, and the determination equation of the global underexposure image is:

[0042] ;

[0043] wherein, Q x represent the global underexposure image, T x represent the total sum of the region point pixel values of the region image.

[0044] Optionally, the feature points in the exposure image are obtained to obtain image information, including:

[0045] The exposure rate of all region images in the exposure image is obtained;

[0046] Based on the exposure rate, the information amount of the region image is obtained, and the information amount determination equation is:

[0047] ;

[0048] wherein, p i represent the exposure rate of the i-th region image; i represent the number index of the region image; i represent the number index of the region image; N H R represent the image entropy of the exposure image; represent the maximum exposure rate of the exposure image, k represent the exposure rate;

[0049] Based on the information amount of the region image, the region image used for feature extraction is obtained to obtain a feature extraction region image;

[0050] The feature extraction region image is decomposed to obtain a plurality of decomposed images, and the decomposed image with illumination invariance is determined based on an illumination invariant filter, and the illumination invariant filter equation is:

[0051] ;

[0052] wherein, D ​​​​​iif represents the Hamming distance between the decomposed images; f iif represents the distance between two decomposed images described by the illumination invariance at different brightness determined by the Hamming distance; σ 2 represents the variance between the decomposed images;

[0053] Obtain the decomposed image with the information amount higher than the preset information amount to obtain the image carrying information.

[0054] Optionally, the image information in the image is obtained based on the image information, and the line segment feature in the image is obtained, and the line segment feature in the image is obtained.

[0055] Obtain the image information in the image carrying information.

[0056] Based on the EDLines algorithm, the line segment feature in the image information is obtained.

[0057] Optionally, the line segment feature in the image and the inertial feature information are used to perform positioning and space map construction of the device, and the line segment feature in the image and the inertial feature information are used to perform positioning and space map construction of the device.

[0058] Based on all the line segment features, the spatial relationship between the line segments is obtained.

[0059] Based on the spatial relationship between the line segments, the scene image of the dark light environment is constructed.

[0060] Based on the scene image of the dark light environment, the space scene is constructed.

[0061] Obtain the acquisition time of the dark light image corresponding to the scene image of the dark light image to obtain the image acquisition time point.

[0062] Obtain the extraction time of the inertial feature information to obtain the feature acquisition time point.

[0063] Based on the image acquisition time point and the feature acquisition time point, the feature extraction system and the corresponding dark light image are aligned on the time axis.

[0064] Based on the inertial feature information and the space coordinate system set on the space scene, the spatial position of the device is obtained.

[0065] The beneficial effects of the present application include:

[0066] 1. Improve the mapping accuracy. In the technical solution of the present application, based on the adopted dual exposure algorithm, the effective feature point density is greatly improved after exposure processing of the obtained dark light image. Through actual detection, the feature point density can be improved to 82.3% of the normal light level. On the basis of guaranteeing the number of effective feature points, the mapping result accuracy obtained based on the obtained effective feature points can meet the high-precision technical application scene.

[0067] 2. Improve the light robustness feature extraction performance. In the technical solution of the present application, a new exposure algorithm is adopted to perform exposure processing on the dark light image collected in the dark light environment, and based on the obtained exposure processing result, the feature points in the exposure image are determined. At this time, the extracted features can be considered as being obtained under normal light conditions, and the light robustness features can be fully improved.

[0068] 3. Improve the processing performance. In the technical solution of the present application, after obtaining the feature points in the image processing, the EDLines algorithm is used to identify the line segments in the image to improve the construction efficiency of the scene. Based on the obtained construction result, the inertial characteristics of the device are combined to realize simultaneous mapping and positioning, thereby improving the information processing performance of the whole system. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings used in the present application or the prior art will be briefly introduced as follows. Obviously, the following description is only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. The drawings are used to provide further understanding of the present disclosure and constitute a part of the specification, and together with the following detailed description, serve to explain the present disclosure, but do not constitute a limitation on the present disclosure. In the drawings:

[0070] Figure 1 A SLAM method flow chart in a dark light environment provided by an embodiment of the present application;

[0071] Figure 2 A SLAM system schematic diagram in a dark light environment provided by an embodiment of the present application. DETAILED DESCRIPTION

[0072] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application. In addition, in the embodiments of the present application, “first”, “second”, and the like are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence.

[0073] The current SLAM method has been widely used in industrial systems, but this method still has some application problems. First, the information processing effect in a dark light environment is weak, and information loss often occurs. Once this problem occurs, it is difficult to achieve accurate composition, and the positioning performance of the related equipment will also be reduced, which is difficult to meet the requirements of all-weather and all-environment industrial applications. Second, the information processing efficiency is low. For composition and positioning functions, a large amount of information is needed for information support, and various types of information need to be responded quickly. However, in the implementation of the current technical solution, the processing efficiency of image processing and feature extraction is low, and it is difficult to ensure that the obtained image information can be processed quickly, which is particularly incompatible with the device positioning accuracy guarantee. Finally, the robustness of the feature extraction process is insufficient. In industrial production, the light may change, which requires that the SLAM method can obtain environmental and device position information regardless of the light environment. However, in the current method, it is easy to have the problem that the method cannot quickly identify and extract dark light environment information when the light environment suddenly changes to a dark light environment, resulting in insufficient robustness.

[0074] To solve the problems in the prior art, the present application discloses a SLAM method in a dark light environment, as shown in Figure 1 The SLAM method flow chart in a dark light environment provided by the embodiments of the present application specifically includes:

[0075] S110, obtaining inertial feature information based on a feature extraction system.

[0076] S120, obtaining an image in a dark light environment to obtain a dark light image.

[0077] S130, processing the dark light image to obtain an exposure image.

[0078] S140, obtaining feature points in the exposure image to obtain image information.

[0079] S150, obtaining line segment features in the image based on the image information.

[0080] S160, based on the line segment feature in the image and the inertial feature information, positioning and space map construction of the device are performed.

[0081] The purpose of all the above steps is to construct a SLAM method, and to ensure that the obtained method can realize accurate and efficient feature extraction of dark light patterns to realize precise mapping and device positioning.

[0082] In the following, the specific technical solutions of all the above steps will be described in detail:

[0083] As described in step S110, the technical specific technical solution of this step includes:

[0084] Based on the feature extraction system provided on the device, the inertial feature information of the device is obtained.

[0085] The inertial feature information includes device speed, movement deflection angle, and acceleration.

[0086] Among them, for the feature extraction process, the entire SLAM algorithm framework runs on a movable single camera device, which can be extended to a device equipped with an inertial measurement unit.

[0087] Based on the inertial measurement unit carried, the inertial feature information of the current device carrying the algorithm framework can be obtained.

[0088] As described in step S120, the specific technical solution of this step includes:

[0089] A camera is provided on the device, and a continuous image is obtained.

[0090] Based on the continuous image, an image in a dark light environment is obtained to obtain a dark light image.

[0091] Among them, the camera provided on the movable device is usually a monocular camera.

[0092] As described in step S130, the specific technical solution of this step includes:

[0093] Based on the illumination matrix, the illumination map of the dark light image is obtained.

[0094] Based on the preprocessed illumination map, a preset exposure image is obtained.

[0095] The preset exposure image is verified to obtain an exposure image.

[0096] Among them, for the obtained image, the monocular camera provided on the device is used to obtain a dark light image.

[0097] Wherein, for the acquired dark light image, exposure processing is performed thereon, so as to obtain an exposure image.

[0098] Wherein, for the acquired exposure image, there is a high possibility that although exposure is performed, there are still bright light regions and dark light regions on the image, wherein the dark light region usually carries less image information, so in the judgment of the scene, the dark light region is no longer identified, but is identified as a non-information carrying region, and then only information in the bright light region is acquired, so as to realize verification of the exposure image.

[0099] For all steps in step S130, specifically:

[0100] S131, based on the illumination matrix, the illumination map of the dark light image is acquired, including:

[0101] The illumination matrix of the space where the device is located is acquired, and the determination equation of the illumination matrix is:

[0102] ;

[0103] Wherein, B ( x ) represents the illumination matrix; I c ( x ) represents the input image; c represents the contrast; R , G , B represents three primary color components of the pixel value in the input dark light image;

[0104] Based on the illumination matrix, the illumination map of the dark light image is acquired, and the determination equation of the illumination map of the dark light image is:

[0105] ;

[0106] Wherein, represents the illumination map; represents the filter coefficient; represents derivation of the dimension in the direction of the local pixel; d represents the direction dimension; d represents the direction dimension; h and v represent the longitudinal direction and the transverse direction; I ( x , y ) represents a pixel at any point on the image.

[0107] The purpose of this step is to process the obtained dark light image, and in the acquisition of the result, actually a double exposure algorithm is performed, so as to obtain the result based on the algorithm.

[0108] Among them, the dual exposure algorithm: for the dual exposure of the dark light image collected by the monocular camera has been restored to the original normal image. The algorithm module simplifies the original multiple exposure framework and introduces biological heuristic to propose a real-time running dual exposure algorithm. The original multiple exposure consists of sample generator, sample sampler and fuser. The dual exposure algorithm proposed by this algorithm simplifies the original sample generator through the characteristics that the weight of human eye to different brightness interval will automatically adjust the brightness of the image. For the original multiple exposure sample generator, multiple samples are generated, and the sample sampler is used to sample the generated samples to discard the samples that do not meet the requirements, and finally the sample fuser is used for fusion to obtain the final image. This method is relatively complex in terms of use, and it is difficult to meet the rapid processing demand of image information. By using the biological characteristics that human beings will automatically calculate the weight of different brightness areas, the algorithm simplifies the above process. The dual exposure algorithm no longer generates multiple samples with different brightness, but directly calculates the brightness and generates an exposure sample using the brightness. Finally, the fuser is used for fusion to obtain the final image.

[0109] The specific formula of the dual exposure algorithm is:

[0110] The overall formula is:

[0111] ;

[0112] Among them, and output and input images respectively. is the image fusion weight, is the exposure function.

[0113] From the overall formula, three parts determine the final result of the algorithm: (1) gray dual exposure evaluator; (2) gray dual exposure generator; (3) gray dual sampling device;

[0114] The formula of the gray dual exposure evaluator (1) is as follows:

[0115] ;

[0116] Among them T is the scene illumination evaluation map, μ is a parameter that controls the degree of enhancement. When μ = 0, the output image is equal to the input image, that is, no enhancement is performed. When μ = 1, both underexposed and well-exposed pixels will be enhanced. When μ > 1, the pixels may be saturated, and the output image may lose details. Therefore, the parameterμ to obtain an enhanced mirror image of a specific environment.

[0117] To obtain an illumination map of an image, we usually need to solve the following optimization problem:

[0118] ;

[0119] where, B ( x ) denotes the illumination matrix; I c ( x ) denotes the input image; c denotes the contrast; R , G , B denotes the three primary color components of the pixel values in the input dark image. The illumination image needs to preserve the texture structure and remove the noise information in the image, so it is optimized by solving the following way T :

[0120] ;

[0121] first-order derivative filter contains (horizontal) and (vertical). M is the weight matrix, λ is the coefficient. The first term of the equation minimizes the difference between the initial image L and the refined image T , while the second term preserves the smoothness of T . But this dual exposure algorithm improves the original algorithm based on the characteristics of grayscale images, and the specific formula is as follows:

[0122] ;

[0123] where B is the matrix based on the characteristics of grayscale, and this algorithm uses 0.5 as the standard of no difference brightness in [0, 1]. is the input image after filtering. In order to reduce complexity, we use the following formula to approximate the above formula:

[0124] ;

[0125] Finally, this algorithm assumes that the T we get is infinitely close to the standard illumination. We get the illumination map T as follows:

[0126] ;

[0127] From the above formula, it can be seen that the efficiency can be improved by changing the image size. Since the change of image scale will not change the brightness of the original image area, the application inserts and adjusts the image to 32*32.

[0128] For the second step in step S130, the technical solution described in S132 includes:

[0129] Obtaining the running parameters of the dark light image acquisition device, obtaining the response model and BTF model of the dark light image acquisition device;

[0130] The BTF model equation is:

[0131] ;

[0132] Wherein, represents the exposure result; represents the basic exposure function; represents the original image; represents the exposure rate; β and γ is the parameter related to the exposure rate k in the BTF model;

[0133] Wherein, the equation is:

[0134] ;

[0135] Wherein, a and b represent the fixed camera parameters.

[0136] For the third step in S130, i.e. S133, it specifically includes:

[0137] Obtaining the pixel average value of the region image in the preset exposure image, the pixel average value equation of the region image is:

[0138] ;

[0139] Wherein, f ( x ) and Mean ( x ) represent the pixel average value of the region image; x represents the pixel value of the pixel point of the region image; i and j represent the horizontal and vertical indexes of the region image respectively; N represents the total amount of the horizontal and vertical indexes of the region image; I i，j ( x ) represents the region image of the dark light image.

[0140] Based on the pixel average value, a global underexposed image in the dark light image is obtained, and the determination equation of the global underexposed image is:

[0141] ;

[0142] wherein, Q x represents the global underexposed image, T x represents the regional point pixel value sum of the regional image.

[0143] This part realizes the double exposure generator by means of the camera response model. The camera response model is composed of two parts: the camera response function (CRF) model and the BTF model. The parameters of the CRF model are only determined by the camera, while the parameters of the BTF model are determined by the camera and the exposure rate. If the pixel value is linearly scaled before the traditional gray coefficient correction, the generated image will be very close to the well-exposed real image. Therefore, the BTF model can be described using a two-parameter function as follows:

[0144] ;

[0145] wherein β and γ are the parameters related to the exposure rate k in the BTF model. Since some camera manufacturers design the exposure as a gamma curve, it can perfectly fit these cameras. When γ = 1, the CRF model is a two-parameter function, and the BTF model is a nonlinear function. Since the BTF is nonlinear for most cameras, we mainly consider the case of γ = 1. The solution of g in the BTF is:

[0146] ;

[0147] It is assumed that no information about the camera is provided, and fixed camera parameters (a = -0.3293, b = 1.1258) suitable for most cameras are used.

[0148] The algorithm excludes well-exposed pixels and obtains a global underexposed image. Although the standard value of the gray image is 0.5, which can be used to judge whether it is well-exposed, the histogram distribution of the low-light image is mostly concentrated at the bottom, so using a hard threshold will cause overexposure and slow down the program running time. Therefore, this algorithm uses the pixel average value of the regional image to determine whether the pixels in the region are well-exposed, as shown in the following formula:

[0149] ​​ ;

[0150] ;

[0151] where Q(x) is the evaluation function, f(x) is the average function to obtain the region pixel indication. After obtaining the underexposed region, the algorithm can provide more information to the human according to the fact that the visibility of the well-exposed image is higher than that of the underexposed image. Therefore, the optimal k should provide the maximum amount of information. In order to measure the amount of information and quickly obtain the result, the algorithm uses one-dimensional image entropy to measure the information contained in the exposed image, which is defined as:

[0152] ;

[0153]

[0154] Since the image entropy increases first and then decreases with the increase of the exposure rate, therefore can be solved by a one-dimensional minimizer. In addition, considering the real-time performance, the image size is adjusted to 32*32 when optimizing .

[0155] As step S140 described, the purpose of this step is to further verify the acquired images after exposure, in order to find the images carrying more scene information, including:

[0156] Obtain the exposure rate of all region images in the exposed image;

[0157] Based on the exposure rate, obtain the information amount of the region image, and the information amount determination equation is:

[0158] ;

[0159] where, p i represents the exposure rate of the i-th region image; i represents the number index of the region image; i represents the total amount of the number index of the region image; N represents the total amount of the number index of the region image; H R represents the image entropy of the exposed image; represents the maximum exposure rate amount of the exposed image, k represents the exposure rate;

[0160] Based on the information amount of the region image, obtain the region image for feature extraction, to obtain the feature extraction region image;

[0161] ​The feature extraction region image is decomposed to obtain a plurality of decomposed images, and a decomposed image with illumination invariance is determined based on an illumination invariance filter, and the equation of the illumination invariance filter is:

[0162] ;

[0163] wherein, D iif indicates a Hamming distance between the decomposed images; f iif indicates a distance between two decomposed images described by the illumination invariance under different brightnesses by using the Hamming distance; σ 2 indicates a variance between the decomposed images;

[0164] The decomposed image with the information amount higher than the preset information amount is obtained to obtain an image carrying information.

[0165] The filter algorithm is used to solve the problem of feature point tracking. The current popular optical flow method uses local region brightness invariance to track feature points, but the image repaired by the exposure algorithm will have inconsistency in brightness. The illumination invariance filter filters out the points with brightness change to enhance the robustness of the optical flow method. The working algorithm of the filter is as follows: the illumination characteristics of the feature are obtained mainly by calculating the descriptor of the feature point. If the feature point has the property of illumination invariance, the feature point is retained, otherwise the feature point is removed, and the optical flow method is not used for tracking in the future.

[0166] The mathematical expression and proof process of the illumination invariance filter are as follows:

[0167] ;

[0168] ;

[0169] wherein, is the variance of the region (here we choose the size of 8*8). The Hamming distance is used to determine the distance between two same regions described by the brightness invariance under different brightnesses.

[0170] Proof: let D iif be marked as D and . Only when and , it can be proved that

[0171] ;

[0172] For D iif , the above formula can be written as:

[0173] ;

[0174] is a constant, as shown below:

[0175] ;

[0176] Since ,

[0177] ;

[0178] As described in step S150, after obtaining the verified exposure image, the key information in the surrounding scene image can be obtained based on the exposure image, so feature information needs to be obtained based on this method, including:

[0179] Obtain image information in the image carrying information;

[0180] Based on the EDLines algorithm, obtain the line segment features in the image information.

[0181] Among them, this part uses the EDLines algorithm based on fuzzy Confucius, which is used to solve the problem of difficult line feature detection in dark light state. EDLines algorithm has a large number of adjustable hyperparameters, which can change the performance of the algorithm by adjusting these hyperparameters in the face of specific environments. These adjustments mostly depend on experience and cannot be self-adaptive. This algorithm abstracts specific phenomena as fuzzy laws, and adjusts hyperparameters through the fuzzy laws to obtain self-adaptivity. The specific workflow is to extract the line segment features in the image from the input two-dimensional image through the EDLines based on fuzzy control.

[0182] The regulation library table of fuzzy control is as follows:

[0183] ;

[0184] The threshold formula is as follows:

[0185] ;

[0186] The FIS rule base is defined in Table 1 using the detected line number. The EDLines edge detector threshold is determined using the number of feature points detected in the feature line segment detection module. When the number of detected lines is higher / lower / close to the expected value, the EDLines edge detector threshold is adjusted accordingly.

[0187] As shown in step S160, after obtaining the space environment image in which the device is located and extracting the feature information in the image, it is necessary to perform mapping and position determination based on the obtained image and the inertial information of the device, including:

[0188] Based on all the line segment features, the spatial relationship between the line segments is obtained.

[0189] Based on the spatial relationship between the line segments, a scene image of the dark light environment is constructed.

[0190] Based on the scene image of the dark light environment, a space scene is constructed.

[0191] Obtain the acquisition time of the dark light image corresponding to the scene image of the dark light image, and obtain the image acquisition time point.

[0192] Obtain the extraction time of the inertial feature information, and obtain the feature acquisition time point.

[0193] Based on the image acquisition time point and the feature acquisition time point, the feature extraction system and the corresponding dark light image are aligned on the time axis.

[0194] Based on the inertial feature information and the space coordinate system set on the space scene, the spatial position of the device is obtained.

[0195] Wherein, on the basis of having obtained the line segment features, the correlation between each line segment is obtained, and based on the correlation, the environment space information is obtained and mapping processing is performed.

[0196] Wherein, for the positioning operation of the device, based on the inertial information of the device, the inertial feature parameters generated by the motion of the device at different positions in the entire scene are determined, and then the position information is obtained based on the inertial feature parameters, so that the device positioning can be performed.

[0197] For all the above technical solutions, as shown in Figure 2 A SLAM system in a dark light environment provided by the embodiment of the present application is shown.

[0198] The beneficial effects of the present application include:

[0199] 1. Improve the mapping accuracy. In the technical solution of the present application, after the obtained dark light image is processed by the dual exposure algorithm, the effective feature point density is greatly improved. Through actual detection, the feature point density can be improved to 82.3% of the normal light level. On the basis of ensuring the number of effective feature points, when mapping is performed based on the obtained effective feature points, the accuracy of the obtained mapping result can meet the high-precision technical application scene.

[0200] 2、 improve the performance of illumination robust feature extraction. In the technical solution of the present application, a new exposure algorithm is used to perform exposure processing on the dark light image collected in the dark light environment, and based on the obtained exposure processing result, the feature points in the exposure image are determined. At this time, the extracted features can be considered similar to those obtained under normal lighting conditions, which can fully improve the illumination robustness of the features.

[0201] 3、 improve the processing performance. In the technical solution of the present application, in the processing of the image, after obtaining the feature points, the line segments existing in the image are identified based on the EDLines algorithm to improve the construction efficiency of the scene, and based on the obtained construction result, the inertial characteristics of the device are combined to realize the simultaneous realization of mapping and positioning, thereby improving the information processing performance of the whole system.

[0202] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by computer program instruction related hardware, and the aforementioned computer program can be stored in a non-volatile storage medium. When the computer program is executed, the steps of the method embodiments are executed. Alternatively, when the integrated units of the present application are realized in the form of software function modules and sold or used as independent products, they can also be stored in a non-volatile storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products, which are stored in a non-volatile storage medium and include instructions for making an electronic device (which can be a personal computer, a server, a network device, etc.) execute all or part of the methods described in the embodiments of the present application.

[0203] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A SLAM method for low-light environments, the method comprising: Based on the feature extraction system, inertial feature information is obtained; Acquire images in low-light environments; obtain low-light images. The low-light image is processed to obtain an exposed image; Feature points in the exposed image are obtained to acquire image information; Based on the image information, obtain the line segment features in the image; Based on the line segment features and inertial feature information in the image, the device is located and a spatial map is constructed. The process of processing the low-light image to obtain the exposed image includes: Based on the illumination matrix, the illumination map of the dark light image is obtained; Preprocess the lighting map to obtain a preset exposure image; The preset exposure image is verified to obtain the exposure image; Based on the illumination matrix, the illumination map of the dark-light image is obtained, including: Obtain the illumination matrix of the space where the device is located. The equation for determining the illumination matrix is: ， in, Represents the lighting matrix; represents the input image; c represents the contrast; R, G, and B represent the three primary color components of the pixel values ​​in the input low-light image; Based on the illumination matrix, the illumination map of the dark-light image is obtained, and the equation for determining the illumination map of the dark-light image is: ， in, Represents a lighting diagram; Indicates the filter coefficients; This represents the derivative with respect to the dimension along the d-direction of a local pixel; d represents the directional dimension; h and v represent the vertical and horizontal directions; I(x, y) represents any pixel on the image; The preprocessing of the illumination map, to obtain a preset exposure image, includes: Obtain the operating parameters of the low-light image acquisition device, and obtain the response model and BTF model of the low-light image acquisition device; The BTF model equation is as follows: ， in, Indicates the exposure result; Represents the basic exposure function; Represents a lighting diagram; β and γ represent the exposure rate; they are parameters related to the exposure rate k in the BTF model. in, The equation is: ， Where a and b represent fixed camera parameters; The step of verifying the preset exposure image and obtaining the exposure image includes: Obtain the average pixel value of a region in the preset exposure image. The equation for the average pixel value of the region image is: ， in, f ( x )and Mean ( x () represents the average pixel value of the region image; x Represents the pixel values ​​of pixels in a region of the image; i and j These represent the horizontal and vertical indices of the region image, respectively; N This represents the total horizontal and vertical indexes of the region image; I i，j ( x () represents a region of a low-light image; Based on the average pixel value, a global underexposed image is obtained in the low-light image. The equation for determining the global underexposed image is: ， Where Q(x) represents the globally underexposed image, and T(x) represents the sum of the pixel values ​​of the region points in the region image.

2. The SLAM method for low-light environments according to claim 1, characterized in that, The feature extraction system acquires inertial feature information, including: Based on the feature extraction system installed on the device, the inertial feature information of the device is obtained; The inertial characteristic information includes the device speed, movement deflection angle, and acceleration.

3. The SLAM method for low-light environments according to claim 1, characterized in that, The acquisition of images in low-light environments, including the acquisition of low-light images, comprises: Set up a camera on the device and capture continuous images; Based on the continuous images, images under low-light conditions are obtained to obtain low-light images.

4. The SLAM method for low-light environments according to claim 1, characterized in that, The step of obtaining feature points in the exposed image to acquire image information includes: Obtain the exposure rate of all regions in the exposed image; Based on the exposure rate, the information content of the region image is obtained, and the equation for determining the information content is: ， in, p i Indicates the first i Exposure rate of the image in each region; i Indicates the index of the region image; N This represents the total horizontal and vertical indexes of the region image; H ( R ) represents the image entropy of the exposed image; This indicates the maximum exposure rate of the exposed image. k Indicates exposure rate; Based on the information content of the region image, a region image for feature extraction is obtained, resulting in a feature extraction region image. The image of the feature extraction region is decomposed to obtain multiple decomposed images. A light-invariant filter is used to determine the decomposed image with light invariance properties. The equation of the light-invariant filter is: ， in, D iif Represents the Hamming distance between the decomposed images; f iif This means using Hamming distance to determine the distance between two decomposed images described by illumination invariance under different brightness levels; σ 2 This represents the variance between the decomposed images; The image after decomposition, where the information content is higher than a preset information content, is obtained to obtain an image carrying information.

5. The SLAM method for low-light environments according to claim 1, characterized in that, The step of obtaining line segment features in the image based on the image information includes: To extract image information from an image that carries information; Based on the EDLines algorithm, line segment features in the image information are obtained.

6. The SLAM method for low-light environments according to claim 1, characterized in that, The process of locating the device and constructing a spatial map based on the line segment features and inertial feature information in the image includes: Based on all the line segment features, obtain the spatial relationships between the line segments; Based on the spatial relationship between the line segments, a scene image of a low-light environment is constructed; Based on the scene images of the low-light environment, a spatial scene is constructed; Obtain the acquisition time of the dark light image corresponding to the scene image of the dark light image, and obtain the image acquisition time point; The extraction time of the inertial feature information is obtained to obtain the feature acquisition time point; Based on the image acquisition time point and the feature acquisition time point, align the feature extraction system and the corresponding dark light image on the time axis; Based on the inertial feature information and the spatial coordinate system set on the spatial scene, the spatial position of the device is obtained.

Images