Mobile robot positioning and mapping method and device, equipment and storage medium

Abstract

The invention discloses a mobile robot positioning and mapping method, device and equipment and a storage medium, and relates to the technical field of computer vision and the technical field of intelligent robots, and the method comprises the steps: obtaining an original image sequence of a mobile robot in a target complex environment; wherein the original image sequence is composed of multiple frames of continuous original images; performing illumination compensation processing on the original image to obtain an illumination optimization image corresponding to the original image; performing glass area identification on the illumination optimization image to obtain a target optimization image corresponding to the original image; performing image optimization processing on the original image sequence according to the target optimization image corresponding to each original image to obtain a target image sequence; and according to the target image sequence, constructing a global map of the target complex environment, and determining a motion track of the mobile robot in the target complex environment. According to the method, the self-positioning accuracy of the mobile robot is improved, and the mapping accuracy of the mobile robot is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and intelligent robots, and particularly to a method, device, equipment and storage medium for mobile robot positioning and mapping. Background Art

[0002] Due to the need to improve production efficiency and the yearning for an intelligent lifestyle in modern society, mobile robots are widely used in people's production and life. In order to complete intelligent tasks in unknown working scenarios, mobile robots need to accurately locate themselves in unknown working scenarios and at the same time be able to construct high-quality environmental maps in unknown working scenarios to guide the mobile robots to move autonomously in unknown scenarios. This is the classic SLAM (Simultaneous Localization and Mapping) problem. Visual SLAM has attracted much attention because of its ability to use low-cost cameras for feature extraction and pose estimation.

[0003] However, traditional visual SLAM algorithms, such as ORB-SLAM3, still have significant limitations in unknown indoor complex environments (such as airport indoor environments and shopping mall indoor environments, etc.), mainly manifested as: when the lighting conditions change significantly, the accuracy of feature point extraction and matching decreases; due to the transparency and high reflectivity of the glass area, feature points are missing or mis-matched, affecting the accuracy of the mobile robot's own positioning and mapping. Summary of the Invention

[0004] The present invention provides a method, device, equipment and storage medium for mobile robot positioning and mapping to improve the accuracy of the mobile robot's self-positioning and the accuracy of the mobile robot's mapping.

[0005] According to an aspect of the present invention, a method for mobile robot positioning and mapping is provided, and the method includes:

[0006] Obtain an original image sequence of the mobile robot in a target complex environment; wherein, the original image sequence is composed of multiple consecutive original images;

[0007] Perform light compensation processing on the original images to obtain light-optimized images corresponding to the original images;

[0008] Perform glass area recognition on the light-optimized images to obtain target-optimized images corresponding to the original images;

[0009] According to the target-optimized images corresponding to the respective original images, perform image optimization processing on the original image sequence to obtain a target image sequence;

[0010] Construct a global map of the target complex environment based on the target image sequence, and determine the motion trajectory of the mobile robot in the target complex environment.

[0011] According to another aspect of the present invention, there is provided a mobile robot positioning and mapping device, which includes:

[0012] An original image sequence acquisition module, configured to acquire an original image sequence of the mobile robot in the target complex environment; wherein, the original image sequence is composed of multiple consecutive original images;

[0013] A lighting optimization image determination module, configured to perform lighting compensation processing on the original image to obtain a lighting optimization image corresponding to the original image;

[0014] A target optimization image determination module, configured to identify the glass area of the lighting optimization image to obtain a target optimization image corresponding to the original image;

[0015] A target image sequence determination module, configured to perform image optimization processing on the original image sequence according to the target optimization images corresponding to the respective original images to obtain a target image sequence;

[0016] A global map construction module, configured to construct a global map of the target complex environment based on the target image sequence, and determine the motion trajectory of the mobile robot in the target complex environment.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device includes:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the mobile robot positioning and mapping method of any embodiment of the present invention.

[0021] According to another aspect of the present invention, there is provided a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the mobile robot positioning and mapping method of any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, there is provided a computer program product, including a computer program, and the computer program implements the mobile robot positioning and mapping method of any embodiment of the present invention when executed by a processor.

[0023] The technical solution of the embodiment of the present invention includes: obtaining an original image sequence of a mobile robot in a target complex environment, where the original image sequence consists of multiple consecutive original images; performing illumination compensation processing on the original images to obtain illumination-optimized images corresponding to the original images; performing glass region recognition on the illumination-optimized images to obtain target-optimized images corresponding to the original images; performing image optimization processing on the original image sequence according to the target-optimized images corresponding to the respective original images to obtain a target image sequence; constructing a global map of the target complex environment based on the target image sequence, and determining the motion trajectory of the mobile robot in the target complex environment. The above technical solution performs illumination compensation processing on each original image in the original image sequence to obtain illumination-optimized images corresponding to each original image in the original image sequence, realizes the brightness adjustment of each original image in the original image sequence, reduces the influence of changes in illumination conditions in the target complex environment on feature point extraction and matching in the subsequent mapping process, thereby improving the feature point extraction ability and matching accuracy in the subsequent mapping process; then, performs glass region recognition on the illumination-optimized images corresponding to each original image in the original image sequence to obtain target-optimized images corresponding to each original image in the original image sequence, accurately recognizes the glass regions in each original image in the original image sequence, thereby solving the problem of missing or mismatched feature points caused by the transparency and high reflectivity of glass in the subsequent mapping process, and further improving the accuracy of the mobile robot's self-positioning and the accuracy of the mobile robot's mapping, that is, improving the accuracy of the motion trajectory of the mobile robot in the target complex environment and the accuracy of the global map of the target complex environment.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1A is a flowchart of a method for mobile robot positioning and mapping according to Embodiment 1 of the present invention;

[0027] Figure 1B is a set of exemplary diagrams for performing glass region recognition on an illumination-optimized image to obtain a target-optimized image according to Embodiment 1 of the present invention;

[0028] Figure 1C They are two distribution diagrams of feature points in different glass regions of image a provided in the first embodiment of the present invention;

[0029] Figure 2A It is a flowchart of a method for mobile robot positioning and mapping provided in the second embodiment of the present invention;

[0030] Figure 2B It is a comparison diagram of image brightness before and after light compensation for the same original image under different lighting environments provided in the second embodiment of the present invention;

[0031] Figure 2C It is the movement trajectory of a mobile robot in the glass region of a target complex environment provided in the second embodiment of the present invention;

[0032] Figure 3 It is a schematic structural diagram of a device for mobile robot positioning and mapping provided in the third embodiment of the present invention;

[0033] Figure 4 It is a schematic structural diagram of an electronic device for implementing the method for mobile robot positioning and mapping in the embodiment of the present invention. Detailed implementation manners

[0034] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0035] It should be noted that the terms "target", "original", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0036] In addition, it should be noted that in the technical solution of the present invention, the collection, storage, use, processing, transmission, provision, and disclosure of the original image sequence and the like comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0037] Embodiment 1

[0038] Figure 1A The figure is a flowchart of a method for mobile robot positioning and mapping provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation where a mobile robot accurately locates itself in an unknown indoor complex environment and automatically constructs an environmental map. This method can be executed by a mobile robot positioning and mapping device, which can be implemented in the form of hardware and / or software and can be configured in an electronic device. As Figure 1A shown, the method includes:

[0039] S101. Obtain the original image sequence of the mobile robot in the target complex environment; wherein, the original image sequence is composed of multiple consecutive original images.

[0040] Among them, the target complex environment refers to an unknown indoor complex environment for the mobile robot. For example, the target complex environment can be the indoor complex environment of a certain airport or the indoor complex environment of a certain shopping mall. The original image refers to the image directly captured by the target camera mounted on the mobile robot. Among them, the target camera refers to the camera mounted on the mobile robot; optionally, the target camera can be a single-sided camera or an RGB-D depth camera (Red Green Blue-Depth Camera).

[0041] Specifically, the original image sequence of the mobile robot in the target complex environment can be obtained through the target camera mounted on the mobile robot.

[0042] S102. Perform light compensation processing on the original image to obtain the light-optimized image corresponding to the original image.

[0043] Among them, the light-optimized image refers to the image obtained after the original image is subjected to light compensation processing. It should be noted that one frame of the original image corresponds to one light-optimized image.

[0044] Specifically, for each frame of the original image in the original image sequence, a preset light compensation model is used to perform light compensation processing on the original image to obtain the light-optimized image corresponding to the original image, so that the light-optimized images corresponding to each frame of the original image in the original image sequence can be obtained. Among them, the preset light compensation model is obtained by training a deep learning model with a large number of sample images.

[0045] S103. Identify the glass area in the light-optimized image to obtain the target optimized image corresponding to the original image.

[0046] Among them, the target optimized image refers to the image obtained after the glass area of the illumination optimized image is recognized. It should be noted that one illumination optimized image corresponds to one target optimized image, and one frame of the original image corresponds to one target optimized image.

[0047] Specifically, a glass detection network model can be used to recognize the glass area of the illumination optimized image to obtain a preliminary optimized image; the boundary of the initial glass area in the preliminary optimized image is optimized to obtain the target optimized image corresponding to the original image and containing the target glass area.

[0048] Among them, the glass detection network model refers to a neural network model used to recognize and segment the glass area in the image; optionally, the glass detection network model can be the GDNet (Geometric and Dynamic-aware Network) model. The preliminary optimized image refers to the image obtained after the illumination optimized image is processed by the glass detection network model. The initial glass area refers to the glass area in the preliminary optimized image. The target optimized image refers to the image obtained after the boundary of the glass area in the preliminary optimized image is optimized. The target glass area refers to the glass area in the target optimized image.

[0049] More specifically, for the illumination optimized image corresponding to each frame of the original image in the original image sequence, the illumination optimized image is input into the glass detection network model. After being processed by the glass detection network model, the preliminary optimized image corresponding to the illumination optimized image is obtained; then, the conditional random field CRF (conditional random field) is used to optimize the boundary of the initial glass area in the preliminary optimized image corresponding to the illumination optimized image to obtain the target optimized image corresponding to the original image and containing the target glass area. Among them, the energy function of the conditional random field CRF is as follows:

[0050] E(x) = ∑ i ψ u (x i ) + ∑ i<j ψ p (x i , x j );

[0051] Among them, E(x) represents the energy function of the conditional random field CRF; ψ u (x i ) represents the unary potential function, which is used to characterize the cost of the i-th pixel taking the label value x i ; ψ p (x i , x j ) represents the binary potential function, which is used to model the label consistency constraint between the i-th pixel and the j-th pixel; xi represents the label value of the i-th pixel point, and its value is 0 or 1. For example, if the i-th pixel point belongs to the glass area, then x i = 1; if the i-th pixel point does not belong to the glass area, then x i = 0.

[0052] Exemplarily, if the illumination-optimized image corresponding to the original image 1 in the original image sequence is Figure 1B the image a in, for Figure 1B the image a in, input the image a into the GDNet model. After being processed by the GDNet model, the preliminary optimized image corresponding to the image a is obtained, that is, Figure 1B the image b in; then, use the conditional random field CRF to optimize the boundary of the initial glass area (i.e., the white area in the image b) in the image b, and obtain the target optimized image containing the target glass area (i.e., the white area in the image c) corresponding to the original image 1 in the original image sequence, that is, Figure 1B the image c in.

[0053] Table 1

[0054]

[0055] Referring to Table 1, the experimental results show that, compared with the method of only using the GDNet model to identify the glass area in the illumination-optimized image, the technical solution of the embodiment of the present invention (that is, the method of using the GDNet model and the conditional random field CRF to identify the glass area in the illumination-optimized image) has the intersection over union (IoU) increased from 0.8156 to 0.8321, the pixel accuracy (PA) increased from 0.945 to 0.953, the weighted harmonic mean (denoted as F β ) increased from 0.884 to 0.897, while the mean absolute error (MAE) decreased from 0.059 to 0.052.

[0056] It can be understood that after the glass area in the illumination-optimized image is identified by the glass detection network model, the boundary of the identified glass area is optimized again by the conditional random field CRF, thereby improving the recognition accuracy of the glass area in the illumination-optimized image, improving the accuracy of the target optimized image, and solving the problem of missing feature points caused by the transparency and high reflectivity of the glass in the subsequent process (see Figure 1C ), or the problem of incorrect feature point matching, and further reducing the impact of the glass area in the image on the subsequent self-localization and map construction of the mobile robot. Through Figure 1C it can be seen that the same is for Figure 1CFeature points are extracted from the glass region of the middle image a (i.e., the white region in image a). Compared with the distribution map of feature points in the glass region of image a obtained by the prior art (i.e., ORB-SLAM3) (i.e., the image b in Figure 1C ), the distribution map of feature points in the glass region of image a obtained by the technical solution of the embodiment of the present invention (i.e., the image c in Figure 1C ) has more feature points and a more uniform distribution.

[0057] S104. According to the target optimized images corresponding to the original images, perform image optimization processing on the original image sequence to obtain a target image sequence.

[0058] Among them, the target image sequence refers to the image sequence obtained after the original image sequence is subjected to image optimization processing. Specifically, each frame of the original image in the original image sequence is replaced with its corresponding target optimized image, so as to obtain the target image sequence.

[0059] S105. According to the target image sequence, construct a global map of the target complex environment and determine the motion trajectory of the mobile robot in the target complex environment.

[0060] Specifically, feature extraction and matching can be performed on the target image sequence to obtain a feature matching result; according to the feature matching result, determine the camera pose of the target camera carried on the mobile robot; according to the feature matching result and the camera pose, construct a global map of the target complex environment and determine the motion trajectory of the mobile robot in the target complex environment.

[0061] Among them, the feature matching result is used to reflect the matching situation of feature points between two adjacent frames of target optimized images in the target image sequence; optionally, the feature matching result includes matching feature point pairs, matching similarity, and the credibility of the matching result. Among them, the matching similarity is used to measure the similarity degree between the matching feature point pairs. The camera pose refers to the position and orientation of the target camera in the target complex environment.

[0062] More specifically, the ORB (Oriented FAST and Rotated BRIEF) algorithm can be used to extract features from each frame of the target optimized image in the target image sequence, obtaining the ORB feature points corresponding to each frame of the target optimized image in the target image sequence; among them, the ORB feature points consist of two parts: key feature points and descriptors; the descriptors are used to describe the pixel information around the key feature points. Then, the descriptors of the ORB feature points corresponding to two adjacent frames of the target optimized image in the target image sequence are matched to obtain a feature matching result; based on the feature matching result and the camera pose estimation algorithm, such as the PnP (Perspective-n-Point) algorithm, the camera pose of the target camera mounted on the mobile robot is determined; a series of local maps in the target complex environment are generated according to the feature matching result and the camera pose; and the generated series of local maps are fused according to the camera pose to obtain the global map of the target complex environment. In addition, the position of the mobile robot at each moment in the target complex environment can also be directly determined according to the camera pose; the positions of the mobile robot at each moment in the target complex environment are connected to obtain the motion trajectory of the mobile robot in the target complex environment.

[0063] Optionally, in order to more evenly extract the feature points in the glass area and non-glass area of each frame of the target optimized image in the target image sequence, a density factor used in feature extraction can be set according to the actual business requirements and combined with the expert experience in this field. For example, the density factor can be set to 0.03.

[0064] Optionally, in order to obtain a more accurate global map of the target complex environment, the BA (Bundle Adjustment) algorithm can also be used to optimize the global map of the target complex environment to obtain the optimized global map of the target complex environment.

[0065] The technical solution of the embodiment of the present invention is as follows: obtain the original image sequence of the mobile robot in the target complex environment; wherein, the original image sequence consists of multiple consecutive original images; perform illumination compensation processing on the original images to obtain the illumination-optimized images corresponding to the original images; perform glass region recognition on the illumination-optimized images to obtain the target-optimized images corresponding to the original images; perform image optimization processing on the original image sequence according to the target-optimized images corresponding to each original image to obtain the target image sequence; construct the global map of the target complex environment according to the target image sequence, and determine the motion trajectory of the mobile robot in the target complex environment. In the above technical solution, by performing illumination compensation processing on each original image in the original image sequence, the illumination-optimized images corresponding to each original image in the original image sequence are obtained, realizing the brightness adjustment of each original image in the original image sequence, reducing the influence of the change of illumination conditions in the target complex environment on the extraction and matching of feature points in the subsequent mapping process, thereby improving the extraction ability and matching accuracy of feature points in the subsequent mapping process; then, perform glass region recognition on the illumination-optimized images corresponding to each original image in the original image sequence to obtain the target-optimized images corresponding to each original image in the original image sequence, accurately identifying the glass regions in each original image in the original image sequence, thereby solving the problem of missing or mis-matching of feature points caused by the transparency and high reflectivity of glass in the subsequent mapping process, and further improving the accuracy of the mobile robot's self-positioning, improving the accuracy of the mobile robot's mapping, that is, improving the accuracy of the motion trajectory of the mobile robot in the target complex environment and improving the accuracy of the global map of the target complex environment.

[0066] Embodiment 2

[0067] Figure 2A The flowchart of a mobile robot positioning and mapping method provided by Embodiment 2 of the present invention. On the basis of the above embodiment, this embodiment further optimizes "perform illumination compensation processing on the original images to obtain the illumination-optimized images corresponding to the original images", and provides an optional implementation solution. It should be noted that for the parts not detailed in the embodiments of the present invention, reference may be made to the relevant descriptions of other embodiments. As Figure 2A shown, the method includes:

[0068] S201. Obtain the original image sequence of the mobile robot in the target complex environment; wherein, the original image sequence consists of multiple consecutive original images.

[0069] S202. Determine the median brightness of the original images.

[0070] Among them, the median brightness refers to the image brightness determined by the median of the pixel brightness in the original image. Specifically, for each frame of the original image in the original image sequence, the median brightness of the original image is determined through the following median brightness determination formula:

[0071]

[0072] Among them, V m represents the median brightness of the original image; k represents the brightness value; C(k) represents the total number of pixel points in the original image whose brightness value is less than or equal to k; C(255) represents the total number of pixel points in the original image; represents selecting the smallest k from multiple C(k) that is greater than or equal to as the median brightness of the original image. For example, for the original image 1 in the original image sequence, if there are in the original image 1 and k1 < k2, then the median brightness of the original image 1 is k1.

[0073] S203. Determine the target brightness range corresponding to the original image in the target complex environment.

[0074] Among them, the target brightness range is used to limit the brightness interval of the original image in the target complex environment. It should be noted that the target brightness ranges corresponding to all the original images in the target complex environment are the same. For example, the target brightness range corresponding to each frame of the original image in a certain indoor complex environment of an airport is (100, 120).

[0075] Specifically, the brightness feature point distribution map in the target complex environment can be obtained; according to the number of feature points at different brightness levels in the brightness feature point distribution map, the brightness interval with the largest number of feature points is selected from the brightness feature point distribution map as the target brightness range corresponding to the original image in the target complex environment, so as to ensure that more feature points can be extracted from the original image within a certain brightness range. Among them, the abscissa of the brightness feature point distribution map is brightness, and the ordinate is the number of feature points, which is used to reflect the number of feature points that can be extracted from the image in the target complex environment at different brightness levels.

[0076] S204. Obtain the original brightness, original hue, and original saturation of each pixel point in the original image.

[0077] Among them, the original brightness refers to the brightness of the pixel point before adjustment; the original hue refers to the hue of the pixel point before adjustment; the original saturation refers to the saturation of the pixel point before adjustment.

[0078] Specifically, for each pixel in each frame of the original image sequence, obtain the original RGB value of the pixel, denoted as (R1, G1, B1); perform normalization processing on the original RGB value of the pixel to obtain the normalized RGB value of the pixel, denoted as (r, g, b); where r = R1 / 255, g = G1 / 255, b = B1 / 255; take the maximum value in the normalized RGB value of the pixel as the original brightness of the pixel, that is, V = max(r, g, b); detect whether the original brightness of the pixel is 0; if it is detected that the original brightness of the pixel is 0, then the original saturation of the pixel is 0; if it is detected that the original brightness of the pixel is not 0, then according to the original brightness of the pixel and the normalized RGB value, use the following saturation determination formula to determine the original saturation of the pixel:

[0079]

[0080] Where S represents the original saturation of the pixel; V represents the original brightness of the pixel; min(r, g, b) represents the minimum value among r, g, and b.

[0081] After that, detect whether the original brightness of the pixel is equal to the minimum value among r, g, and b; if it is detected that the original brightness of the pixel is equal to the minimum value among r, g, and b, that is, if it is detected that V = min(r, g, b), then the original hue of the pixel is 0; if it is detected that the original brightness of the pixel is not equal to the minimum value among r, g, and b, that is, if it is detected that V ≠ min(r, g, b), then continue to detect whether the original brightness of the pixel is equal to r; if it is detected that the original brightness of the pixel is equal to r, then according to the original brightness of the pixel and the normalized RGB value, use the following first hue determination formula to determine the original hue of the pixel:

[0082]

[0083] Where H represents the original hue of the pixel. If it is detected that the original brightness of the pixel is not equal to r, then continue to detect whether the original brightness of the pixel is equal to g; if it is detected that the original brightness of the pixel is equal to g, then according to the original brightness of the pixel and the normalized RGB value, use the following second hue determination formula to determine the original hue of the pixel:

[0084]

[0085] If the original brightness of the pixel is detected to be not equal to g, continue to detect whether the original brightness of the pixel is equal to b; if the original brightness of the pixel is detected to be equal to b, determine the original hue of the pixel according to the original brightness of the pixel and the normalized RGB values through the following third hue determination formula:

[0086]

[0087] After that, detect whether the original hue of the pixel is less than 0; if so, adjust the original hue of the pixel through the following hue adjustment formula to obtain the adjusted original hue of the pixel:

[0088] H1 = H + 360;

[0089] where H1 represents the adjusted original hue of the pixel.

[0090] S205. Determine the average brightness of the original image according to the original brightness of each pixel in the original image.

[0091] The average brightness refers to the average value of the brightness of all pixels in the original image. Specifically, for each frame of the original image in the original image sequence, determine the average brightness of the original image according to the original brightness of each pixel in the original image through the following average brightness determination formula:

[0092]

[0093] where μ V represents the average brightness of the original image, N represents the total number of pixels in the original image; i represents the i-th pixel in the original image; V i represents the original brightness of the i-th pixel in the original image.

[0094] S206. Determine the adjusted brightness corresponding to the original image according to the average brightness, median brightness, and target brightness range.

[0095] The adjusted brightness refers to a value used to dynamically adjust the brightness of the original image. Specifically, if it is detected that the average brightness is less than the brightness lower limit of the target brightness range, the difference between the brightness lower limit and the median brightness is used as the adjusted brightness corresponding to the original image; if it is detected that the average brightness is greater than the brightness upper limit of the target brightness range, the difference between the brightness upper limit and the median brightness is used as the adjusted brightness corresponding to the original image; otherwise, the average value of the brightness lower limit and the brightness upper limit is calculated to obtain the intermediate brightness; according to the intermediate brightness and the median brightness, determine the adjusted brightness corresponding to the original image.

[0096] More specifically, for each original image in the original image sequence, the lower limit of the target brightness range corresponding to the original image in the target complex environment is denoted as targetMin; the upper limit of the target brightness range is denoted as targetMax; and the average brightness of the original image is denoted as μ V If it is detected that μ V < targetMin, then the difference between the lower brightness limit and the median brightness is used as the adjusted brightness corresponding to the original image, that is, ΔV = targetMin - V m ; where ΔV represents the adjusted brightness corresponding to the original image; V m represents the median brightness of the original image; if it is detected that μ V > targetMax, then the difference between the upper brightness limit and the median brightness is used as the adjusted brightness corresponding to the original image, that is, ΔV = targetMax - V m ; if it is detected that targetMin ≤ μ V ≤ targetMax, then the lower and upper brightness limits of the target brightness range are averaged to obtain the intermediate brightness, that is where V α represents the intermediate brightness; the difference between the intermediate brightness and the median brightness is used as the adjusted brightness corresponding to the original image, that is, ΔV = V α - V m .

[0097] It can be understood that the adjusted brightness corresponding to the original image is determined based on the average brightness, median brightness of the original image, and the target brightness range corresponding to the original image, making the subsequent brightness adjustment of the original image smoother and more controllable, and avoiding the adjusted original image from being too bright or too dark.

[0098] S207. Perform brightness adjustment on the original image according to the adjusted brightness, as well as the original brightness, original hue, and original saturation of each pixel point in the original image, to obtain the illumination-optimized image corresponding to the original image.

[0099] Specifically, for each pixel point in the original image, the target brightness of the pixel point is determined according to the adjusted brightness and the original brightness of the pixel point; the RGB value of the pixel point is adjusted according to the target brightness, as well as the original hue and original saturation of the pixel point, to obtain the adjusted RGB value of the pixel point; and the original image is subjected to brightness adjustment according to the adjusted RGB values of each pixel point, to obtain the illumination-optimized image corresponding to the original image.

[0100] Among them, the target brightness refers to the brightness obtained after adjusting the original brightness of the pixel. More specifically, for each pixel in the original image, according to the adjusted brightness corresponding to the original image and the original brightness of the pixel, the target brightness of the pixel is determined through the following brightness adjustment formula:

[0101] V1 = clip(V + ΔV, 0, 255);

[0102] Among them, V1 represents the target brightness of the pixel; V represents the original brightness of the pixel; ΔV represents the adjusted brightness corresponding to the original image; clip(V + ΔV, 0, 255) is used to limit V1 within the brightness range of [0, 255]. If V + ΔV < 0, then V1 = 0; if V + ΔV > 255, then V1 = 255; if 0 ≤ V + ΔV ≤ 255, then V1 = V + ΔV.

[0103] After that, the original hue of the pixel is normalized to obtain the normalized original hue of the pixel, that is where H 1 represents the normalized original hue of the pixel, and H represents the original hue of the pixel. It should be noted that the value of H 1 is within the range of [0, 1].

[0104] After that, according to the target brightness and the original saturation of the pixel, the first color difference of the pixel is determined through the following first color difference determination formula:

[0105] C = V1 × S;

[0106] Among them, C represents the first color difference of the pixel; V1 represents the target brightness of the pixel; S represents the original saturation of the pixel. After that, according to the normalized original hue of the pixel and the first color difference of the pixel, the second color difference of the pixel is determined through the following second color difference determination formula:

[0107] X = C × (1 - |(H × 6) mod 2 - 1|);

[0108] Among them, X represents the second color difference of the pixel. After that, according to the target brightness and the first color difference of the pixel, the adjustment factor corresponding to the pixel is determined through the following adjustment factor determination formula:

[0109] m = V1 - C;

[0110] Among them, m represents the adjustment factor corresponding to the pixel. If it is detected that the normalized original hue of the pixel is within the first preset range If it is within, then use the first color difference of this pixel point as the R value of this pixel point, that is, R = C; use the second color difference of this pixel point as the G value of this pixel point, that is, G = X; use 0 as the B value of this pixel point, that is, B = 0. If it is detected that the normalized original hue of this pixel point is within the second preset range If it is within, then use the second color difference of this pixel point as the R value of this pixel point, that is, R = X; use the first color difference of this pixel point as the G value of this pixel point, that is, G = C; use 0 as the B value of this pixel point, that is, B = 0. If it is detected that the normalized original hue of this pixel point is within the third preset range If it is within, then use 0 as the R value of this pixel point, that is, R = 0; use the first color difference of this pixel point as the G value of this pixel point, that is, G = C; use the second color difference of this pixel point as the B value of this pixel point, that is, B = X. If it is detected that the normalized original hue of this pixel point is within the fourth preset range If it is within, then use 0 as the R value of this pixel point, that is, R = 0; use the second color difference of this pixel point as the G value of this pixel point, that is, G = X; use the first color difference of this pixel point as the B value of this pixel point, that is, B = C. If it is detected that the normalized original hue of this pixel point is within the fifth preset range If it is within, then use the second color difference of this pixel point as the R value of this pixel point, that is, R = X; use 0 as the G value of this pixel point, that is, G = 0; use the first color difference of this pixel point as the B value of this pixel point, that is, B = C. If it is detected that the normalized original hue of this pixel point is within the sixth preset range If it is within, then use the first color difference of this pixel point as the R value of this pixel point, that is, R = C; use 0 as the G value of this pixel point, that is, G = 0; use the second color difference of this pixel point as the B value of this pixel point, that is, B = X. After that, according to the adjustment factor corresponding to this pixel point, through the following RGB value adjustment formula, adjust the RGB value of this pixel point obtained above to within [0, 255] to obtain the adjusted RGB value of this pixel point, that is:

[0111]

[0112] Among them, R2 represents the R value in the adjusted RGB value of this pixel point; G2 represents the G value in the adjusted RGB value of this pixel point; B2 represents the B value in the adjusted RGB value of the pixel point.

[0113] After that, for each frame of the original image in the original image sequence, replace the original RGB value of each pixel point in this original image with the adjusted RGB value, so as to obtain the illumination-optimized image corresponding to this original image. Similarly, the illumination-optimized images corresponding to each original image in the original image sequence can be obtained.

[0114] See Figure 2B, by performing illumination compensation processing on the same original image under different illumination environments, the experimental results show (see Table 2) that after performing illumination compensation processing on the original image in low-light environments or shadow environments, the brightness of the original image in low-light environments or shadow environments is improved; after performing illumination compensation processing on the original image in overexposed environments, the brightness of the original image in overexposed environments is reduced, so that the brightness distribution of the illumination-optimized image corresponding to the obtained original image becomes more uniform, enhancing the image contrast of the illumination-optimized image and reducing the influence of illumination condition changes on feature point extraction and matching in the subsequent mapping process, thereby improving the feature point extraction ability and matching accuracy in the subsequent mapping process.

[0115] It should be noted that Figure 2B in it, dark represents the original image in a low-light environment, whose corresponding brightness range is (0, 255), and its brightness average value is 5.43; shade represents the original image in a shadow environment, whose corresponding brightness range is (0, 255), and its brightness average value is 47.52; overexposure represents the original image in an overexposed environment, whose corresponding brightness range is (0, 255), and its brightness average value is 130.45; trained dark represents the illumination-optimized image corresponding to dark, whose corresponding target brightness range is (100, 120), and its brightness average value is 105.43; trained shade represents the illumination-optimized image corresponding to shade, whose corresponding target brightness range is (100, 120), and its brightness average value is 108.48; trained overexposure represents the illumination-optimized image corresponding to overexposure, whose corresponding target brightness range is (100, 120), and its brightness average value is 108.57.

[0116] Table 2

[0117]

[0118] S208. Identify the glass area of the illumination-optimized image to obtain the target optimized image corresponding to the original image.

[0119] S209. According to the target optimized image corresponding to each original image, perform image optimization processing on the original image sequence to obtain the target image sequence.

[0120] S210. According to the target image sequence, construct the global map of the target complex environment and determine the motion trajectory of the mobile robot in the target complex environment.

[0121] The technical solution of the embodiment of the present invention includes: obtaining an original image sequence of a mobile robot in a target complex environment, where the original image sequence consists of multiple consecutive original images; determining the median brightness of the original images; determining the target brightness range corresponding to the original images in the target complex environment; obtaining the original brightness, original hue, and original saturation of each pixel point in the original images; determining the average brightness of the original images according to the original brightness of each pixel point in the original images; determining the adjusted brightness corresponding to the original images according to the average brightness, median brightness, and target brightness range; performing brightness adjustment on the original images according to the adjusted brightness, as well as the original brightness, original hue, and original saturation of each pixel point in the original images, to obtain the illumination-optimized images corresponding to the original images; performing glass region recognition on the illumination-optimized images to obtain the target optimized images corresponding to the original images; performing image optimization processing on the original image sequence according to the target optimized images corresponding to the respective original images to obtain a target image sequence; constructing a global map of the target complex environment according to the target image sequence, and determining the movement trajectory of the mobile robot in the target complex environment. The above technical solution realizes the brightness adjustment of each original image in the original image sequence by performing illumination compensation processing on each original image in the original image sequence, and obtains the illumination-optimized image corresponding to each original image in the original image sequence, reducing the influence of illumination changes in the target complex environment on the extraction and matching of feature points in the subsequent mapping process, thereby improving the extraction ability and matching accuracy of feature points in the subsequent mapping process; then, performing glass region recognition on the illumination-optimized image corresponding to each original image in the original image sequence to obtain the target optimized image corresponding to each original image in the original image sequence, accurately identifying the glass region in each original image in the original image sequence, thereby solving the problem of missing or mismatching feature points caused by the transparency and high reflectivity of glass in the subsequent mapping process, and further improving the accuracy of the mobile robot's self-positioning and the accuracy of the mobile robot's mapping, that is, improving the accuracy of the movement trajectory of the mobile robot in the target complex environment (see Figure 2C ), improving the accuracy of the global map of the target complex environment.

[0122] It should be noted that Figure 2C the red dotted rectangle frame in [reference figure] shows the movement trajectory intercepted by the mobile robot in the glass region of the target complex environment; Figure 2C the black dotted trajectory in [reference figure] refers to the actual movement trajectory of the mobile robot in the glass region of the target complex environment; where the target complex environment is a complex indoor environment of an airport; Figure 2C the blue solid trajectory in [reference figure] refers to the movement trajectory of the mobile robot in the glass region of the target complex environment determined by the existing technology (i.e., ORB-SLAM3), Figure 2CThe green solid line trajectory in [it] refers to the movement trajectory of the mobile robot within the glass area of the target complex environment determined by the technical solution of the embodiment of the present invention. By Figure 2C It can be seen that the technical solution of the embodiment of the present invention improves the accuracy of the movement trajectory of the mobile robot in the target complex environment.

[0123] Embodiment III

[0124] Figure 3 FIG. is a schematic structural diagram of a mobile robot positioning and mapping device provided in Embodiment III of the present invention. This embodiment is applicable to the situation where a mobile robot accurately locates itself in an unknown indoor complex environment and automatically constructs an environmental map. The device can be implemented in the form of hardware and / or software and can be configured in an electronic device. As Figure 3 shown, the device includes:

[0125] An original image sequence acquisition module 301, configured to acquire an original image sequence of the mobile robot in the target complex environment; wherein, the original image sequence is composed of multiple consecutive original images;

[0126] A lighting optimization image determination module 302, configured to perform lighting compensation processing on the original image to obtain a lighting optimization image corresponding to the original image;

[0127] A target optimization image determination module 303, configured to perform glass area recognition on the lighting optimization image to obtain a target optimization image corresponding to the original image;

[0128] A target image sequence determination module 304, configured to perform image optimization processing on the original image sequence according to the target optimization images corresponding to the respective original images to obtain a target image sequence;

[0129] A global map construction module 305, configured to construct a global map of the target complex environment according to the target image sequence and determine the movement trajectory of the mobile robot in the target complex environment.

[0130] The technical solution of the embodiment of the present invention includes: obtaining an original image sequence of a mobile robot in a target complex environment, where the original image sequence is composed of multiple consecutive original images; performing illumination compensation processing on the original images to obtain illumination-optimized images corresponding to the original images; performing glass region recognition on the illumination-optimized images to obtain target-optimized images corresponding to the original images; performing image optimization processing on the original image sequence according to the target-optimized images corresponding to the respective original images to obtain a target image sequence; constructing a global map of the target complex environment according to the target image sequence, and determining the motion trajectory of the mobile robot in the target complex environment. The above technical solution performs illumination compensation processing on each original image in the original image sequence to obtain illumination-optimized images corresponding to each original image in the original image sequence, realizing the brightness adjustment of each original image in the original image sequence, reducing the influence of changes in illumination conditions in the target complex environment on feature point extraction and matching in the subsequent mapping process, thereby improving the feature point extraction ability and matching accuracy in the subsequent mapping process; then, performing glass region recognition on the illumination-optimized images corresponding to each original image in the original image sequence to obtain target-optimized images corresponding to each original image in the original image sequence, accurately identifying the glass regions in each original image in the original image sequence, thus solving the problem of missing or mismatched feature points caused by the transparency and high reflectivity of glass in the subsequent mapping process, and further improving the accuracy of the mobile robot's self-positioning and the accuracy of the mobile robot's mapping, that is, improving the accuracy of the motion trajectory of the mobile robot in the target complex environment and the accuracy of the global map of the target complex environment.

[0131] Optionally, the illumination-optimized image determination module 302 includes:

[0132] A median brightness determination unit for determining the median brightness of the original image;

[0133] A target brightness range determination unit for determining the target brightness range corresponding to the original image in the target complex environment;

[0134] A pixel point color information acquisition unit for acquiring the original brightness, original hue, and original saturation of each pixel point in the original image;

[0135] An average brightness determination unit for determining the average brightness of the original image according to the original brightness of each pixel point in the original image;

[0136] An adjusted brightness determination unit for determining the adjusted brightness corresponding to the original image according to the average brightness, median brightness, and target brightness range;

[0137] An illumination optimization image determination unit, configured to perform brightness adjustment on the original image according to the adjusted brightness, as well as the original brightness, original hue, and original saturation of each pixel point in the original image, so as to obtain an illumination optimization image corresponding to the original image.

[0138] Optionally, the adjusted brightness determination unit is specifically configured to:

[0139] If it is detected that the average brightness is less than the lower limit of the target brightness range, then use the difference between the lower limit of the brightness and the median brightness as the adjusted brightness corresponding to the original image;

[0140] If it is detected that the average brightness is greater than the upper limit of the target brightness range, then use the difference between the upper limit of the brightness and the median brightness as the adjusted brightness corresponding to the original image;

[0141] Otherwise, perform an averaging process on the lower limit of the brightness and the upper limit of the brightness to obtain an intermediate brightness;

[0142] Determine the adjusted brightness corresponding to the original image according to the intermediate brightness and the median brightness.

[0143] Optionally, the illumination optimization image determination unit is specifically configured to:

[0144] For each pixel point in the original image, determine the target brightness of the pixel point according to the adjusted brightness and the original brightness of the pixel point;

[0145] Adjust the RGB value of the pixel point according to the target brightness, as well as the original hue and original saturation of the pixel point, to obtain the adjusted RGB value of the pixel point;

[0146] Perform brightness adjustment on the original image according to the adjusted RGB values of each pixel point, so as to obtain an illumination optimization image corresponding to the original image.

[0147] Optionally, the target optimization image determination module 303 is specifically configured to:

[0148] Use a glass detection network model to identify the glass area in the illumination optimization image to obtain a preliminary optimization image;

[0149] Perform boundary optimization on the initial glass area in the preliminary optimization image to obtain a target optimization image containing the target glass area corresponding to the original image.

[0150] Optionally, the global map construction module 305 is specifically configured to:

[0151] Perform feature extraction and matching on the target image sequence to obtain a feature matching result;

[0152] Determine the camera pose of the target camera mounted on the mobile robot according to the feature matching result;

[0153] Construct a global map of the target complex environment based on the feature matching results and the camera pose, and determine the motion trajectory of the mobile robot in the target complex environment.

[0154] The mobile robot positioning and mapping device provided by the embodiments of the present invention can execute the mobile robot positioning and mapping method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing each mobile robot positioning and mapping method.

[0155] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0156] Embodiment 4

[0157] Figure 4 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0158] As Figure 4 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0159] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0160] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the mobile robot positioning and mapping method.

[0161] In some embodiments, the mobile robot positioning and mapping method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the mobile robot positioning and mapping method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the mobile robot positioning and mapping method by any other suitable means (e.g., by means of firmware).

[0162] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a special or general programmable processor, can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0163] A computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing device, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0164] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0165] In order to provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0166] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected with each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0167] A computing system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0168] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0169] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A mobile robot positioning and mapping method, characterized in that: include: Acquire an original image sequence of the mobile robot in a target complex environment; wherein the original image sequence is composed of multiple frames of continuous original images; Performing illumination compensation processing on the original image to obtain an illumination optimized image corresponding to the original image; Performing glass area recognition on the illumination optimized image to obtain a target optimized image corresponding to the original image; According to the target optimized images corresponding to the original images, the original image sequence is subjected to image optimization processing to obtain a target image sequence; A global map of the target complex environment is constructed according to the target image sequence, and a motion trajectory of the mobile robot in the target complex environment is determined.

2. The method according to claim 1, characterized in that: The performing illumination compensation processing on the original image to obtain an illumination optimized image corresponding to the original image includes: Determining the median brightness of the original image; Determine the target brightness range corresponding to the original image in the target complex environment; Obtaining the original brightness, original hue and original saturation of each pixel in the original image; Determine the average brightness of the original image according to the original brightness of each pixel in the original image; Determining an adjusted brightness corresponding to the original image according to the average brightness, the median brightness, and the target brightness range; The brightness of the original image is adjusted according to the adjusted brightness, and the original brightness, original hue and original saturation of each pixel in the original image to obtain a lighting optimized image corresponding to the original image.

3. The method according to claim 2, characterized in that The determining, according to the average brightness, the median brightness and the target brightness range, the adjusted brightness corresponding to the original image includes: If it is detected that the average brightness is less than the lower limit of the brightness of the target brightness range, the difference between the lower limit of the brightness and the median brightness is used as the adjusted brightness corresponding to the original image; If it is detected that the average brightness is greater than the upper brightness limit of the target brightness range, the difference between the upper brightness limit and the median brightness is used as the adjusted brightness corresponding to the original image; Otherwise, the lower brightness limit and the upper brightness limit are averaged to obtain an intermediate brightness; An adjusted brightness corresponding to the original image is determined according to the middle brightness and the median brightness.

4. The method according to claim 2, characterized in that: The step of adjusting the brightness of the original image according to the adjusted brightness and the original brightness, original hue, and original saturation of each pixel in the original image to obtain a lighting optimized image corresponding to the original image includes: For each pixel in the original image, determining a target brightness of the pixel according to the adjusted brightness and the original brightness of the pixel; According to the target brightness, and the original hue and original saturation of the pixel, the RGB value of the pixel is adjusted to obtain an adjusted RGB value of the pixel; The brightness of the original image is adjusted according to the adjusted RGB value of each pixel to obtain a lighting optimized image corresponding to the original image.

5. The method according to claim 1, characterized in that The step of performing glass area recognition on the illumination optimized image to obtain a target optimized image corresponding to the original image includes: Using a glass detection network model, the glass region is identified on the illumination optimized image to obtain a preliminary optimized image; The boundary of the initial glass region in the preliminary optimized image is optimized to obtain a target optimized image including the target glass region corresponding to the original image.

6. The method according to claim 1, characterized in that The step of constructing a global map of the target complex environment according to the target image sequence and determining a motion trajectory of the mobile robot in the target complex environment includes: Extracting and matching features of the target image sequence to obtain feature matching results; Determining the camera pose of the target camera carried by the mobile robot according to the feature matching result; A global map of the target complex environment is constructed according to the feature matching results and the camera pose, and a motion trajectory of the mobile robot in the target complex environment is determined.

7. A mobile robot positioning and mapping device, characterized in that: include: The original image sequence acquisition module is used to acquire the original image sequence of the mobile robot in the target complex environment; wherein the original image sequence is composed of multiple frames of continuous original images; A lighting optimized image determination module, used to perform lighting compensation processing on the original image to obtain a lighting optimized image corresponding to the original image; A target optimized image determination module, used to perform glass area recognition on the illumination optimized image to obtain a target optimized image corresponding to the original image; A target image sequence determination module is used to perform image optimization processing on the original image sequence according to the target optimization image corresponding to each original image to obtain a target image sequence; The global map construction module is used to construct a global map of the target complex environment according to the target image sequence, and determine the motion trajectory of the mobile robot in the target complex environment.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the mobile robot positioning and mapping method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the mobile robot positioning and mapping method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the mobile robot positioning and mapping method according to any one of claims 1 to 6 is implemented.

Images