Method for constructing regional map, electronic device and storage medium

Abstract

The application discloses a method for constructing a regional map, an electronic device and a storage medium. The method comprises the following steps: obtaining current track point features and current environment features of a target object in a target region about a current round, and historical track point features and historical environment features about a historical round; wherein the motion collection track of the current round and the historical round at least partially overlaps; in response to the current environment features and the historical environment features satisfying a same region condition, optimizing the historical track point features by using the current track point features to obtain updated track point features; and constructing a regional map of the target region based on the updated track point features. The above scheme can improve the accuracy of the construction of the regional map.

Description

Technical Field

[0001] This application relates to the field of map construction technology, and in particular to a method for constructing regional maps, an electronic device, and a storage medium. Background Technology

[0002] With the continuous development of Simultaneous Localization and Mapping (SLAM) technology, it has been widely applied in more and more technical fields.

[0003] In the field of robotics, SLAM technology enables robots to start from an unknown location in an unknown environment, locate their own position and posture by repeatedly observing environmental features, and then build an incremental map of the surrounding environment based on their own position, thereby achieving simultaneous localization and map building.

[0004] Currently, for certain areas, such as indoor areas of public places and shopping malls, the accuracy of map construction is relatively low due to factors such as the complex environment of these areas. Summary of the Invention

[0005] The main technical problem addressed by this application is to provide a method for constructing regional maps, an electronic device, and a storage medium that can improve the accuracy of regional map construction.

[0006] To address the aforementioned issues, the first aspect of this application provides a method for constructing a regional map. This method includes: acquiring current trajectory point features and current environmental features of a target object in a target region with respect to the current wheel, as well as historical trajectory point features and historical environmental features with respect to historical wheels; wherein the motion acquisition trajectories of the current wheel and the historical wheel at least partially overlap; in response to the current environmental features and historical environmental features satisfying the condition of being in the same region, optimizing the historical trajectory point features using the current trajectory point features to obtain updated trajectory point features; and constructing a regional map of the target region based on the updated trajectory point features.

[0007] To address the aforementioned problems, a second aspect of this application provides an electronic device comprising a memory and a processor coupled to each other, wherein the memory stores program data and the processor executes the program data to implement any step of the aforementioned method for constructing a regional map.

[0008] To address the aforementioned problems, a third aspect of this application provides a computer-readable storage medium storing program data executable by a processor, the program data being used to implement any step of the above-described method for constructing a regional map.

[0009] The above scheme acquires the current trajectory point features and current environmental features of the target object in the target area with respect to the current wheel, as well as the historical trajectory point features and historical environmental features of the historical wheels; wherein the motion acquisition trajectories of the current wheel and the historical wheels at least partially overlap. In response to the condition that the current environmental features and historical environmental features satisfy the same region condition, the historical trajectory point features are optimized using the current trajectory point features to obtain updated trajectory point features. This provides a method for loop closure detection using environmental features, and when the same region condition is met, it can determine whether the target object has returned to the previous area, providing a loop closure detection constraint for trajectory point optimization. Based on this, further optimization of the historical trajectory point features using the current trajectory point features can improve the accuracy of the updated trajectory point features. Therefore, based on the updated trajectory point features, a regional map of the target area can be constructed, improving the accuracy of the regional map construction.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Among them:

[0012] Figure 1 This is a flowchart illustrating an embodiment of the method for constructing a regional map according to this application;

[0013] Figure 2 This is a schematic diagram of the structure of an embodiment of the data acquisition platform of this application;

[0014] Figure 3 This is a flowchart illustrating an embodiment of step S11 of this application;

[0015] Figure 4 This is the gait of the application. Figure 1 Example diagram of the embodiment;

[0016] Figure 5 This is a flowchart illustrating another embodiment of step S11 of this application;

[0017] Figure 6 This is a flowchart illustrating an embodiment of step S12 of this application;

[0018] Figure 7 This is a flowchart illustrating another embodiment of step S12 of this application;

[0019] Figure 8This is a schematic diagram of an embodiment of the apparatus for constructing a regional map according to this application;

[0020] Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0021] Figure 10 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0024] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0025] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] This application provides the following embodiments, and each embodiment is described in detail below.

[0027] It is understood that the method for constructing the regional map in this application can be executed by an electronic device, which can be any device with processing capabilities, such as a mobile phone, computer, server, etc., and this application does not impose any restrictions on it.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the method for constructing a regional map according to this application. The method may include the following steps:

[0029] S11: Obtain the current trajectory point features and current environmental features of the target object in the target area with respect to the current wheel, as well as the historical trajectory point features and historical environmental features with respect to the historical wheels; wherein the motion acquisition trajectories of the current wheel and the historical wheels have at least partial overlap.

[0030] The target objects can be pedestrians, mobile robots, vehicles, etc., and can be determined according to the specific application scenario. For example, the target area can be an indoor area, such as a shopping mall or museum in a public place. For instance, when the target area is an indoor area, the target objects can be pedestrians or mobile robots, etc.; when the target area is an underground parking lot, the target objects can be vehicles, etc. This application does not impose any restrictions on this.

[0031] This application uses indoor SLAM as an example for illustration. In indoor pedestrian SLAM, the trajectory point features of pedestrians and environmental features can be collected. Specifically, millimeter-wave radar equipment can be used to collect environmental features, which are used to represent the motion information of surrounding objects relative to the target object. In addition, odometry sensors such as IMU (Inertial Measurement Unit) can be used to collect trajectory point features.

[0032] In some application scenarios, when collecting motion data, taking indoor pedestrians as an example, pedestrians can hold a data collection platform and walk indoors. To provide more material for subsequent pedestrian trajectory optimization, pedestrians can repeat the same or similar walking trajectory at least two times to obtain multiple rounds of collected motion data. The collected motion data can be saved offline for subsequent processing. This motion data includes raw point cloud data and mileage data.

[0033] During motion data acquisition, the mobile phone's IMU (Induction Unit) and millimeter-wave radar can simultaneously collect trajectory features of pedestrian movement and environmental features of the surrounding environment. The mobile phone's IMU is a combination of a gyroscope, magnetometer, and accelerometer; combining these three data points allows for the calculation of the pedestrian's posture, speed, and distance from the starting point, among other trajectory features. Millimeter-wave radar can obtain the distance and relative speed between the pedestrian and surrounding objects, thus constructing an environmental profile of the environment during the pedestrian's movement.

[0034] For example, please refer to Figure 2 The data acquisition platform consists of a mobile phone equipped with an IMU, a millimeter-wave radar, a DCA (Data Collection Analysis) data acquisition board, a power bank, and a computer stick. The mobile phone's IMU collects pedestrian mileage data. The millimeter-wave radar board and the DCA data acquisition board acquire the raw ADC (Analog-to-Digital Converter) data from the millimeter-wave radar, i.e., raw point cloud data. The power bank supplies power to the millimeter-wave radar and the computer stick, which issues acquisition commands to the millimeter-wave radar and collects radar data in real time. Ultimately, this data acquisition platform yields two types of data: pedestrian mileage data (acceleration and heading angle) and the raw ADC data from the millimeter-wave radar, i.e., raw point cloud data.

[0035] The target object can undergo multiple rounds of motion data collection, and in each round, it can move according to the motion collection trajectory. At least two rounds of motion collection trajectories must have at least partial overlap, such as identical or similar trajectories. While identical trajectories are not required, following the same road is acceptable to facilitate subsequent road modeling and map construction. This application does not impose restrictions on the motion collection trajectory for multiple rounds of motion data.

[0036] By acquiring the target object's mileage data and raw point cloud data for the current wheel within the target area, we can obtain the current trajectory point features and current environmental features of the current wheel. Similarly, by acquiring the target object's mileage data and raw point cloud data for previous wheels within the target area, we can obtain the historical trajectory point features and historical environmental features of those previous wheels. The motion trajectories of the current wheel and the previous wheels at least partially overlap, and the current wheel represents the wheel from which motion data was acquired after the previous wheel.

[0037] In this step, millimeter-wave radar can be used to measure distance and angle, providing accurate ranging and positioning information. Millimeter-wave radar has good penetration and anti-interference capabilities, allowing it to operate stably in complex environments. An IMU (Integrated Measurement Unit), composed of a gyroscope and accelerometer, provides the pedestrian's acceleration and angular velocity information. From this motion data, the pedestrian's speed, attitude (such as orientation and pitch angle), and position information can be calculated. The accuracy of the IMU sensor is affected by many factors, such as individual device differences, magnetic field interference, and accelerometer zero-point drift. The combined use of these two sensor technologies provides more accurate and comprehensive position and attitude information for indoor pedestrian SLAM, thereby better enabling localization and map building.

[0038] In some embodiments, please refer to Figure 3 This embodiment can further extend step S11 of the above embodiment. To obtain the current trajectory point features of the target object in the target region with respect to the current wheel, and / or the historical trajectory point features with respect to historical wheels, this embodiment may include the following steps:

[0039] S111: Use the current round or a previous round as the target round.

[0040] The process of collecting trajectory point features is the same for the current round or the historical rounds. The following explanation uses the target round as an example.

[0041] S112: Use an odometer sensor to collect odometer data of the target object's motion trajectory around the target wheel within the target area. The odometer data includes acceleration and heading angle.

[0042] Taking an IMU (Integrated Measurement Unit) as an example, an odometer can collect odometer data, i.e., acceleration and heading angle, of a target object moving along its wheel trajectory within a target area. Acceleration represents the acceleration of the target object's motion, and heading angle represents the angular velocity of the target object's motion.

[0043] S113: Dead reckoning is performed using acceleration and heading angle to obtain the target trajectory point features of the target wheel; wherein, the target trajectory point features include multiple sequentially arranged trajectory points and / or the step size between adjacent trajectory points.

[0044] Dead reckoning is performed using acceleration and heading angle. For example, the Pedestrian Dead Reckoning (PDR) algorithm can be used to measure and statistically analyze the number of steps, stride length, and direction of a pedestrian's movement, thereby calculating the pedestrian's trajectory and position, and obtaining the target trajectory point features of the target wheel. The PDR algorithm primarily uses an Inertial Measurement Unit (IMU) to sense data such as acceleration, angular velocity, magnetic force, and pressure during a person's movement in a beacon-free environment. This data is then used to calculate the person's stride length and direction, achieving the goal of locating and tracking the person. The main processes involved include gait detection, stride length calculation, and direction calculation.

[0045] Specifically, dead reckoning may include the following steps:

[0046] (1) Accelerometer data processing: Before using the acceleration data collected by the accelerometer, the gravitational acceleration component is removed. The acceleration of the pedestrian relative to the ground can be obtained by subtracting the collected acceleration vector from the gravitational acceleration vector.

[0047] (2) Gait detection: Gait detection is performed using acceleration to obtain the pedestrian's step frequency. In this step, the peak detection method can be used to detect the pedestrian's gait. The component perpendicular to the ground is extracted from the acceleration obtained in (1) above, which can represent the amplitude of the vertical acceleration, and thus a gait diagram can be drawn.

[0048] Please see Figure 4 Gait graphs are plotted using the amplitude-time component perpendicular to the ground. The gait graph will show many peaks and troughs. Each peak is the start time of each step taken by the pedestrian, and each trough is the end time of each step taken by the pedestrian. In other words, the total number of steps taken within the time period can be obtained.

[0049] After estimating the gait, the number of steps a pedestrian takes per unit time, i.e., the step frequency SF, can be obtained using the following formula, which is expressed as follows:

[0050]

[0051] Where Step_nums represents the total number of steps in the gait, and Times represents the duration.

[0052] (3) Obtaining stride length: The stride length of the target object can be obtained by using relevant information and stride frequency. Among them, relevant information includes height, etc.

[0053] The stride length SL of a pedestrian can be calculated based on the following formula, which is expressed as follows:

[0054]

[0055] Where a and b are coefficients, H is height, SF is stride frequency, and c is a constant, which can be set to 1. For example, the values ​​of each parameter are as follows: a = 0.371, b = 0.227, H = 1.74. Furthermore, other constants in the above formula, such as 0.7 / 1.75 / 1.79, can be determined according to the specific application scenario, and this application is not limited to this.

[0056] (4) Obtain the heading angle: The heading angle is calculated based on the magnetic induction intensity of the magnetometer and the acceleration of the accelerometer.

[0057] Specifically, the heading angle can be calculated based on the magnetic flux density of the magnetometer in the IMU. Since it is difficult to keep the phone perfectly horizontal while walking, an accelerometer can also be used for compensation calculations. For example, the accelerometer obtains the acceleration along three axes, the magnetometer obtains the magnetic flux density along three axes, the accelerometer can calculate the pitch and roll angles, and the heading angle can be calculated from the magnetic flux density of the magnetometer.

[0058] The heading angle yaw can be expressed as follows:

[0059]

[0060] in, Represents the magnetic flux density component along the y-axis. This represents the magnetic flux density component along the x-axis. Taking a three-axis system as an example, it can be specifically represented as follows:

[0061]

[0062] In the above formula, m x m y m z The magnetic flux density of the three axes of the magnetometer is represented by θ and γ, respectively, and the pitch and roll angles are represented by a. x a y a z These represent the accelerations along the three axes of the accelerometer, with g representing the acceleration due to gravity.

[0063] (5) Obtaining Trajectory Point Features: The trajectory point features can be calculated using the heading angle and step size mentioned above. Specifically, the pedestrian's position can be divided into two directions within the plane: the N-axis (e.g., north as the positive direction) and the E-axis (e.g., east as the positive direction). The pedestrian's position can be calculated using the following formula:

[0064]

[0065] Where k represents the k-th step of the pedestrian, Nk+1 E represents the trajectory point at the (k+1)th step on the N-axis. k+1 Let α represent the trajectory point at the (k+1)th step along the E-axis. k SL represents the heading angle at step k. k This represents the step size between the trajectory point at step k and the trajectory point at step (k+1).

[0066] By combining the trajectory points in the N-axis and E-axis directions and the step size between adjacent trajectory points, the trajectory point features of the target object can be obtained. The target trajectory point features include multiple sequentially arranged trajectory points and / or the step size between adjacent trajectory points.

[0067] Using the above method, we can obtain the current trajectory point features of the target object in the target area with respect to the current wheel, as well as the historical trajectory point features with respect to the historical wheels.

[0068] In some embodiments, please refer to Figure 5 This embodiment can further extend step S11 of the above embodiment. To obtain the current environmental features of the target object in the target region regarding the current wheel, and / or the historical environmental features regarding historical wheels, this embodiment may include the following steps:

[0069] S121: Use the current round or a previous round as the target round.

[0070] The process of collecting environmental features is the same for the current round or the previous rounds. The following explanation uses the target round as an example.

[0071] S122: Use millimeter-wave radar equipment to collect raw point cloud data of the target object's motion trajectory of the target wheel within the target area. The raw point cloud data includes multiple dimensions of data arranged according to preset dimensions.

[0072] The process of obtaining the original point cloud data of the wipe in this embodiment can refer to the original ADC data obtained by using the data acquisition platform to collect data from the target object as described above, and will not be repeated here.

[0073] In some application scenarios, the raw ACD data mentioned above can be in the form of a radar data cube (DataCube). It stores various cross-combinations of dimensions and metrics, and is a three-dimensional graphical description of the stored radar data after spatiotemporal processing. Multiple frames of raw point cloud data can be parsed to obtain multi-dimensional data and rearranged so that each frame of raw point cloud data includes multiple dimensions arranged according to preset dimensions.

[0074] Optionally, the multiple dimensions of data arranged in the preset dimensions can be: sampling point data, linear frequency modulation data (Chirp number) and antenna data, and the antenna data can include transmitting antenna and receiving antenna.

[0075] S123: Perform feature transformations on the data of each dimension in sequence to obtain the target environment features of the target wheel.

[0076] Feature transformations can be performed sequentially on data from each dimension. For example, raw point cloud data containing sampling point data, linear frequency modulation data (Chirp number), and antenna data can be represented as a three-dimensional matrix. Three feature transformations can be performed on this three-dimensional matrix to obtain the target environment features. Among these, the feature transformation can be a Fourier transform.

[0077] In some implementations, before each Fourier transform, the original point cloud data or the original point cloud data after Fourier transform can be windowed to obtain windowed original point cloud data, thereby preventing spectral leakage after the Fourier transform.

[0078] In some implementations, performing a cubic Fourier transform on the three-dimensional matrix of the original point cloud data may include the following steps:

[0079] The sampling point data is subjected to a first feature transformation to obtain distance information, which represents the distance of surrounding objects relative to the target object.

[0080] A second feature transformation is performed on the linear frequency modulation data to obtain velocity information, which represents the velocity of surrounding objects relative to the target object.

[0081] The antenna data is subjected to a third feature transformation to obtain angle information, which represents the angle of surrounding objects relative to the target object. The angle information can include elevation angle and azimuth angle.

[0082] After three Fourier transforms, a new three-dimensional matrix is ​​obtained, which is the target environment feature. The three dimensions of the target environment feature correspond to distance, velocity, and angle, respectively. In other words, the target environment feature can include distance information, velocity information, and angle information.

[0083] Using the above methods, we can obtain the current environmental characteristics of the target object in the target area regarding the current round, and the historical environmental characteristics regarding the previous round.

[0084] In some implementations, after acquiring the environmental characteristics, the following steps may also be performed:

[0085] S124: Compare the speed information of the target object with the speed information contained in the target environment features to obtain the comparison result; wherein, the speed information of the target object is obtained by using the mileage data of the target object about the target wheel collected by the mileage sensor device.

[0086] In indoor scenes, pedestrians' reference points are usually stationary objects, such as tables and chairs. In this case, other moving objects (such as other pedestrians) will affect the perception of the surrounding environment, so it is necessary to denoise the environmental features.

[0087] In this embodiment, the noise reduction method used is relative static clutter filtering. Since the IMU already provides the pedestrian's own speed information, that is, the speed information of the target object is obtained by using the mileage data of the target object's wheels collected by the mileage sensor device. The speed information of the target object can be compared with the speed information contained in the target environment features to obtain the comparison result, that is, the comparison result is the same or different.

[0088] S125: Based on the comparison results, filter out target environment features that differ from the speed information of the target object.

[0089] Considering the need to select surrounding stationary objects as references, based on the comparison results, target environmental features with speed information different from the target object are filtered out. In other words, environmental features with speeds different from pedestrians are filtered out, thus eliminating interference from non-stationary objects in the surrounding environment. Only data corresponding to objects that are relatively stationary to the pedestrian are retained.

[0090] The above scheme, in indoor SLAM, primarily uses stationary objects in the surrounding environment as beacons. By filtering out target environment features with velocity information different from the target object, interference from other moving objects can be eliminated, making the beacons more accurate. Furthermore, using IMU data and PDR algorithms to calculate the pedestrian's own velocity information, and millimeter-wave devices to sense the velocity of the surrounding environment relative to the pedestrian, filtering out environmental features with different speeds can remove interference from other moving objects. Since beacons in indoor SLAM scenarios are typically stationary, the aforementioned beacon feature enhancement method that filters out dynamic objects is more suitable for indoor SLAM scenarios.

[0091] In some implementations, the order of execution of steps S111 to S113 and steps S121 to S125 in the above embodiments is not limited.

[0092] Continue reading Figure 1 The above step S11 is followed by the following step S12:

[0093] S12: In response to the fact that the current environmental features and historical environmental features meet the same regional conditions, the current trajectory point features are used to optimize the historical trajectory point features to obtain updated trajectory point features.

[0094] After constructing a regional map of pedestrian trajectory points, the map can be optimized. However, due to unavoidable errors in sensor data (such as IMUs) during actual use, the accumulated error of the PDR algorithm for trajectory point estimation will increase over time. Therefore, this application employs a closed-loop detection method. By judging whether the current environmental features and historical environmental features meet the same regional condition, it determines whether the pedestrian has returned to a previously visited location, providing closed-loop constraints for regional map optimization.

[0095] Based on the current and historical environmental features obtained above, it can be determined whether the current and historical environmental features correspond to the same region, and whether the same region condition is met. If the same region condition is met, the current trajectory point features can be used to optimize the historical trajectory point features to obtain updated trajectory point features.

[0096] In some embodiments, please refer to Figure 6 The above embodiment's step S12 can be further extended. Before step S12 responds to the current environmental characteristics and historical environmental characteristics satisfying the same regional conditions, the following steps may also be included:

[0097] S211: Use the regional feature extraction model to extract the first regional feature corresponding to the current environmental feature and the second regional feature corresponding to the historical environmental feature.

[0098] Among them, the regional feature extraction model can be used to extract the features of the region of interest. The regional feature extraction model is used to extract the first regional feature corresponding to the current environmental features and the second regional feature corresponding to the historical environmental features.

[0099] In some implementations, the region feature extraction model is a self-attention model. For example, the self-attention model can be a Transformer architecture model, consisting of an encoder and a decoder, both of which are stacks of multi-head self-attention modules. The input sequence is divided into two parts: a source input sequence and a target output sequence. The source input is fed into the encoder, and the target output is fed into the decoder. The Transformer encoder consists of multiple identical layers stacked together, each layer having two sublayers: the first is a multi-head self-attention convergence layer; the second is a location-based feedforward neural network. Each sublayer employs residual connections. The Transformer decoder consists of multiple decoder modules stacked together, with a final linear layer. The stacking of decoder modules maps the context-dependent encoded sequence and the preceding input of each target vector to the encoded sequence of the target vector. It is understood that the above-mentioned regional feature extraction model can also adopt other network structures, such as Long Short-Term Memory (LSTM) networks or other deep learning networks. This application does not limit the specific structure of the regional feature extraction model.

[0100] In some implementations, the self-attention model can be pre-trained before use. This pre-training utilizes a set of environmental feature samples from the target region to obtain a pre-trained self-attention model. During pre-training, environmental feature samples from various regions of the target region are acquired. These samples include region feature samples from multiple regions within the target region. The regions referred to here can include regions of interest within the target region, such as indoor scenes. Since not all objects in an indoor environment have the same importance or reference value when a pedestrian walks, for example, corners or areas with tables and chairs have greater reference value than walls. Environmental feature samples from corners and areas with tables and chairs can be collected to pre-train the self-attention model. During pre-training, the model weight parameters are adjusted so that the pre-trained self-attention model can extract the region features corresponding to corners and areas with tables and chairs from the environmental features to a greater extent.

[0101] In this step, a pre-trained region feature extraction model is used to extract region features. This model can extract the region features of the region of interest from the environmental features. The region of interest can be obtained during the pre-training process, and this application does not impose any restrictions on this.

[0102] The above scheme employs a Transformer-based region feature extraction model for loop closure detection. First, this model is pre-trained to obtain a pre-trained region feature extraction model. Then, region features for loop closure detection are extracted based on this model, improving the extraction performance of region features for areas of interest. This, in turn, enhances the accuracy of subsequent loop closure detection under the same regional conditions.

[0103] S212: Obtain the regional similarity between the features of the first region and the features of the second region.

[0104] The similarity between the first and second region features can be obtained using Euclidean distance to describe the similarity between them; a smaller Euclidean distance indicates higher region similarity. It is understood that other feature similarity methods, such as Hamming distance, Chebyshev distance, and cosine similarity, can also be used to obtain region similarity, and this application is not limited to these.

[0105] In some implementations, when the historical environmental features include multiple frames of historical environmental features, the regional features corresponding to the historical environmental features of each frame can be obtained separately, and the historical environmental feature with the highest regional similarity can be used as the regional similarity between the first regional feature and the second regional feature. In some application scenarios, in order to reduce the amount of computation, the regional features of multiple frames of historical environmental features at a preset period can be obtained according to the time sequence of environmental feature acquisition, etc., to obtain the regional similarity in this implementation.

[0106] S213: Utilize regional similarity to determine whether current environmental characteristics and historical environmental characteristics meet the conditions of the same region.

[0107] The "same region" condition includes a regional similarity greater than a preset similarity threshold. If the regional similarity between the current environmental features and historical environmental features is greater than the preset similarity threshold, then the current environmental features and historical environmental features are determined to meet the same region condition; otherwise, they are determined not to meet the same region condition.

[0108] Optionally, after determining that the current environmental characteristics and historical environmental characteristics meet the conditions for the same region, the above can be performed. Figure 1 Step S12 in the process is to perform a step of optimizing the historical trajectory point features using the current trajectory point features in response to the current environmental features and historical environmental features satisfying the same regional conditions, thereby obtaining updated trajectory point features.

[0109] Optionally, after determining that the current environmental features and historical environmental features do not meet the same regional conditions, the current trajectory point features are added to the historical trajectory point features, and the current environmental features are included in the historical environmental features. New current trajectory point features and current environmental features are then obtained again to continue to re-execute the steps of this embodiment, such as steps S11 to S14.

[0110] The above scheme extracts regional features from environmental features through a pre-trained regional feature extraction module, which can extract regional features of the region of interest to a greater extent. By obtaining the regional similarity between the first and second regional features, the matching accuracy of environmental features can be improved, and the robustness of matching can be enhanced. In addition, even when environmental features outside the region of interest are slightly changed due to external interference, it can still successfully match and determine whether the same region condition is met for loop closure detection.

[0111] Furthermore, using 3D data from millimeter-wave radar equipment's DataCube for loop closure detection offers more standardized data formats and richer information, thus improving the accuracy of loop closure detection. Compared to pre-deploying beacons at specific locations to provide positional references during trajectory optimization, this application utilizes millimeter-wave radar equipment to collect environmental information surrounding pedestrians, eliminating the need for pre-deployment and offering greater versatility. The rich DataCube data from the millimeter-wave radar is more suitable for perceiving the pedestrian's surroundings and allows for the application of more denoising algorithms in different scenarios. Using it for loop closure detection can detect more beacon features, thereby improving the accuracy of similarity matching of regional features.

[0112] Furthermore, by using closed-loop detection to determine whether a pedestrian has walked to a place they have visited before, i.e., the same area, closed-loop constraints can be provided for subsequent optimization of the area map corresponding to the trajectory points.

[0113] In some embodiments, please refer to Figure 7 Step S12 of the above embodiment can be further extended. Optimizing the historical trajectory point features using the current trajectory point features to obtain updated trajectory point features can include the following steps:

[0114] S221: Determine the trajectory optimization model using the features of the current trajectory points and the features of the historical trajectory points; wherein, the trajectory optimization model includes constraints on the trajectory points and constraints on the step size.

[0115] First, identify the first and second trajectory points that meet the conditions of the same region. The first trajectory point belongs to the current trajectory point, and the second trajectory point belongs to the historical trajectory point.

[0116] Among them, the historical trajectory point features include multiple historical trajectory points x0 to x1 arranged in sequence. k-1 The step size between adjacent historical trajectory points (l0~l) k-1 The current trajectory point features include the current trajectory point x. k Multiple historical trajectory points are trajectory points preceding the current trajectory point. The characteristics of all trajectory points of the target object walking indoors are x0 to x... k x k x0 represents the trajectory point that satisfies the condition of the same location, i.e. the same region, as detected above, and is respectively used as the first trajectory point and the second trajectory point.

[0117] In this step, the first trajectory point and the second trajectory point can be obtained by the closed-loop detection described above. When it is determined that the current environmental features and the historical environmental features meet the same regional conditions, since the environmental features and trajectory point features are collected simultaneously, the first trajectory point can be determined based on the collection time of the current environmental features that meet the same regional conditions; and the second trajectory point can be determined based on the collection time of the historical environmental features.

[0118] Then, the first trajectory point is set to be equivalent to the second trajectory point, thus obtaining the constraints on the trajectory points. Using multiple sequentially arranged third trajectory points and the step sizes between adjacent third trajectory points, the constraints on the step sizes are obtained; where the third trajectory points include: multiple sequentially arranged historical trajectory points and the current trajectory point, and the variables x0 to x... k These are the variables that the model aims to optimize.

[0119] In summary, based on the constraints of the trajectory points and the step size, a trajectory optimization model is obtained. This trajectory optimization model can be expressed as follows:

[0120]

[0121] The trajectory optimization model described above is then simplified to obtain the following simplified trajectory optimization model:

[0122]

[0123] S222: Solve the trajectory optimization model to obtain updated trajectory point features.

[0124] From the above formula (7), it can be seen that x k Since the number of variables is less than the number of equations, the least squares method can be used to optimize the equations of the above trajectory optimization model.

[0125] In solving the trajectory optimization model, we can first calculate the residual sum of squares function corresponding to each equation, as follows:

[0126]

[0127] To obtain the optimal result of the trajectory optimization model, we can take the partial derivative of each variable of the residual sum of squares function above and set the partial derivatives to 0, as follows:

[0128]

[0129] Rearranging the above equations, we can obtain information about vectors. A system of linear equations, with coefficients of a k×k dimensional matrix A. k×k As shown below:

[0130]

[0131] The solution to the above model is x0~x k The optimal solution, which is to update the trajectory point features, can be used to replace the previous historical trajectory point features and the current trajectory point features.

[0132] The above process is an optimization process for trajectory points. During implementation, it can be repeated multiple times for a single route or trajectory, that is, optimized multiple times, thereby obtaining more accurate trajectory points and more precise positioning results.

[0133] Continue reading Figure 1 The above step S12 is followed by the following step S13:

[0134] S13: Construct a regional map of the target area based on the updated trajectory point features.

[0135] The updated trajectory point features described above can be used to replace previous historical and current trajectory point features, and a map can be constructed based on these updated features to obtain a regional map of the target area. This method allows for optimization of the regional map during the map construction process, enabling continuous updates to the regional map during iterations to improve its accuracy and achieve more precise positioning.

[0136] The above scheme acquires the current trajectory point features and current environmental features of the target object in the target area with respect to the current wheel, as well as the historical trajectory point features and historical environmental features of the historical wheels; wherein the motion acquisition trajectories of the current wheel and the historical wheels at least partially overlap. In response to the condition that the current environmental features and historical environmental features satisfy the same region condition, the historical trajectory point features are optimized using the current trajectory point features to obtain updated trajectory point features. This provides a method for loop closure detection using environmental features, and when the same region condition is met, it can determine whether the target object has returned to the previous area, providing a loop closure detection constraint for trajectory point optimization. Based on this, further optimization of the historical trajectory point features using the current trajectory point features can improve the accuracy of the updated trajectory point features. Therefore, based on the updated trajectory point features, a regional map of the target area can be constructed, improving the accuracy of the regional map construction.

[0137] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as mileage sensors or millimeter-wave radar equipment, a clear and prominent sign is set up to inform the user that they have entered the scope of personal information collection and that personal information will be collected. If the user voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, the personal information processing rules are clearly informed through signs / information, and authorization is obtained through pop-up information or by asking the user to upload their personal information. The personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0138] The area map construction method of this application, as described in the above embodiments, can be used to implement indoor pedestrian SLAM technology. Furthermore, it can be applied to many other scenarios and fields. This application is not limited thereto. Examples include the following scenarios and fields:

[0139] In the field of intelligent navigation: In complex environments such as large shopping malls, airports, and underground passages, pedestrians and robots require precise navigation and positioning technologies to avoid collisions and getting lost. The aforementioned method of constructing regional maps can provide high-precision navigation and positioning support for these scenarios, helping pedestrians and robots achieve autonomous navigation and path planning.

[0140] In the field of indoor positioning: In indoor environments, satellite signals such as GPS (Global Positioning System) cannot penetrate buildings, so other types of sensors are needed to achieve positioning. The above-mentioned method of constructing regional maps can use millimeter-wave radar equipment and odometers to obtain pedestrian location information, which can help people in the interior of buildings to locate and navigate.

[0141] In the field of intelligent driving, such as in autonomous vehicles and drones, the above-mentioned regional map construction method can provide more accurate position and attitude information for intelligent driving, helping unmanned systems to achieve autonomous driving and obstacle avoidance.

[0142] In the field of robotics, such as cleaning robots and nursing robots, the above-mentioned method of constructing regional maps can use millimeter-wave radar equipment and odometers to obtain information about the surrounding environment, helping robots to achieve autonomous navigation, obstacle avoidance and interaction, thereby improving their intelligence level and quality of life.

[0143] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0144] In addition to the above embodiments, this application also provides a regional map construction apparatus for implementing the above-described regional map construction method.

[0145] Please see Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the area map construction apparatus of this application. The area map construction apparatus 30 includes an acquisition module 31, an optimization module 32, and a construction module 33.

[0146] The acquisition module 31 is used to acquire the current trajectory point features and current environmental features of the target object in the target area with respect to the current wheel, as well as the historical trajectory point features and historical environmental features with respect to the historical wheels; wherein the motion acquisition trajectories of the current wheel and the historical wheels have at least partial overlap;

[0147] Optimization module 32 is used to optimize the historical trajectory point features by using the current trajectory point features to obtain updated trajectory point features when the current environmental features and historical environmental features meet the same regional conditions.

[0148] Module 33 is used to construct a regional map of the target area based on the updated trajectory point features.

[0149] The specific implementation of this embodiment can be referred to the implementation process of the above embodiments, and will not be repeated here.

[0150] Regarding the above embodiments, this application provides an electronic device; please refer to [link / reference]. Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 40 includes a memory 41 and a processor 42, wherein the memory 41 and the processor 42 are coupled to each other. The memory 41 stores program data, and the processor 42 is used to execute the program data to implement the steps of any embodiment of the above-described method for constructing a regional map.

[0151] In this embodiment, processor 42 can also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 42 can be any conventional processor.

[0152] The methods described in the above embodiments can be implemented as computer programs; therefore, this application proposes a computer-readable storage medium. Please refer to [link to relevant documentation]. Figure 10 , Figure 10 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 50 stores program data 51 that can be executed by a processor. The program data 51 can be executed by the processor to implement the steps of any embodiment of the above-described method for constructing a regional map.

[0153] In this embodiment, the computer-readable storage medium 50 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store program data 51. Alternatively, it can be a server that stores the program data 51. The server can send the stored program data 51 to other devices for execution, or it can run the stored program data 51 itself.

[0154] In some embodiments, the functions or modules of the apparatus provided in the above embodiments of this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0155] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0157] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0159] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0160] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, and thus stored in a computer-readable storage medium for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this application is not limited to any particular hardware and software combination.

[0161] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for constructing a regional map, characterized in that, include: The current trajectory point features and current environmental features of the target object in the target area with respect to the current wheel, as well as the historical trajectory point features and historical environmental features with respect to the historical wheels, are obtained; wherein the motion acquisition trajectories of the current wheel and the historical wheels at least partially overlap. In response to the current environmental features and the historical environmental features satisfying the condition of the same region, the historical trajectory point features are optimized using the current trajectory point features to obtain updated trajectory point features; Based on the updated trajectory point features, a regional map of the target area is constructed.

2. The method according to claim 1, characterized in that, The response before the current environmental characteristics and the historical environmental characteristics satisfy the same regional conditions includes: The first regional feature corresponding to the current environmental feature and the second regional feature corresponding to the historical environmental feature are extracted using a regional feature extraction model. Obtain the regional similarity between the features of the first region and the features of the second region; Using the regional similarity, it is determined whether the current environmental features and the historical environmental features satisfy the same regional condition.

3. The method according to claim 2, characterized in that, The region feature extraction model is a self-attention model, which is pre-trained using an environmental feature sample set of the target region. The environmental feature sample set includes region feature samples of multiple regions in the target region. The same region condition includes: the region similarity is greater than a preset similarity threshold.

4. The method according to claim 1, characterized in that, The acquisition of the target object's current trajectory point features and current environment features in the target region with respect to the current wheel, as well as its historical trajectory point features and historical environment features with respect to historical wheels, includes: Use the current wheel or the historical wheel as the target wheel; The original point cloud data of the target object's motion trajectory about the target wheel within the target area is collected using millimeter-wave radar equipment. The original point cloud data includes multiple dimensional data arranged according to a preset dimension. The target environment features of the target wheel are obtained by sequentially performing feature transformations on each of the aforementioned dimensional data. The mileage data of the target object's motion trajectory about the target wheel within the target area is collected using an odometer device, wherein the mileage data includes acceleration and heading angle; Dead reckoning is performed using the acceleration and heading angle to obtain the target trajectory point features of the target wheel; wherein, the target trajectory point features include multiple sequentially arranged trajectory points and the step size between adjacent trajectory points.

5. The method according to claim 4, characterized in that, The multi-dimensional data includes: sampling point data, linear frequency modulation data, and antenna data; the target environment features include: distance information, velocity information, and angle information. The step of sequentially performing feature transformations on each of the aforementioned dimensional data to obtain the target environment features of the target wheel includes: The sampling point data is subjected to a first feature transformation to obtain the distance information, which represents the distance of surrounding objects relative to the target object; A second feature transformation is performed on the linear frequency modulation data to obtain the velocity information, which represents the velocity of surrounding objects relative to the target object. The antenna data is subjected to a third feature transformation to obtain the angle information, which represents the angle of the surrounding objects relative to the target object.

6. The method according to claim 4, characterized in that, Before sequentially performing feature transformations on each of the aforementioned dimensional data to obtain the target environment features of the target wheel, the method further includes: The original point cloud data is windowed to obtain windowed original point cloud data.

7. The method according to claim 4, characterized in that, After sequentially performing feature transformations on each of the aforementioned dimensional data to obtain the target environment features of the target wheel, the process further includes: The speed information of the target object is compared with the speed information contained in the target environment features to obtain a comparison result; wherein, the speed information of the target object is obtained by using the mileage data of the target object about the target wheel collected by the mileage sensor device; Based on the comparison results, target environment features that differ from the speed information of the target object are filtered out.

8. The method according to claim 1, characterized in that, The step of optimizing the historical trajectory point features using the current trajectory point features to obtain updated trajectory point features includes: A trajectory optimization model is determined using the features of the current trajectory point and the features of the historical trajectory point; wherein, the trajectory optimization model includes constraints on the trajectory points and constraints on the step size; The trajectory optimization model is solved to obtain the updated trajectory point features.

9. The method according to claim 8, characterized in that, The historical trajectory point features include multiple sequentially arranged historical trajectory points and the step size between adjacent historical trajectory points; the current trajectory point features include the current trajectory point; the multiple historical trajectory points are trajectory points preceding the current trajectory point; The process of determining the trajectory optimization model using the features of the current trajectory points and the features of the historical trajectory points includes: A first trajectory point and a second trajectory point that satisfy the same region condition are determined, wherein the first trajectory point belongs to the current trajectory point and the second trajectory point belongs to the historical trajectory point; By setting the first trajectory point to be equivalent to the second trajectory point, the constraint conditions of the trajectory point are obtained; and, The constraint condition of the step length is obtained by using multiple sequentially arranged third trajectory points and the step length between adjacent third trajectory points; wherein, the third trajectory points include: multiple sequentially arranged historical trajectory points and the current trajectory point; Based on the constraints of the trajectory points and the constraints of the step size, the trajectory optimization model is obtained.

10. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program data and the processor executing the program data to implement the steps of the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The system stores program data that can be executed by a processor, the program data being used to implement the steps of the method according to any one of claims 1 to 9.

Images