A data fusion map construction method, system and electronic device

Abstract

The application provides a data fusion map construction method and system and an electronic device, and relates to the technical field of artificial intelligence. In the application, first, point cloud data, image data and odometer data collected in the driving process of a target object on a preset driving path are obtained, then a first fusion pose is obtained based on the point cloud data, the odometer data and a first pose constraint condition set for the corresponding point cloud data, a second fusion pose is obtained based on the image data, the odometer data and a second pose constraint condition set for the corresponding image data, then the first fusion pose and the second fusion pose are fused to obtain a target fusion pose, and finally, target map data is generated based on the target fusion pose, the point cloud data and the image data. In this way, the target object can obtain an accurate target fusion pose, the prior map of the scene can be constructed according to the target fusion pose, and the accuracy of constructing the prior map is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a data fusion map construction method and system and electronic device. BACKGROUND

[0002] With the development of artificial intelligence technology and the improvement of industrial manufacturing level, more and more high-precision and intelligent devices are manufactured to assist or replace manual operation to complete daily tasks and provide convenience for life. For example, some robots applied to handle cargo handling tasks or some robots applied to handle household cleaning tasks.

[0003] Before the robot is put into actual task operation, in order to determine the position of itself in the current scene, it is usually necessary to construct a prior map of the scene. When a movable robot or a vehicle is used to construct a prior map of the scene, the position of the robot can be obtained in real time by matching the information collected by the sensor carried by the movable robot with the pre-calibrated map, thereby providing positioning support for subsequent task operation.

[0004] In the prior art, a prior map of the scene is usually constructed according to laser data collected by a radar sensor. However, the accuracy of the radar sensor is certain, and the laser data is constantly changing and is easily disturbed by moving objects in the scene. Therefore, the laser data collected may have a positional deviation compared with ideal laser data, and the laser data with the positional deviation is used to estimate the pose of the robot. Then, the target pose of the robot obtained by the pose constraint established according to the estimated pose and the joint back-end optimization will be inaccurate, resulting in an inaccurate prior map constructed according to the target pose. SUMMARY

[0005] The present application provides a data fusion map construction method and system and electronic device to improve the accuracy of constructing a prior map. The specific technical solutions are as follows:

[0006] In a first aspect, the present application provides a data fusion map construction method, comprising:

[0007] Obtaining point cloud data, image data and odometry data collected in a driving process of a target object on a preset driving path;

[0008] Obtaining a first fusion pose based on the point cloud data, the odometry data and a first pose constraint condition set corresponding to the point cloud data;

[0009] Obtaining a second fusion pose based on the image data, the odometry data and a second pose constraint condition set corresponding to the image data;

[0010] fuse the first fused pose and the second fused pose to obtain a target fused pose;

[0011] generate target map data based on the target fused pose, the point cloud data, and the image data.

[0012] Based on the above method, the target object can obtain an accurate target fused pose, and the prior map of the scene can be constructed according to the target fused pose, thereby improving the accuracy of constructing the prior map.

[0013] In a possible implementation, the first fused pose is obtained based on the point cloud data, the odometer data, and a first pose constraint condition set according to the point cloud data.

[0014] A first laser timestamp is determined from the point cloud data, and a first odometer timestamp adjacent to the first laser timestamp is found, wherein the laser timestamp and the odometer timestamp are located on the same time axis.

[0015] The first pose of the target object in the odometer coordinate system is calculated based on the first odometer timestamp and the odometer data corresponding to the first odometer timestamp.

[0016] The fourth pose of the target object in the map coordinate system corresponding to the first laser timestamp is determined based on the first pose of the target object, a second pose corresponding to a second laser timestamp in the odometer coordinate system, and a third pose corresponding to the second laser timestamp in the map coordinate system, wherein the second laser timestamp is the last time of the first laser timestamp.

[0017] Based on the above method, the pose of the target object in the map coordinate system corresponding to each laser timestamp can be obtained.

[0018] In a possible implementation, the first laser timestamp is determined from the point cloud data, including:

[0019] The laser data frame corresponding to the first laser timestamp is extracted.

[0020] The first point cloud data satisfying the data feature condition is filtered out from the point cloud data, and each laser data key frame having a time interval from the laser data frame not exceeding a set threshold is found from the first point cloud data, and a laser data key frame set is formed.

[0021] Any target laser data key frame in the laser data key frame set is selected, and a first laser data key frame and a second laser data key frame adjacent to the target laser data key frame are determined.

[0022] In a case where it is determined that the laser data frame is not included in a first data chain composed of the first laser data key frame and the second laser data key frame, the first laser data key frame and the second laser data key frame are removed from the set of laser data key frames, and third laser data key frames and fourth laser data key frames adjacent to the target laser data key frame are determined;

[0023] In a case where it is determined that the laser data frame is not included in a second data chain composed of the first data chain, the third laser data key frame and the fourth laser data key frame, a local submap is established based on laser data key frames included in the second data chain;

[0024] The laser data frame is matched with the local submap to obtain a matching pose of the target object;

[0025] In a case where it is determined that a similarity score of the matching pose is greater than a set threshold, a fifth laser data key frame with a smallest time interval from the laser data frame is selected in the second data chain;

[0026] A first pose constraint condition between the laser data frame and the fifth laser data key frame is established to obtain a first fusion pose.

[0027] Based on the above method, the pose of the target object corresponding to each laser timestamp in the map coordinate system can be preliminarily optimized to obtain a first fusion pose.

[0028] In a possible implementation, the second fusion pose is obtained based on the image data, the odometer data and a second pose constraint condition set according to the image data, and the second fusion pose comprises:

[0029] If the preset first position identifier is not scanned by the loop scanning, a first image timestamp and a second image timestamp are determined from the image data, wherein the first image timestamp is a time point before the second image timestamp, and the first image timestamp is a time point at which the first position identifier is scanned for the first time;

[0030] A third laser timestamp adjacent to the first image timestamp and a fourth laser timestamp adjacent to the second image timestamp are found, wherein the image timestamp and the laser timestamp are located on the same time axis;

[0031] A second odometer timestamp adjacent to the third laser timestamp and a third odometer timestamp adjacent to the fourth laser timestamp are found;

[0032] determine a fifth pose of the target object in the odometry coordinate system corresponding to the first image timestamp and a seventh pose of the target object in the odometry coordinate system corresponding to the first image timestamp based on the second odometry timestamp and second odometry data corresponding to the second odometry timestamp;

[0033] determine a sixth pose of the target object in the odometry coordinate system corresponding to the second image timestamp and an eighth pose of the target object in the odometry coordinate system corresponding to the second image timestamp based on the third odometry timestamp and third odometry data corresponding to the third odometry timestamp;

[0034] construct a second pose constraint condition between the first image data frame corresponding to the first image timestamp and the second image data frame corresponding to the second image timestamp based on a pose change amount between the seventh pose and the eighth pose.

[0035] Based on the above method, the second pose constraint condition of the target object at the adjacent camera time when reaching the position identifier 2 can be constructed, and the second pose constraint condition is added to the back-end optimizer to optimize the pose of the target object in the camera time in the map coordinate system.

[0036] In a possible implementation, if the first position identifier is scanned by the loop scanning, the third image timestamp is determined as the time when the first position identifier is scanned for the second time;

[0037] find a fifth laser timestamp adjacent to the third image timestamp and a fourth odometry timestamp adjacent to the fifth laser timestamp;

[0038] determine a ninth pose of the target object in the odometry coordinate system corresponding to the third image timestamp and a tenth pose of the target object in the map coordinate system corresponding to the third image timestamp based on the fourth odometry timestamp and fourth odometry data corresponding to the fourth odometry timestamp;

[0039] construct a third pose constraint condition between the first image data frame and a third image data frame corresponding to the third image timestamp based on a pose change amount between the tenth pose and the seventh pose;

[0040] obtain a second fusion pose based on the second pose constraint condition and the third pose constraint condition.

[0041] Based on the above method, the third pose constraint condition (loop constraint condition) constructed can be added to the back-end optimizer to obtain the second fusion pose of the target object in the camera time in the map coordinate system.

[0042] The first fusion pose and the second fusion pose are finally fused according to the respective assigned weight values, a target fusion pose is obtained, the pose estimation of the target object in the map coordinate system is more accurate, and finally the target map data is generated by combining the target fusion pose, the point cloud data and the image data, thereby improving the accuracy of constructing the prior map.

[0043] In a second aspect, the present application provides a data fusion map construction system, comprising:

[0044] A data acquisition module is configured to acquire point cloud data, image data and odometer data collected during the driving process of a target object on a preset driving path.

[0045] A data fusion module is configured to obtain a first fusion pose based on the point cloud data, the odometer data and a first pose constraint condition set for the point cloud data.

[0046] Obtain a second fusion pose based on the image data, the odometer data and a second pose constraint condition set for the image data.

[0047] Fuse the first fusion pose and the second fusion pose to obtain a target fusion pose.

[0048] A map construction module is configured to generate target map data based on the target fusion pose, the point cloud data and the image data.

[0049] In a possible implementation, the data fusion module is specifically configured to:

[0050] Determine a first laser timestamp from the point cloud data, and find a first odometer timestamp adjacent to the first laser timestamp, wherein the laser timestamp and the odometer timestamp are located on the same time axis.

[0051] Calculate a first pose of the target object in the odometer coordinate system based on the first odometer timestamp and the odometer data corresponding to the first odometer timestamp.

[0052] Determine a fourth pose of the target object in the map coordinate system corresponding to the first laser timestamp based on the first pose of the target object, a second pose in the odometer coordinate system corresponding to a second laser timestamp, and a third pose in the map coordinate system corresponding to the second laser timestamp, wherein the second laser timestamp is the previous time of the first laser timestamp.

[0053] In a possible implementation, the data fusion module is further configured to:

[0054] Extract a laser data frame corresponding to the first laser timestamp.

[0055] From the point cloud data, first point cloud data satisfying a data feature condition is screened out, and from the first point cloud data, each laser data key frame with a time interval from the laser data frame not exceeding a set threshold is found out and a laser data key frame set is formed;

[0056] Any target laser data key frame in the laser data key frame set is selected, and a first laser data key frame and a second laser data key frame adjacent to the target laser data key frame are determined;

[0057] When it is determined that the laser data frame is not contained in a first data chain formed by the first laser data key frame and the second laser data key frame, the first laser data key frame and the second laser data key frame are eliminated from the laser data key frame set, and a third laser data key frame and a fourth laser data key frame adjacent to the target laser data key frame are determined;

[0058] When it is determined that the laser data frame is not contained in a second data chain formed by the first data chain, the third laser data key frame and the fourth laser data key frame, a local submap is established based on laser data key frames contained in the second data chain;

[0059] The laser data frame is matched with the local submap to obtain a matching pose of the target object;

[0060] When it is determined that a similarity score of the matching pose is greater than a set threshold, a fifth laser data key frame with a minimum time interval from the laser data frame is selected in the second data chain;

[0061] A first pose constraint condition between the laser data frame and the fifth laser data key frame is established to obtain a first fusion pose.

[0062] In a third aspect, the present application provides an electronic device, comprising:

[0063] a memory for storing a computer program;

[0064] a processor for executing the computer program stored on the memory to realize the steps of the above-mentioned data fusion map construction method.

[0065] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the above-mentioned data fusion map construction method.

[0066] The technical effects that can be achieved by each of the above-mentioned second to fourth aspects and each aspect are described above in relation to the first aspect or the various possible solutions in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 A flowchart of a data fusion map construction method provided in the present application is shown in the following figure:

[0068] Figure 2 A data fusion map construction system architecture schematic diagram provided in the present application is shown in the following figure:

[0069] Figure 3 A schematic diagram of an odometer timestamp and a radar timestamp provided in the present application is shown in the following figure:

[0070] Figure 4 A data fusion map construction system architecture schematic diagram provided in the present application is shown in the following figure:

[0071] Figure 5 A structure schematic diagram of an electronic device provided in the present application is shown in the following figure. DETAILED DESCRIPTION

[0072] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be described in further detail below with reference to the accompanying drawings. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments. It should be noted that in the description of the present application, “multiple” is understood as “at least two”. The association relationship of “and / or” between the associated objects indicates that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. A is connected with B, which can represent two cases: A is directly connected with B and A is connected with B through C. In addition, in the description of the present application, “first”, “second”, etc. are used only for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order.

[0073] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0074] With the development of artificial intelligence technology and the improvement of industrial manufacturing level, more and more high-precision and intelligent devices are manufactured to assist or replace manual operation to complete daily tasks and provide convenience for life. For example, some robots applied to handle cargo handling tasks, or some robots applied to handle household cleaning tasks.

[0075] Before a robot is put into actual task operation, in order to determine the position of itself in the current scene, it is usually necessary to construct a prior map of the scene. When a movable robot or a trolley is used to construct a prior map of the scene, the position of the robot can be obtained in real time according to the information collected by the sensor device carried by the movable robot and the pre-calibrated map, thereby providing positioning support for subsequent task operation.

[0076] In the prior art, a prior map of the scene is usually constructed according to laser data collected by a radar sensor. However, the accuracy of the radar sensor is certain, and the laser data changes all the time and is easily disturbed by moving objects in the scene. Therefore, the laser data with position deviation may be collected compared with ideal laser data. The robot pose is estimated according to the laser data with position deviation, and the target pose of the robot obtained according to the estimated pose constraint and joint back-end optimization is inaccurate, resulting in an inaccurate prior map constructed according to the target pose.

[0077] In view of this, in order to improve the accuracy of constructing a prior map, the application provides a data fusion map construction method, which specifically comprises: first acquiring point cloud data, image data and odometry data collected by a target object in a preset driving path during driving, then obtaining a first fusion pose based on the point cloud data, the odometry data and a first pose constraint condition set for the corresponding point cloud data, and obtaining a second fusion pose based on the image data, the odometry data and a second pose constraint condition set for the corresponding image data; fusing the first fusion pose and the second fusion pose to obtain a target fusion pose, and finally generating target map data based on the target fusion pose, the point cloud data and the image data.

[0078] As can be seen, by the above-mentioned manner, the point cloud data, the image data and the odometry data collected by the target object during driving can be acquired first, then the first pose constraint is constructed between the point cloud data to obtain the first fusion pose, and the second pose constraint is constructed between the image data to obtain the second fusion pose, then the first fusion pose and the second fusion pose are fused to obtain the target fusion pose, and finally the target map data is generated based on the fusion pose, the point cloud data and the image data. In this way, the accurate target fusion pose of the target object can be obtained according to the constructed first pose constraint, the second pose constraint and the joint back-end optimization, and the prior map of the scene can be constructed according to the target fusion pose, thereby improving the accuracy of constructing the prior map.

[0079] Referring to Figure 1 As shown in the figure, the method comprises the following steps:

[0080] S1, acquiring point cloud data, image data and odometer data collected by the target object in a preset driving path during driving.

[0081] Firstly, the method provided by the application can be applied to Figure 2 As shown in the system architecture, the system architecture comprises a target terminal, an image detection device, a radar detection device and a controller.

[0082] The number of the image detection device and the radar detection device is not limited in the embodiments of the application, for example Figure 2 As shown, only the target terminal, the image detection device, the radar detection device and the controller are taken as examples for description, and the above-mentioned devices and their respective functions are briefly introduced as follows.

[0083] The target terminal is configured to send a map construction request to the controller; the controller is configured to receive the map construction request sent by the target terminal, and acquire prior information of a scene from the map construction request, for example, distance information, angle information between position identifier 1 and position identifier 2, and a connection relationship between each position identifier in a map coordinate system; the image detection device is configured to acquire image data collected by the target object during driving, identify a preset position identifier in the map coordinate system, and obtain the prior information associated with the position identifier after identifying the position identifier; the radar detection device is configured to acquire point cloud data (laser radar data) collected by the target object during driving, to provide data support for subsequent construction of a prior map of the scene; the controller is configured to calculate a pose of the target object in combination with odometer data after receiving the collected point cloud data, then fuse the point cloud data and the image data to determine a target fusion pose of the target object, and finally construct a prior map of a current environment according to the target fusion pose.

[0084] In the embodiments of the application, in order to construct a prior map of a scene, a target object needs to be determined first, for example, the target object can be a mobile robot or a car, and various sensors are arranged on the mobile robot to collect various data of the robot during driving, wherein the mobile robot further comprises a mobile chassis, a motion controller, a driving motor, a storage battery, an embedded computer, an odometer and various modules of a power system, and the controller can control the various modules of the power system to drive the mobile robot to drive in a set driving path, and the functions of the various modules of the power system are not described herein.

[0085] For example Figure 2As shown, after determining the target object, the controller first controls the image detection device to scan the preset position mark 1, position mark 2, and position mark 3 in the map coordinate system. Here, it is assumed that the starting position of the target object is the position where the position mark 1 is located, and the center zero position of the position mark 1 is taken as the origin of the map coordinate system. According to the image detection device, the radar detection device, and the odometer carried by the mobile robot, the point cloud data, the image data, and the odometer data collected in real time by the target object in the preset driving path can be obtained.

[0086] Through the above method, the point cloud data, the image data, and the odometer data collected by the target object in the driving process can be obtained.

[0087] S2, based on the point cloud data, the odometer data, and the first pose constraint condition set for the corresponding point cloud data, a first fused pose is obtained.

[0088] In the embodiments of the present application, when the point cloud data, the image data, and the odometer data of the target object are obtained, the controller first unifies the point cloud data, the image data, and the odometer data to the same time axis, and then performs real-time data analysis and processing on the point cloud, the image, and the odometer data. For example, Figure 3 As shown, a schematic diagram of the odometer timestamp and the radar timestamp provided by the present application is shown. It should be pointed out that each odometer timestamp has corresponding odometer data, and similarly, each radar timestamp has corresponding point cloud data. In Figure 3 In the above, to1, to2, …tom are each odometer timestamp; Tl1, Tl2, …Tln are each radar timestamp, and the image timestamp is not labeled in Figure 3 In the above, to1, to2, …tom are each odometer timestamp; Tl1, Tl2, …Tln are each radar timestamp, and the image timestamp is not labeled in

[0089] In a possible implementation, when step 2 is performed, a first laser timestamp is determined from the point cloud data, and a first odometer timestamp adjacent to the first laser timestamp is found. Based on the first odometer timestamp and the odometer data corresponding to the first odometer timestamp, a first pose of the target object in the odometer coordinate system is calculated. Based on the first pose of the target object, a second pose corresponding to a second laser timestamp in the odometer coordinate system, and a third pose corresponding to the second laser timestamp in the map coordinate system, a fourth pose of the target object corresponding to the first laser timestamp in the map coordinate system is determined.

[0090] In this embodiment, after the controller unifies the point cloud data, image data, and odometer data onto the same timeline, it determines the first laser timestamp as the moment when the target object arrives at the location of location marker 2 and is detected by the radar. Figure 3 As shown, assuming the first laser timestamp is Tl2, the first odometer timestamp adjacent to the first laser timestamp is found. When there is only one first odometer timestamp to3 adjacent to the first laser timestamp on the time axis, that is, when the odometer has not yet recorded the odometer data at time to4, for example, the first pose T of the target object in the odometer coordinate system is calculated based on the first odometer timestamp and the odometer data corresponding to the first odometer timestamp. Tl2 The calculation method for the first pose is as follows: Formula 1:

[0091] T Tl2 =T to3 +[vw] T ×(Tl2-to3)

[0092] Among them, T to3 Let v be the pose of the target object in the odometer coordinate system at time to3. The pose of the target object in the odometer coordinate system can be obtained by the odometer. v and w are the linear velocity and angular velocity of the odometer at time to3, respectively. When using Formula 1, it is assumed that the target object moves at a constant speed within the time range of to3-Tl2.

[0093] For example, when there are to3 and to4 on the time axis that are adjacent to the first odometer timestamp of the first laser, the first pose T of the target object in the odometer coordinate system is calculated based on the first odometer timestamp and the odometer data corresponding to the first odometer timestamp. Tl2 The first pose is calculated using the following formula 2:

[0094] T Tl2 = (to4-Tl2)×T t03 +(Tl2-to3)×T t04

[0095] Among them, T t04 This represents the pose of the target object in the odometry coordinate system at time to4.

[0096] Immediately afterwards, the controller obtains the first pose of the target object in the odometer coordinate system. Tl2Then, according to the first pose of the target object, the second pose corresponding to the second laser timestamp in the odometry coordinate system, the third pose corresponding to the second laser timestamp in the map coordinate system, a fourth pose of the target object corresponding to the first laser timestamp in the map coordinate system is determined mapTl2 , the calculation method of the fourth pose refers to the following formula 3:

[0097] T mapTl2 = T Tl1 × (T mapTl1 ) -1 × T Tl2

[0098] Wherein, T Tl1 is the second pose of the target object corresponding to the second laser timestamp in the odometry coordinate system, and the second laser timestamp is the last time of the first laser timestamp, that is, the time corresponding to Tl1; T mapTl1 is the third pose of the target object corresponding to the second laser timestamp in the map coordinate system.

[0099] Through the above method, the pose of the target object corresponding to each laser timestamp in the map coordinate system can be obtained.

[0100] After the controller obtains the pose of each laser timestamp, in order to ensure the real-time and effectiveness of processing the data corresponding to each laser timestamp, the laser data frame corresponding to each laser timestamp also needs to be screened.

[0101] For example, the controller first extracts the laser data frame corresponding to the first laser timestamp, and then filters the first point cloud data that meets the data feature condition from the point cloud data. The filtering method is as follows: when the time interval between the first laser timestamp and the previously saved laser timestamp is greater than a set threshold, the laser data frame corresponding to the laser timestamp that meets the time interval threshold condition is selected as the target laser data frame, and the target laser data frame is processed subsequently. For example, the first laser timestamp is 2023.1.1 (10h.12min.1s.59ms), the previously saved laser timestamp is 2023.1.1 (10h.12min.1s.20ms), and the set threshold is 38ms. Therefore, the laser data frame corresponding to the laser timestamp 2023.1.1 (10h.12min.1s.20ms) is selected as the target laser data frame, and then each target laser data frame is searched along the time axis according to the laser timestamp 2023.1.1 (10h.12min.1s.20ms). Method two: when the position change of the target object in the map coordinate system between the first position corresponding to the first laser timestamp and the second position corresponding to the second laser timestamp exceeds a set distance threshold, for example, the position change exceeds 10cm, the laser data frame corresponding to the second laser timestamp is processed subsequently. Method three: when the angle change of the target object in the map coordinate system between the first angle corresponding to the first laser timestamp and the second angle corresponding to the second laser timestamp exceeds a set threshold, for example, the angle change exceeds 2°, the laser data frame corresponding to the second laser timestamp is processed subsequently. For the laser data frame that does not meet the above threshold condition, no processing is performed.

[0102] After the controller filters the first point cloud data, it searches for each laser data key frame with a time interval from the first point cloud data that does not exceed a set threshold, and forms a laser data key frame set by combining each laser data key frame. It should be noted that the number of laser data key frames in the laser data key frame set is not less than 30 frames, and the position change of the target object corresponding to any two laser data key frames in each laser data key frame does not exceed 10M.

[0103] Then, any target laser data key frame is selected from the laser data key frame set, and the first laser data key frame and the second laser data key frame adjacent to the target laser data key frame are determined. The local priority search method can be used to determine the first laser data key frame and the second laser data key frame adjacent to the target laser data key frame.

[0104] For example, it is assumed that the target laser data key frame is P si , the first laser data key frame and the second laser data key frame are P si-j and P si+kwherein j and k are positive integers greater than or equal to 1.

[0105] Step one: first, j and k are both assigned as 1, then the first laser data key frame and the second laser data key frame constitute a first data chain.

[0106] Step two: when it is determined that the first data chain does not contain laser data frames, the first laser data key frame P si-1 and the second laser data key frame P si+1 assigned as 1 are removed from the laser data frame set.

[0107] Step three: j and k are both assigned as 2, the third laser data key frame P si-2 and the fourth laser data key frame P si+2 adjacent to the target laser data key frame are determined, then the first laser data key frame P si-1 and the second laser data key frame P si+1 and the third laser data key frame P si-2 and the fourth laser data key frame P si+2 constitute a second data chain.

[0108] Step four: when it is determined that the second data chain does not contain laser data frames, the above step one and step two are repeated until the laser data key frame adjacent to the target laser data key frame cannot be found from the laser data key frame set, all the removed laser data key frames are selected as the second data chain, and a local sub-map is established according to the laser data key frames in the second data chain.

[0109] Then, the controller matches the laser data frame corresponding to the first laser timestamp with the local sub-map after constructing the local sub-map, and obtains the matching pose of the target object at the first laser timestamp.

[0110] Then when it is determined that the similarity score of the matching pose is greater than a set threshold, for example, 0.6, the fifth laser data key frame with the minimum laser data frame time interval corresponding to the first laser timestamp, or the closest position to the target object at the first laser timestamp is selected from the second data chain.

[0111] Because the poses of the target object corresponding to each laser timestamp in the map coordinate system calculated by the odometer may have accumulated errors, the controller needs to establish a first pose constraint condition between the laser data frame and the fifth laser data key frame, and input the first pose constraint condition into the back-end optimizer to obtain a first fusion pose.

[0112] In the above manner, the poses of the target object corresponding to each laser timestamp in the map coordinate system can be preliminarily optimized to obtain a first fusion pose.

[0113] S3, based on the image data, odometer data, and the second pose constraint conditions set for the corresponding image data, obtains the second fused pose.

[0114] After obtaining the first fused pose, the controller needs to analyze the image data collected by the image detection device in order to further optimize the first fused pose, and finally jointly optimize the pose of the target object in the map coordinate system.

[0115] In this embodiment of the application, when the target object arrives at the location of each preset location marker, the pose of the target object at adjacent camera moments can be accurately estimated by scanning the QR code or texture code by the image detection device. Here, adjacent camera moments refer to the two adjacent camera moments when the target object arrives at any location marker.

[0116] The image detection device scans along a preset driving path according to a set scanning cycle. Assuming that during the scanning process, it can scan various location markers set in the map coordinate system, the controller first determines whether to perform a loopback scan and scan the set location marker 2. If the loopback scan does not scan the location marker 2, assuming that the adjacent camera timestamps at location marker 2 are Tc1 and Tc2, then Tc1 and Tc2 can be determined as the first image timestamp and the second image timestamp, where Tc1 is the previous camera timetamp of Tc2.

[0117] Next, the controller finds the third laser timestamp Tli, which is adjacent to the first image timestamp, and the fourth laser timestamp Tls, which is adjacent to the second image timestamp, based on the first image timestamp. The third and fourth laser timestamps are... Figure 3 None of them are marked. The same steps as above are used to find the first odometer timestamp adjacent to the first laser timestamp, and to find the second odometer timestamp adjacent to the third laser timestamp and the third odometer timestamp adjacent to the fourth laser timestamp.

[0118] Then, based on the second odometer timestamp and the corresponding second odometer data, the fifth pose T of the target object in the odometer coordinate system corresponding to the first image timestamp is determined. Tc1 Similarly, based on the third odometer timestamp and the corresponding third odometer data, the sixth pose T of the target object in the odometer coordinate system corresponding to the second image timestamp is determined. Tc2 The calculation methods for the fifth and sixth poses can be found in Formula 1 or Formula 2, and will not be elaborated here.

[0119] After obtaining the fifth pose of the target object in the odometry coordinate system corresponding to the first image timestamp and the sixth pose of the second image timestamp, the controller, using the method described in step 2 above, determines the seventh pose T of the target object in the map coordinate system corresponding to the first image timestamp. mapTc1 and the eighth pose T corresponding to the second image timestamp mapTc2 The calculation method for the seventh pose can refer to the following formula 4:

[0120] T mapTc1 =T mapTli ×(T Tli ) -1 ×T Tls

[0121] Among them, T mapTli T represents the pose of the target object in the map coordinate system corresponding to the third laser timestamp. Tli T Tls The poses of the target object in the odometry coordinate system, corresponding to the third and fourth laser timestamps, can be represented by (T). Tli ) -1 ×T Tls This is called relative pose. Right now:

[0122] Finally, based on the pose change between the seventh pose of the first image timestamp and the eighth pose of the second image timestamp, a second pose constraint is constructed between the first image data frame corresponding to the first image timestamp and the second image data frame corresponding to the second image timestamp. The second pose constraint is the residual e. ci The formula for calculating the residual can be found in Formula 5 below:

[0123]

[0124] By importing the second pose constraint into the backend optimizer, the solution can be obtained. The value; The translation amount of the target object's pose at camera time Tc1; The yaw angle is the pose of the target object at time Tc1; The translation amount of the target object's pose at camera time Tc2; The yaw angle is the pose of the target object at time Tc2; Relative poses The translation and rotation amounts.

[0125] Following the construction of the second pose constraint from camera time Tc1 to camera time Tc2, the controller also needs to construct the fourth pose constraint from the third laser timestamp Tli to camera time Tc1. Similar to the steps for constructing the second pose constraint, the fourth pose constraint e is determined by referring to Formula 5. Tli :

[0126]

[0127] in, The translation amount of the target object's pose at the laser moment Tli; Let yaw angle be the pose of the target object at time Tli; The translation amount of the target object's pose at camera time Tc1; The yaw angle is the pose of the target object at time Tc1; Relative poses Translation and rotation, relative pose The calculation method can be found in Formula 4, and will not be repeated here.

[0128] Similarly, a fifth pose constraint condition needs to be constructed between the fourth laser timestamp Tls and the camera time Tc2, similar to the construction of the fourth pose constraint condition e described above. Tli Following the steps outlined in Formula 5, determine the fifth pose constraint condition e. Tls :

[0129]

[0130] in, The translation amount of the target object's pose at the laser moment (Tls); The yaw angle is the pose of the target object at time Tls; The translation amount of the target object's pose at camera time Tc2; The yaw angle is the pose of the target object at time Tc2; Relative poses Translation and rotation, relative pose The calculation method can also refer to Formula 4, and will not be repeated here.

[0131] The controller needs to further establish the relative pose constraint between the image detection device and the position markers after establishing the fourth and fifth pose constraints. For the plurality of first camera data frames collected when the image detection device scans the position marker 2 and the plurality of second camera data frames collected when the image detection device scans the position marker 3, a relative pose constraint can be established by selecting any one first target camera data frame from the plurality of first camera data frames and any one second target camera data frame from the plurality of second camera data frames. The calculation formula of the relative pose constraint can refer to formula 5, which will not be described here.

[0132] If the loop scanning is performed and the position marker 2 is scanned, the controller first determines the third timestamp as the time when the image detection device scans the position marker 2 for the second time, then finds the fifth laser timestamp adjacent to the third image timestamp and the fourth odometry timestamp adjacent to the fifth laser timestamp, and determines the ninth pose of the target object in the odometry coordinate system corresponding to the third image timestamp according to the fourth odometry timestamp and the fourth odometry data corresponding to the fourth odometry timestamp. The calculation method of the ninth pose can refer to formulas 1 and 2, which will not be described here.

[0133] After the ninth pose is obtained, the tenth pose of the target object in the map coordinate system corresponding to the third image timestamp is calculated according to formula 3. Finally, the seventh pose T mapTc1 According to the pose change between the seventh pose and the tenth pose, a third pose constraint condition between the first image data frame and the third image data frame corresponding to the third image timestamp is constructed, which is called a loop constraint condition of the camera. The calculation method of the loop constraint condition can refer to formula 5, which will not be described here.

[0134] Finally, the second, fourth, fifth pose constraints and the loop constraint condition of the camera are added to the back-end optimizer, and the second fused pose of the target object in the map coordinate system at the camera time can be obtained.

[0135] Through the above method, the second pose constraint condition between adjacent camera times when the target object arrives at the position marker 2, the fourth pose constraint condition between the third laser timestamp Tli and the camera time Tc1, the fifth pose constraint condition between the fourth laser timestamp Tls and the camera time Tc2, and the third pose constraint condition between the first image data frame and the third image data frame when the camera scans the position marker 2 can be constructed, and the second fused pose of the target object in the map coordinate system at the camera time can be obtained.

[0136] S4, fuse the first fused pose and the second fused pose to obtain the target fused pose.

[0137] In the embodiment of the present application, after obtaining the first fused pose and the second fused pose respectively, in order to further make the pose estimation of the target object in the map coordinate system more accurate, the controller can fuse the first fused pose and the second fused pose, for example, a first weight value can be assigned to the first fused pose, and a second weight value can be assigned to the second fused pose, then the result obtained by multiplying the first fused pose by the first weight value and the second fused pose by the second weight value is added, to obtain the target fused pose of the target object in the map coordinate system. It should be pointed out here that because the distance between the position identifier 1 and the position identifier 2 and the deviation angle are pre-set, the relative position constraint constructed between different position identifiers is relatively accurate, so when fusing the first fused pose and the second fused pose, the weight value corresponding to the first pose constraint between laser data frames is reduced, that is, the first weight value is reduced, and the second weight value assigned to the second fused pose is increased.

[0138] In the above manner, the first fused pose and the second fused pose can be data fused according to the respective weight values assigned thereto to obtain the target fused pose, so that the pose estimation of the target object in the map coordinate system is more accurate.

[0139] S5, generating target map data based on the target fused pose, the point cloud data, and the image data.

[0140] In the embodiment of the present application, after obtaining the target fused pose, the controller can convert the point cloud data associated with the laser data frame to the grid coordinate in time sequence, update the obstacle information on the grid while converting the point cloud data to the grid coordinate, then traverse each grid to obtain the respective obstacle probability value corresponding to each grid, when the obtained obstacle probability value reaches a set threshold, for example, 0.7, it is considered that the grid is occupied by the obstacle, finally, the updated grid coordinate is restored to the map coordinate system to obtain the target map data, and the target map is generated from the target map data.

[0141] In summary, the data fusion map construction method provided in the application can obtain point cloud data, image data and odometry data collected by a target object during driving, then perform preliminary optimization on the pose of the target object in a map coordinate system at each laser timestamp corresponding to the point cloud data and the odometry data, to obtain a first fusion pose, then obtain a second fusion pose of the camera moment of the target object in the map coordinate system according to the third pose constraint condition between the first image data frame and the third image data frame, perform data fusion on the first fusion pose and the second fusion pose according to the respective assigned weight values, to obtain a target fusion pose, so that the pose estimation of the target object in the map coordinate system is more accurate, and finally, the target fusion pose, the point cloud data and the image data are combined to generate target map data, thereby improving the accuracy of constructing a prior map.

[0142] Based on the method provided in the above embodiments, the embodiments of the application further provide a data fusion map construction system, as shown in Figure 4 FIG. 1 shows a structure schematic diagram of a data fusion map construction system in an embodiment of the application, which comprises:

[0143] The data acquisition module 401 is configured to acquire point cloud data, image data and odometry data collected by a target object during driving on a preset driving path.

[0144] The data fusion module 402 is configured to obtain a first fusion pose based on the point cloud data, the odometry data and a first pose constraint condition set for the point cloud data.

[0145] Obtain a second fusion pose based on the image data, the odometry data and a second pose constraint condition set for the image data.

[0146] Fuse the first fusion pose and the second fusion pose to obtain a target fusion pose.

[0147] The map construction module 403 is configured to generate target map data based on the target fusion pose, the point cloud data and the image data.

[0148] In a possible implementation, the data fusion module 402 is specifically configured to:

[0149] Determine a first laser timestamp from the point cloud data, and find a first odometry timestamp adjacent to the first laser timestamp, wherein the laser timestamp and the odometry timestamp are located on the same time axis.

[0150] Calculate a first pose of the target object in an odometry coordinate system based on the first odometry timestamp and odometry data corresponding to the first odometry timestamp.

[0151] determine, based on the first pose of the target object, a second pose corresponding to a second laser timestamp in a lidar coordinate system, a third pose corresponding to the second laser timestamp in a map coordinate system, a fourth pose of the target object corresponding to the first laser timestamp in the map coordinate system, wherein the second laser timestamp is a previous time point of the first laser timestamp.

[0152] In a possible implementation, the data fusion module 402 is further configured to:

[0153] extract a laser data frame corresponding to the first laser timestamp;

[0154] filter, from the point cloud data, first point cloud data satisfying a data feature condition, find, from the first point cloud data, each laser data key frame having a time interval from the laser data frame not exceeding a set threshold, and form a laser data key frame set;

[0155] select any target laser data key frame in the laser data key frame set, and determine a first laser data key frame and a second laser data key frame adjacent to the target laser data key frame;

[0156] when it is determined that the first laser data key frame and the second laser data key frame do not constitute a first data chain containing the laser data frame, remove the first laser data key frame and the second laser data key frame from the laser data key frame set, and determine a third laser data key frame and a fourth laser data key frame adjacent to the target laser data key frame;

[0157] when it is determined that the first data chain, the third laser data key frame, and the fourth laser data key frame do not constitute a second data chain containing the laser data frame, establish a local submap based on laser data key frames contained in the second data chain;

[0158] match the laser data frame with the local submap to obtain a matching pose of the target object;

[0159] when it is determined that a similarity score of the matching pose is greater than a set threshold, select a fifth laser data key frame having a minimum time interval from the laser data frame in the second data chain;

[0160] establish a first pose constraint condition between the laser data frame and the fifth laser data key frame to obtain a first fusion pose.

[0161] Based on the same inventive concept, the embodiment of the present application further provides an electronic device, which can realize the functions of the data fusion map construction method, and the electronic device comprises Figure 5 , and the electronic device comprises:

[0162] at least one processor 501 and a memory 502 connected with the at least one processor 501, and the specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application, Figure 5 In the embodiment of the present application, the connection between the processor 501 and the memory 502 is taken as an example of connection through a bus 500. The bus 500 is represented by a thick line in Figure 5 , and the connection mode between other components is only schematically illustrated and is not limited. The bus 500 can be divided into an address bus, a data bus, a control bus and the like, and for the convenience of representation, Figure 5 In the embodiment of the present application, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor 501 can also be called a controller, and the name is not limited.

[0163] In the embodiment of the present application, the memory 502 stores instructions executable by the at least one processor 501, and the at least one processor 501 can execute the data fusion map construction method discussed above by executing the instructions stored in the memory 502. The processor 501 can realize the functions of various modules in the system as shown in Figure 4 .

[0164] Among them, the processor 501 is the control center of the device, which can connect each part of the whole control device through various interfaces and lines, and through running or executing the instructions stored in the memory 502 and calling the data stored in the memory 502, the device various functions and processing data, thereby the device is overall monitored.

[0165] In a possible design, the processor 501 can include one or more processing units, and the processor 501 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, and the modem processor mainly processes the wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 can be realized on the same chip, and in some embodiments, they can also be realized on independent chips respectively.

[0166] The processor 501 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the data fusion map construction method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0167] The memory 502 is a non-volatile computer readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 502 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 502 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0168] By designing and programming the processor 501, the code corresponding to the data fusion map construction method introduced in the foregoing embodiments can be fixed into the chip, so that the chip can execute the steps of the data fusion map construction method of the embodiments shown in the running time. Figure 1 How to design and program the processor 501 is a technology known to those skilled in the art, which will not be described here.

[0169] Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions, when the computer instructions run on a computer, the computer executes the data fusion map construction method discussed above.

[0170] In some possible implementation, the various aspects of the data fusion based map construction method provided by the present application can also be implemented in the form of a program product, which includes program codes for causing the control device to perform the steps of the data fusion based map construction method according to various exemplary embodiments of the present application described above in the specification when the program product is run on the device.

[0171] Those skilled in the art should understand that embodiments of the present application can be provided in the form of a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowcharts and / or block diagrams.

[0173] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowcharts and / or block diagrams.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowcharts and / or block diagrams.

[0175] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A data fusion-based map construction method, characterized by, The method comprises the following steps: acquiring point cloud data, image data and odometer data collected by a target object during driving on a preset driving path; obtaining a first fusion pose based on the point cloud data, the odometer data and a first pose constraint condition set according to the point cloud data, and obtaining a second fusion pose based on the image data, the odometer data and a second pose constraint condition set according to the image data; fusing the first fusion pose and the second fusion pose to obtain a target fusion pose; generating target map data based on the target fusion pose, the point cloud data and the image data; if the first position identifier is not scanned by the loop scanning, determining a first image timestamp and a second image timestamp from the image data, finding a third laser timestamp adjacent to the first image timestamp and a fourth laser timestamp adjacent to the second image timestamp, finding a second odometer timestamp adjacent to the third laser timestamp and a third odometer timestamp adjacent to the fourth laser timestamp, determining a fifth pose of the target object corresponding to the first image timestamp in the odometer coordinate system and a seventh pose of the target object corresponding to the first image timestamp in the odometer coordinate system based on the second odometer timestamp and second odometer data corresponding to the second odometer timestamp, determining a sixth pose of the target object corresponding to the second image timestamp in the odometer coordinate system and an eighth pose of the target object corresponding to the second image timestamp in the odometer coordinate system based on the third odometer timestamp and third odometer data corresponding to the third odometer timestamp, and constructing a second pose constraint condition between a first image data frame corresponding to the first image timestamp and a second image data frame corresponding to the second image timestamp based on a pose change amount between the seventh pose and the eighth pose; if the first position identifier is scanned by the loop scanning, determining a third image timestamp as a time when the first position identifier is scanned for the second time, finding a fifth laser timestamp adjacent to the third image timestamp and a fourth odometer timestamp adjacent to the fifth laser timestamp, determining a ninth pose of the target object corresponding to the third image timestamp in the odometer coordinate system and a tenth pose of the target object corresponding to the third image timestamp in the map coordinate system based on the fourth odometer timestamp and fourth odometer data corresponding to the fourth odometer timestamp, constructing a third pose constraint condition between the first image data frame and a third image data frame corresponding to the third image timestamp based on a pose change amount between the tenth pose and the seventh pose, and obtaining the second fusion pose based on the second pose constraint condition and the third pose constraint condition.

2. The method of claim 1, wherein, The method comprises the following steps: determining a first laser timestamp from the point cloud data and finding a first odometer timestamp adjacent to the first laser timestamp, wherein the laser timestamp and the odometer timestamp are located on the same time axis. determine a first pose of the target object in a map coordinate system corresponding to the first laser timestamp based on the first pose of the target object in the map coordinate system, a second pose in the map coordinate system corresponding to a second laser timestamp, and a third pose in the map coordinate system corresponding to the second laser timestamp, wherein the second laser timestamp is a time point prior to the first laser timestamp. The determining, from the point cloud data, of the first laser timestamp comprises:

3. The method of claim 2, wherein, extracting a laser data frame corresponding to the first laser timestamp; filtering, from the point cloud data, first point cloud data satisfying a data feature condition, and finding, from the first point cloud data, each laser data key frame having a time interval from the laser data frame not exceeding a set threshold, and grouping the laser data key frames to form a laser data key frame set; selecting any target laser data key frame in the laser data key frame set, and determining a first laser data key frame and a second laser data key frame adjacent to the target laser data key frame; when it is determined that the laser data frame is not included in a first data chain formed by the first laser data key frame and the second laser data key frame, removing the first laser data key frame and the second laser data key frame from the laser data key frame set, and determining a third laser data key frame and a fourth laser data key frame adjacent to the target laser data key frame; when it is determined that the laser data frame is not included in a second data chain formed by the first data chain, the third laser data key frame, and the fourth laser data key frame, establishing a local submap based on laser data key frames included in the second data chain; matching the laser data frame with the local submap to obtain a matching pose of the target object; when it is determined that a similarity score of the matching pose is greater than a set threshold, selecting a fifth laser data key frame having a minimum time interval from the laser data frame in the second data chain; establishing a first pose constraint condition between the laser data frame and the fifth laser data key frame to obtain a first fusion pose. The method comprises:

4. A data fusion-based map construction system characterized by comprising: a data acquisition module configured to acquire point cloud data, image data, and odometry data collected during a driving process of a target object on a preset driving path; a data fusion module configured to obtain a first fusion pose based on the point cloud data, the odometry data, and a first pose constraint condition set corresponding to the point cloud data; obtain a second fusion pose based on the image data, the odometry data, and a second pose constraint condition set corresponding to the image data; and fuse the first fusion pose and the second fusion pose to obtain a target fusion pose; a map construction module configured to generate target map data based on the target fusion pose, the point cloud data, and the image data. ​ If the loop-back scanning does not scan the preset first position identifier, the first image timestamp and the second image timestamp are determined from the image data; a third laser timestamp adjacent to the first image timestamp and a fourth laser timestamp adjacent to the second image timestamp are found; a second odometry timestamp adjacent to the third laser timestamp and a third odometry timestamp adjacent to the fourth laser timestamp are found; the fifth pose of the target object in the odometry coordinate system corresponding to the first image timestamp and the seventh pose of the target object in the odometry coordinate system corresponding to the first image timestamp are determined based on the second odometry timestamp and second odometry data corresponding to the second odometry timestamp; the sixth pose of the target object in the odometry coordinate system corresponding to the second image timestamp and the eighth pose of the target object in the odometry coordinate system corresponding to the second image timestamp are determined based on the third odometry timestamp and third odometry data corresponding to the third odometry timestamp; and the second pose constraint condition between the first image data frame corresponding to the first image timestamp and the second image data frame corresponding to the second image timestamp is constructed based on the pose change amount between the seventh pose and the eighth pose. If the loop-back scanning scans the first position identifier, the third image timestamp is determined as the time when the first position identifier is scanned for the second time; the fifth laser timestamp adjacent to the third image timestamp and the fourth odometry timestamp adjacent to the fifth laser timestamp are found; the ninth pose of the target object in the odometry coordinate system corresponding to the third image timestamp and the tenth pose of the target object in the map coordinate system corresponding to the third image timestamp are determined based on the fourth odometry timestamp and fourth odometry data corresponding to the fourth odometry timestamp; the third pose constraint condition between the first image data frame and the third image data frame corresponding to the third image timestamp is constructed based on the pose change amount between the tenth pose and the seventh pose; and the second fusion pose is obtained based on the second pose constraint condition and the third pose constraint condition.

5. The system of claim 4, wherein, The data fusion module is specifically configured to: determine a first laser timestamp from the point cloud data and find a first odometry timestamp adjacent to the first laser timestamp, wherein the laser timestamp and the odometry timestamp are located on the same time axis; calculate a first pose of the target object in the odometry coordinate system based on the first odometry timestamp and odometry data corresponding to the first odometry timestamp; determine a fourth pose of the target object in the map coordinate system corresponding to the first laser timestamp based on the first pose of the target object, a second pose in the odometry coordinate system corresponding to a second laser timestamp, and a third pose in the map coordinate system corresponding to the second laser timestamp, wherein the second laser timestamp is the last time of the first laser timestamp.

6. The system of claim 4, wherein, The data fusion module is further configured to: extract a laser data frame corresponding to the first laser timestamp; From the point cloud data, first point cloud data meeting a data feature condition is screened out, and each laser data key frame with a time interval from the laser data frame not exceeding a set threshold is found out from the first point cloud data, and a laser data key frame set is formed; Any target laser data key frame in the laser data key frame set is selected, and a first laser data key frame and a second laser data key frame adjacent to the target laser data key frame are determined; When it is determined that the laser data frame is not contained in a first data chain formed by the first laser data key frame and the second laser data key frame, the first laser data key frame and the second laser data key frame are removed from the laser data key frame set, and a third laser data key frame and a fourth laser data key frame adjacent to the target laser data key frame are determined; When it is determined that the laser data frame is not contained in a second data chain formed by the first data chain, the third laser data key frame and the fourth laser data key frame, a local submap is established based on laser data key frames contained in the second data chain; The laser data frame is matched with the local submap to obtain a matching pose of the target object; When it is determined that a similarity score of the matching pose is greater than a set threshold, a fifth laser data key frame with a minimum time interval from the laser data frame is selected in the second data chain; A first pose constraint condition between the laser data frame and the fifth laser data key frame is established to obtain a first fusion pose.

7. An electronic device, comprising: Comprise: A memory for storing a computer program; A processor for executing the computer program stored on the memory to implement the method of any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-3.

Images