Data fusion method, device and equipment and computer readable storage medium

Abstract

The present disclosure relates to a data fusion method, device, equipment and computer readable storage medium, the present disclosure obtains a first image and a second image corresponding to the first image, fuses the first image and the second image according to the position of historical point cloud included in historical point cloud data, obtains a third image, and provides data basis for subsequent optimization of the third image; based on the global image corresponding to the historical point cloud data of the mobile carrier in the process of driving on the preset track, the third image is optimized, so that the accuracy of the image is improved; the fourth image corresponding to the third image is obtained from the global image, and if the difference between the fourth image and the optimized third image is less than a preset value, the optimized third image is determined as the fused image, the accuracy of image fusion is further improved, local map update and expansion are realized, and work efficiency is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data fusion method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Point clouds are the raw output data of most 3D information acquisition devices. Compared with 2D images, point cloud data can provide a more comprehensive description of real-world scenes. In recent years, with the improvement of computer hardware performance and the popularization of point cloud data acquisition devices, point cloud data utilization technology has been increasingly applied in fields such as intelligent robots, autonomous driving, advanced manufacturing, virtual reality, and augmented reality.

[0003] In some cases, it is necessary to fuse the images corresponding to point clouds. However, during the fusion process, there are instances of inaccurate fusion, resulting in long fusion times and low efficiency. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a data fusion method, apparatus, device, and computer-readable storage medium to improve fusion accuracy and increase efficiency.

[0005] In a first aspect, embodiments of this disclosure provide a data fusion method, including:

[0006] Acquire a first image and a second image corresponding to the first image. The first image is a two-dimensional image of the current point cloud data collected during the movement of the mobile carrier along a route of a preset length. The second image is a two-dimensional image of the historical point cloud data. The current point cloud data and the historical point cloud data correspond to the same route of a preset length.

[0007] Based on the location of the historical point cloud included in the historical point cloud data, the first image and the second image are fused to obtain a third image;

[0008] Based on the global image corresponding to the historical point cloud data during the process of the mobile carrier traveling on the preset trajectory, the third image is optimized. The preset trajectory includes multiple routes of the preset length.

[0009] Obtain a fourth image corresponding to the third image from the global image, wherein the third image and the fourth image correspond to the same route of a preset length;

[0010] If the difference between the fourth image and the optimized third image is less than a preset value, then the optimized third image is determined to be the fused image.

[0011] In some embodiments, if the difference between the fourth image and the optimized third image is greater than or equal to a preset value, the optimized third image is used as the second image, and the first image and the second image are merged again to obtain a new fused image. The new fused image is then optimized until the difference between the fourth image and the optimized new fused image is less than the preset value.

[0012] In some embodiments, fusing the first image and the second image to obtain a third image includes:

[0013] The first image is superimposed on the second image to obtain the third image.

[0014] In some embodiments, optimizing the third image includes:

[0015] The pose graph of the third image is optimized.

[0016] In some embodiments, the first image is a two-dimensional image obtained by flattening the current point cloud data; the second image is a two-dimensional image obtained by flattening the historical point cloud data; the two-dimensional image is used to characterize the intensity and height information of the target object corresponding to the current point cloud data.

[0017] Secondly, embodiments of this disclosure provide a data fusion apparatus, comprising:

[0018] The first acquisition module is used to acquire a first image and a second image corresponding to the first image. The first image is a two-dimensional image of the current point cloud data collected during the process of the mobile carrier traveling on a route of a preset length. The second image is a two-dimensional image of the historical point cloud data. The current point cloud data and the historical point cloud data correspond to the same route of a preset length.

[0019] The fusion module is used to fuse the first image and the second image according to the location of the historical point cloud included in the historical point cloud data to obtain a third image;

[0020] An optimization module is used to optimize the third image based on the global image corresponding to the historical point cloud data during the process of the mobile carrier traveling on a preset trajectory. The preset trajectory includes multiple routes of the preset length.

[0021] The second acquisition module is used to acquire a fourth image corresponding to the third image from the global image, wherein the third image and the fourth image correspond to the same route of a preset length;

[0022] The determining module is configured to determine the optimized third image as the fused image if the difference between the fourth image and the optimized third image is less than a preset value.

[0023] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0024] Memory;

[0025] Processor; and

[0026] Computer programs;

[0027] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0028] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in the first aspect.

[0029] Fifthly, embodiments of this disclosure also provide a computer program product comprising a computer program or instructions which are executed by a processor to implement the method described in the first aspect.

[0030] The data fusion method, apparatus, device, and computer-readable storage medium provided in this disclosure acquire a first image and a second image corresponding to the first image. Based on the location of historical point clouds included in historical point cloud data, the first image and the second image are fused to obtain a third image, providing a data foundation for subsequent optimization of the third image. Based on the global image corresponding to the historical point cloud data during the movement of the mobile carrier on a preset trajectory, the third image is optimized to improve its accuracy. A fourth image corresponding to the third image is acquired from the global image. If the difference between the fourth image and the optimized third image is less than a preset value, the optimized third image is determined to be the fused image, further improving image accuracy, realizing local map updates and expansion, and improving work efficiency. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0032] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart of the data fusion method provided in this embodiment of the disclosure;

[0034] Figure 2 A flowchart of the data fusion method provided in this embodiment of the disclosure;

[0035] Figure 3 A schematic diagram of the structure of the data fusion apparatus provided in the embodiments of this disclosure;

[0036] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0037] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0038] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0039] This disclosure provides a data fusion method, which will be described below with reference to specific embodiments.

[0040] Figure 1 This is a flowchart illustrating the data fusion method provided in this embodiment. This method can be applied to image fusion scenarios. It is understood that the data fusion method provided in this embodiment can also be applied to other scenarios. The following describes... Figure 1 The data fusion method shown is described below, and the specific steps of this method are as follows:

[0041] S101. Obtain a first image and a second image corresponding to the first image. The first image is a two-dimensional image of the current point cloud data collected during the movement of the mobile carrier on a route of a preset length. The second image is a two-dimensional image of the historical point cloud data. The current point cloud data and the historical point cloud data correspond to the same route of a preset length.

[0042] Radar (radio detection and ranging), also known as radio positioning, uses radio waves to detect targets and determine their spatial location. Radar measures velocity based on the Doppler effect, which occurs when there is relative motion between the radar and the target. It measures distance by measuring the time difference between the transmitted pulse and the echo pulse; since electromagnetic waves travel at the speed of light, this time difference can be used to calculate the precise distance between the radar and the target.

[0043] Point cloud, in reverse engineering, is a collection of point data on the surface of a product obtained through measuring instruments. Usually, the number of points obtained by a 3D coordinate measuring machine is relatively small and the distance between points is relatively large, which is called sparse point cloud; while the point cloud obtained by a 3D laser scanner or photogrammetric scanner has a large number of points and is relatively dense, which is called dense point cloud.

[0044] Real-time kinematic (RTK) carrier phase differential technology is a method for processing the carrier phase observations of two measurement stations in real time. It sends the carrier phase collected by the reference station to the user receiver and calculates the coordinates by difference.

[0045] The mobile carrier is equipped with RTK (Real-Time Kerneling) to improve positioning accuracy; for example, the mobile carrier could be a vehicle. The mobile carrier also carries radar sensors to extract point cloud data collected as it travels along a predetermined route. For example, this could be point cloud data from 120 degrees forward of the vehicle's trajectory. When the mobile carrier travels the predetermined route for the predetermined length, this predetermined length can be divided, for example, into multiple data units of 50 meters. It is understood that if the predetermined length is less than or equal to 50 meters, then no division is needed; if the predetermined length is greater than 50 meters and is an integer multiple of 50 meters, then it is divided into 50-meter units; if the predetermined length is greater than 50 meters and is not an integer multiple of 50 meters, it is first divided into 50-meter units until the last data unit is less than 50 meters. It is understood that the predetermined length can be other lengths. This embodiment uses 50-meter units for division; other embodiments may also use different lengths, and this embodiment does not specifically limit this.

[0046] The first image is a two-dimensional image obtained by flattening the current point cloud data collected during the movement of the mobile carrier along a route of a preset length. The second image is a two-dimensional image obtained by flattening historical point cloud data. The current point cloud data and historical point cloud data correspond to the same route of a preset length. This two-dimensional image is used to characterize the intensity and height information of the target object corresponding to the current point cloud data.

[0047] Intensity information refers to the information representing the brightness of the target object obtained after measurement by the measuring instrument. In this embodiment, the measuring instrument is radar. Height information refers to the information representing the height of the target object obtained after measurement by the measuring instrument.

[0048] Optionally, radar measurement can also be replaced by other measurement methods such as laser measurement or photogrammetry; this embodiment does not impose any specific limitations.

[0049] It is understandable that when there is no second image in the cloud corresponding to the first image, the first image can be used as the second image. The mobile carrier collects point cloud data for the second time along a route of a preset length. The cloud receives multiple data units sent by the mobile carrier for the second time. Each data unit includes a two-dimensional image obtained by flattening the current point cloud data collected for the second time.

[0050] S102. Based on the location of the historical point cloud included in the historical point cloud data, the first image and the second image are fused to obtain a third image.

[0051] Point cloud data refers to a collection of vectors in a three-dimensional coordinate system. Scanned data is recorded in the form of points, each containing three-dimensional coordinates, and some may contain information such as reflection intensity.

[0052] Based on historical point cloud data, including the three-dimensional coordinates of historical point clouds, the first image is superimposed on the second image, and the third image is obtained by using the first image as the reference and the second image as the auxiliary.

[0053] For example, if the first image contains point cloud data of a traffic light at its center, but the second image does not contain point cloud data of a traffic light at the same location, it can be assumed that a new traffic light was installed at that location during the time difference between acquiring the first and second images. Using the first image as a reference, and overlaying the data, the third image will also contain point cloud data of a traffic light at the same location. It is understood that this embodiment is provided as an example; the point cloud data is not limited to traffic lights but may also include guardrails, medians, traffic signs, etc., and the location is not limited to the center but can also be other locations.

[0054] S103. Based on the global image corresponding to the historical point cloud data during the process of the mobile carrier traveling on the preset trajectory, the third image is optimized. The preset trajectory includes multiple routes of the preset length.

[0055] Pose graph optimization uses vertices to represent optimization variables and edges to represent error terms. It leverages short-interval relative pose measurements to construct a global optimization problem covering keyframes over a long time span, amortizing accumulated errors. Using the pose of each keyframe as a node and the relative pose calculation results between adjacent keyframes as measurements, it avoids optimizing massive amounts of map points, reducing computational scale.

[0056] Based on the global image corresponding to the historical point cloud data during the process of the mobile vehicle traveling on the preset trajectory, the third image is optimized. The preset trajectory includes multiple routes of the preset length, which are the routes that the mobile vehicle travels on the preset trajectory.

[0057] S104. Obtain a fourth image corresponding to the third image from the global image, wherein the third image and the fourth image correspond to the same route of a preset length.

[0058] Obtain a fourth image corresponding to the third image from the global image. This fourth image corresponds to the same route of a preset length as the third image. In other words, the location information represented by the fourth image and the third image is consistent.

[0059] S105. If the difference between the fourth image and the optimized third image is less than a preset value, then the optimized third image is determined to be the fused image.

[0060] The optimized third image needs to save the pose change information of the boundary nodes on the two neighborhood images and compare it with the adjacent nodes in the global image; if the difference between the fourth image and the optimized third image is less than a preset value, then the optimized third image is determined to be the fused image.

[0061] This embodiment of the disclosure acquires a first image and a second image corresponding to the first image. Based on the location of historical point clouds included in the historical point cloud data, the first image and the second image are fused to obtain a third image, providing a data foundation for subsequent optimization of the third image. Based on the global image corresponding to the historical point cloud data during the movement of the mobile carrier on the preset trajectory, the third image is optimized to improve its accuracy. A fourth image corresponding to the third image is acquired from the global image. If the difference between the fourth image and the optimized third image is less than a preset value, the optimized third image is determined to be the fused image, further improving the image accuracy, realizing local map updates and expansion, and improving work efficiency.

[0062] Figure 2 A flowchart of a data fusion method provided in another embodiment of this disclosure is shown below. Figure 2As shown, the method includes the following steps:

[0063] S201. Obtain the first image and the second image, fuse the first image and the second image to obtain the third image, and optimize the third image.

[0064] Specifically, the implementation principles and specific methods of S201 and the aforementioned S101-S103 are the same, and will not be repeated here.

[0065] S202. Obtain the fourth image corresponding to the third image from the global image.

[0066] Specifically, the implementation principles and specific methods of S202 and S104 mentioned above are the same, and will not be repeated here.

[0067] S203. Determine whether the difference between the fourth image and the optimized third image is less than a preset value. If yes, proceed to step S204; otherwise, proceed to step S205.

[0068] The optimized third image needs to save the pose change information of the boundary nodes on the two-neighborhood image and compare it with the adjacent nodes in the global image to determine whether the difference between the fourth image and the optimized third image is less than a preset value. If yes, then step S204 is executed; if no, then step S205 is executed.

[0069] S204. Determine that the optimized third image is the fused image.

[0070] S205. Use the optimized third image as the second image.

[0071] The optimized third image is used as the second image of the historical point cloud data of the same preset length during the driving process on the preset trajectory.

[0072] S206. Acquire the first image again, fuse the first image with the second image to obtain a new fused image, optimize the new fused image to obtain the optimized third image.

[0073] After the mobile carrier travels on the preset trajectory again, it acquires the optimized third image again. Specifically, the implementation principle and specific method of S206 are the same as those of S101-S103 mentioned above, and will not be repeated here.

[0074] The embodiments disclosed herein improve the accuracy of fusion by continuously iterating and optimizing the third image in the cloud, thereby reducing errors, making the data more accurate, and improving the fusion accuracy.

[0075] Figure 3This is a schematic diagram of the structure of a data fusion apparatus provided in an embodiment of this disclosure. The data fusion apparatus may be an electronic device as described in the above embodiments, or it may be a component or assembly within that electronic device. The data fusion apparatus provided in this embodiment of the disclosure can execute the processing flow provided in the data fusion method embodiments, such as... Figure 3 As shown, the data fusion device 30 includes: a first acquisition module 31, a fusion module 32, an optimization module 33, a second acquisition module 34, and a determination module 35; wherein, the first acquisition module 31 is used to acquire a first image and a second image corresponding to the first image, the first image being a two-dimensional image of current point cloud data collected during the movement of a mobile carrier along a route of a preset length, and the second image being a two-dimensional image of historical point cloud data, the current point cloud data and the historical point cloud data corresponding to the same route of a preset length; the fusion module 32 is used to, based on the location of the historical point cloud included in the historical point cloud data, fusion the first image and the second image. The image is fused with the second image to obtain a third image; the optimization module 33 is used to optimize the third image based on the global image corresponding to the historical point cloud data during the movement of the mobile carrier on the preset trajectory, the preset trajectory including multiple routes of the preset length; the second acquisition module 34 is used to acquire a fourth image corresponding to the third image from the global image, the third image and the fourth image corresponding to the same route of the preset length; the determination module 35 is used to determine the optimized third image as the fused image if the difference between the fourth image and the optimized third image is less than a preset value.

[0076] Optionally, the data fusion device 30 further includes a judgment module 36. The judgment module 36 is configured to, if the difference between the fourth image and the optimized third image is greater than or equal to a preset value, use the optimized third image as the second image, acquire the first image again and fuse it with the second image to obtain a new fused image, and optimize the new fused image until the difference between the fourth image and the optimized new fused image is less than the preset value.

[0077] Optionally, the fusion module 32 is further configured to fuse the first image and the second image to obtain a third image, including:

[0078] The first image is superimposed on the second image to obtain the third image.

[0079] Optionally, the optimization module 33 is used to perform pose graph optimization on the third image.

[0080] Optionally, the first image is a two-dimensional image obtained by flattening the current point cloud data; the second image is a two-dimensional image obtained by flattening the historical point cloud data; the two-dimensional image is used to characterize the intensity and height information of the target object corresponding to the current point cloud data.

[0081] Figure 3 The data fusion apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0082] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device provided in an embodiment of this disclosure can execute the processing flow provided in the data fusion method embodiment, such as... Figure 4 As shown, the electronic device 40 includes: a memory 41, a processor 42, a computer program, and a communication interface 43; wherein the computer program is stored in the memory 41 and configured to be executed by the processor 42 using the data fusion method described above.

[0083] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the data fusion method described in the above embodiments.

[0084] Furthermore, this disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the data fusion method described above.

[0085] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0086] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0087] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0088] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0089] Acquire a first image and a second image corresponding to the first image. The first image is a two-dimensional image of the current point cloud data collected during the movement of the mobile carrier along a route of a preset length. The second image is a two-dimensional image of the historical point cloud data. The current point cloud data and the historical point cloud data correspond to the same route of a preset length.

[0090] Based on the location of the historical point cloud included in the historical point cloud data, the first image and the second image are fused to obtain a third image;

[0091] Based on the global image corresponding to the historical point cloud data during the process of the mobile carrier traveling on the preset trajectory, the third image is optimized. The preset trajectory includes multiple routes of the preset length.

[0092] Obtain a fourth image corresponding to the third image from the global image, wherein the third image and the fourth image correspond to the same route of a preset length;

[0093] If the difference between the fourth image and the optimized third image is less than a preset value, then the optimized third image is determined to be the fused image.

[0094] In addition, the electronic device can also perform other steps in the data fusion method described above.

[0095] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0098] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0099] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0100] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data fusion method, characterized in that, The method includes: Acquire a first image and a second image corresponding to the first image. The first image is a two-dimensional image of the current point cloud data collected during the movement of the mobile carrier along a route of a preset length. The second image is a two-dimensional image of the historical point cloud data. The current point cloud data and the historical point cloud data correspond to the same route of a preset length. Based on the location of the historical point cloud included in the historical point cloud data, the first image and the second image are fused to obtain a third image; Based on the global image corresponding to the historical point cloud data during the process of the mobile carrier traveling on the preset trajectory, the third image is optimized. The preset trajectory includes multiple routes of the preset length. Obtain a fourth image corresponding to the third image from the global image, wherein the third image and the fourth image correspond to the same route of a preset length; If the difference between the fourth image and the optimized third image is less than a preset value, then the optimized third image is determined to be the fused image.

2. The method according to claim 1, characterized in that, The method further includes: If the difference between the fourth image and the optimized third image is greater than or equal to a preset value, then the optimized third image is used as the second image, and the first image and the second image are merged again to obtain a new fused image. The new fused image is then optimized until the difference between the fourth image and the optimized new fused image is less than the preset value.

3. The method according to claim 1, characterized in that, The first image and the second image are fused to obtain a third image, including: The first image is superimposed on the second image to obtain the third image.

4. The method according to claim 1, characterized in that, Optimizing the third image includes: The pose graph of the third image is optimized.

5. The method according to claim 1, characterized in that, The first image is a two-dimensional image obtained by flattening the current point cloud data; the second image is a two-dimensional image obtained by flattening the historical point cloud data; the two-dimensional image is used to characterize the intensity and height information of the target object corresponding to the current point cloud data.

6. A data fusion device, characterized in that, The device includes: The first acquisition module is used to acquire a first image and a second image corresponding to the first image. The first image is a two-dimensional image of the current point cloud data collected during the process of the mobile carrier traveling on a route of a preset length. The second image is a two-dimensional image of the historical point cloud data. The current point cloud data and the historical point cloud data correspond to the same route of a preset length. The fusion module is used to fuse the first image and the second image according to the location of the historical point cloud included in the historical point cloud data to obtain a third image; An optimization module is used to optimize the third image based on the global image corresponding to the historical point cloud data during the process of the mobile carrier traveling on a preset trajectory. The preset trajectory includes multiple routes of the preset length. The second acquisition module is used to acquire a fourth image corresponding to the third image from the global image, wherein the third image and the fourth image correspond to the same route of a preset length; The determining module is configured to determine the optimized third image as the fused image if the difference between the fourth image and the optimized third image is less than a preset value.

7. The apparatus according to claim 6, characterized in that, The device further includes a judgment module, configured to, if the difference between the fourth image and the optimized third image is greater than or equal to a preset value, use the optimized third image as the second image, acquire the first image again and fuse it with the second image to obtain a new fused image, optimize the new fused image until the difference between the fourth image and the optimized new fused image is less than the preset value.

8. The apparatus according to claim 6, characterized in that, The fusion module is further configured to fuse the first image and the second image to obtain a third image, including: The first image is superimposed on the second image to obtain the third image.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images