Map fusion method, device and equipment, readable storage medium and program product
By constructing and fusion of multi -sequentially occupying grid maps, the problem of high difficulty and accuracy of 3D map fusion and low accuracy have been solved, and high -precision map fusion in complex open -air mines.
Patent Information
- Application Number
- CN202310886648.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-19
- Publication Date
- 2025-01-21
AI Technical Summary
In the prior art, 3D maps are difficult to integrate and their accuracy is difficult to ensure, especially in complex and changeable open-pit mine operation scenarios.
By obtaining the observation point cloud data in the current time series of the target scene, an occupancy grid map is constructed, in which each grid has a probability value indicating the status of the grid (occupied, free, or unknown). Then, the occupancy grid maps of the current time series and the historical time series are fused to generate the target occupancy grid map.
By constructing a long -sequence rating map and fusion according to the state of each grid, it can clearly distinguish the newly opened space and change space, and improve the accuracy and difficulty of map fusion.
Smart Images
Figure CN119334331A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of map fusion technology, and in particular, relates to a map fusion method, device, equipment, readable storage medium and program product. Background Art
[0002] In response to the needs of digital production, in some work scenarios, such as open-pit mining operations, it is necessary to dynamically update three-dimensional (3D) maps to ensure the normal operation of the operation. The update process of 3D maps often requires the correct handling of the scene's changing space and newly opened space. It is usually necessary to splice and fuse local spatial data with historical global maps to obtain a more accurate target map. However, in the face of complex and changing work scenarios, the fusion of 3D maps is difficult and it is difficult to ensure its accuracy. Summary of the invention
[0003] The embodiments of the present application provide a map fusion method, apparatus, device, readable storage medium and program product to solve the technical problem that map fusion is difficult and difficult to ensure its accuracy.
[0004] In a first aspect, an embodiment of the present application provides a map fusion method, the method comprising:
[0005] Obtain the observed point cloud data of the target scene in the current time series;
[0006] According to the observed point cloud data in the current time series, an occupancy grid map of the target scene in the current time series is constructed, wherein the occupancy grid map includes a probability value of each grid, and the probability value is determined by the state of each grid, and the state includes an occupied state, an idle state, or an unknown state;
[0007] According to the state and probability value of each grid in the occupancy grid map of the target scene in the current time series, and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series, the target occupancy grid map of the target scene is obtained by fusion.
[0008] In a second aspect, an embodiment of the present application provides a map fusion device, the device comprising:
[0009] The acquisition module is used to obtain the observation point cloud data of the target scene in the current time series;
[0010] A construction module is used to construct an occupancy grid map of the target scene at the current time sequence according to the observed point cloud data at the current time sequence, wherein the occupancy grid map includes a probability value of each grid, and the probability value is determined by the state of each grid, and the state includes an occupied state, an idle state, or an unknown state;
[0011] The fusion module is used to obtain the target occupancy grid map of the target scene according to the state and probability value of each grid in the occupancy grid map of the target scene in the current time series and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series.
[0012] In a third aspect, an embodiment of the present application provides a map fusion device, the device comprising:
[0013] A processor and a memory storing programs or instructions;
[0014] The above method is implemented when the processor executes the program or instruction.
[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and the program or instruction implements the above method when executed by a processor.
[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the above method.
[0017] The map fusion method, apparatus, device, readable storage medium and program product of the embodiments of the present application can obtain the observed point cloud data of the target scene in the current time series; construct an occupancy grid map of the target scene in the current time series based on the observed point cloud data in the current time series, wherein the occupancy grid map includes a probability value for each grid, and the probability value is determined by the state of each grid, and the state includes an occupied state, an idle state or an unknown state; based on the state and probability value of each grid in the occupancy grid map of the target scene in the current time series, and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series, a target occupancy grid map of the target scene is obtained by fusion.
[0018] In this way, we can construct a multi-time series occupancy grid map, and according to the state of each grid, we can clearly obtain whether a grid in the occupancy grid map is a newly opened space or a changed space, and then fuse it according to its probability value to obtain the target occupancy grid map, so as to achieve the purpose of multi-time series map fusion. The multi-time series 3D map fusion problem is converted into the fusion problem of multi-time series probability maps, which reduces the difficulty of map fusion and ensures the accuracy of map fusion. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0020] Figure 1 is a flowchart of a map fusion method provided by an embodiment of the present application;
[0021] Figure 2 is a schematic diagram of constructing an occupancy grid map in the map fusion method provided in an embodiment of the present application;
[0022] Figure 3 is a schematic diagram of a first grid and a second grid in a map fusion method provided in an embodiment of the present application;
[0023] Figure 4 is a structural schematic diagram of a map fusion device provided by another embodiment of the present application;
[0024] Figure 5 It is a structural schematic diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION
[0025] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0026] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0027] In order to solve the problems of the prior art, the embodiments of the present application provide a map fusion method, device, equipment, readable storage medium and program product. The map fusion method provided by the embodiments of the present application is first introduced below.
[0028] Figure 1A flowchart of a map fusion method provided by an embodiment of the present application is shown. The map fusion method can be applied to any scenario that requires real-time map fusion and updating, for example, it can be applied to an open-pit mine working scenario, which is not specifically limited here; and in order to simplify the description, the following mainly takes the open-pit mine working scenario as an example for description.
[0029] like Figure 1 As shown, the map fusion method may include the following steps:
[0030] Step 101, obtaining the observed point cloud data of the target scene in the current time sequence.
[0031] In step 101, the target scene can be any scene that requires map fusion and updating, such as an open-pit mine working scene. In the target scene, multiple sensors can be arranged to collect observation point cloud data in the target scene. The sensor can include a perception sensor and a motion sensor. The perception sensor can include a laser radar, a monocular or multi-eye camera, a depth camera, a time of flight (TOF) camera, a millimeter wave radar, an ultrasonic radar, etc. The motion sensor can include an inertial measurement unit (IMU), an integrated navigation system (INS), etc. Among them, the perception sensor is responsible for measuring the space, and the motion sensor is responsible for restoring the motion of the perception sensor in order to restore the true 3D space.
[0032] The sensor can observe the target scene at a preset frequency within a continuous time sequence. Based on this, the observation point cloud data of the target scene in the current time sequence can be obtained based on the sensor. The observation point cloud data can include the global pose of the target scene, and the observation point cloud data can be transformed into a global coordinate system.
[0033] Step 102, constructing an occupancy grid map of the target scene at the current time sequence based on the observed point cloud data at the current time sequence, wherein the occupancy grid map includes a probability value of each grid, and the probability value is determined by the state of each grid, and the state includes an occupied state, an idle state, or an unknown state.
[0034] In step 102, the occupancy grid map is also called a probability map, which can divide the target scene into multiple grids, and each grid may exist in one of the following three states, namely, occupied state (occupied), free state (free) or unknown state (unmapped). Among them, the occupied state means that there is a target in the grid, the free state means that there is no target in the grid, and the unknown state means that the grid has not been observed, so it is not described whether there is a target. These states can be described by probability values. For example, the probability value corresponding to the unknown state can be 0.5, the probability value corresponding to the occupied state can be 0.5-1.0, and the probability value corresponding to the free state can be 0-0.5. It can be understood that the probability range of the state can be set according to actual needs. The above probability value range is only used as an example for illustration and is not specifically limited here.
[0035] The occupancy grid map of the target scene in the current time series can be constructed based on the observed point cloud data in the current time series. For example, the expression of the probability value of t observations of each grid is shown in formula (1):
[0036]
[0037] Among them, P(n|z 1：t ) represents the probability value of 1 to t observations of grid n, P(n|z t ) represents the probability value of the tth observation of grid n, P(n|z 1：t-1 ) represents the probability value of 1 to t-1 observations of grid n, and P(n) represents the probability prior of grid n.
[0038] Formula (1) can be transformed to obtain the logarithmic expression of the probability value of grid n. The logarithmic expression of the probability value is shown in formula (2):
[0039] Log(n|z 1：t )=Log(n|z 1：t-1 )+Log(n|z t )-Log(n) (2)
[0040] In formula (2), It can be understood that the logarithmic expression of probability values is a reversible operation, that is, P(*) can be obtained from Log(*), and Log(*) can be obtained from P(*). In this way, the complex probability update problem of multiple observations of grid n in formula (1) can be described as the addition problem of logarithmic probabilities in formula (2).
[0041] It can also be understood that in order to balance the efficiency of updating the local spatial probability value, the present application can also define a threshold value of the logarithmic probability of grid n. The expression of the threshold value can be shown as formula (3):
[0042] Log(n|z 1：t )=max(min(Log(n|z 1：t-1 )+Log(n|z t ), Log max ), -Log max ).(3)
[0043] Among them, Log max Represents the maximum value of the log-probability of grid n.
[0044] Step 103, according to the state and probability value of each grid in the occupancy grid map of the target scene in the current time series, and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series, a target occupancy grid map of the target scene is obtained by fusing.
[0045] In step 103, the construction method of the occupancy grid map of the target scene in the historical time series is consistent with the construction method of the occupancy grid map of the target scene in the current time series, which will not be described in detail here.
[0046] The target occupancy grid map of the target scene can be obtained by fusing the state and probability value of each grid in the occupancy grid map of the target scene in the current time series and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series.
[0047] For example, taking any grid as an example, if the state of the grid in the occupancy grid map in the current time series and the state of the grid in the occupancy grid map in the historical time series are both occupied or idle, the logarithmic probability addition method can be used to fuse and obtain the probability value of the grid in the fused target occupancy grid map. If the state of the grid in the occupancy grid map in the historical time series is an unknown state, the logarithmic probability corresponding to the state of the grid in the occupancy grid map in the current time series can be directly determined as the probability value of the grid in the fused target occupancy grid map. If the state of the grid in the occupancy grid map in the current time series is an unknown state, the logarithmic probability corresponding to the state of the grid in the occupancy grid map in the historical time series can be selected to retain or eliminate according to the actual situation.
[0048] In an embodiment of the present application, a map fusion method can obtain observed point cloud data of the target scene in the current time series; based on the observed point cloud data in the current time series, an occupancy grid map of the target scene in the current time series is constructed, wherein the occupancy grid map includes a probability value for each grid, and the probability value is determined by the state of each grid, and the state includes an occupied state, an idle state, or an unknown state; based on the state and probability value of each grid in the occupancy grid map of the target scene in the current time series, and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series, a target occupancy grid map of the target scene is obtained by fusion.
[0049] In this way, we can construct a multi-time series occupancy grid map, and according to the state of each grid, we can clearly obtain whether a grid in the occupancy grid map is a newly opened space or a changed space, and then fuse it according to its probability value to obtain the target occupancy grid map, so as to achieve the purpose of multi-time series map fusion. The multi-time series 3D map fusion problem is converted into the fusion problem of multi-time series probability maps, which reduces the difficulty of map fusion and ensures the accuracy of map fusion.
[0050] In some embodiments, the above step 102 may include the following steps:
[0051] According to the observed point cloud data in the current time series, determine the N grids that the observation ray corresponding to the i-th moment passes through, as well as the states of the N grids, where N and i are positive integers;
[0052] According to the states of N grids, determine the probability values corresponding to the N grids at the i-th moment;
[0053] According to the probability values corresponding to the i-th moment of the N grids, the probability values of the N grids at the current time sequence are updated to obtain the occupancy grid map of the target scene at the current time sequence.
[0054] It is understandable that a continuous time series may include multiple moments, that is, the current time series may include multiple moments. The sensor may perform multiple observations at each moment. For a certain grid, one of the observation data may be selected to determine its state and probability value at a certain moment. The logarithmic probability of each grid may be updated by cast-ray, thereby obtaining the occupancy grid map of the target scene at the current time series.
[0055] To facilitate understanding of the solution of this embodiment, the logarithmic probability Log(n|z t ) is determined to be -1, and the logarithmic probability corresponding to the occupied state Log(n|z t ) is determined to be 1, and the logarithmic probability Log(n|z t ) is determined to be 0.
[0056] like Figure 2 As shown in the figure, at the i-th moment, the sensor can observe the grid P from A, that is, the observation ray corresponding to the i-th moment can be ray AP, and ray AP passes through 4 grids. At this time, the states of these 4 grids can be determined according to the data observed by the sensor, thereby determining the probability values corresponding to the 4 grids. The states of the remaining grids that are passed can be considered to be unknown, that is, the probability value is 0.
[0057] At the i+1th moment, the sensor can move from A to C, and the grid P can be observed from C. That is, the observation ray corresponding to the i+1th moment can be the ray CP, and the ray CP also passes through 4 grids. At this time, the status of the 4 grids can be determined based on the data observed by the sensor, and the probability value can be updated based on the probability value of each grid determined at the original i-th moment. This step is iterated until the latest occupancy grid map is output, which is the occupancy grid map of the final target scene at the current time sequence.
[0058] In this way, the complex probability update problem can be described as the logarithmic probability addition problem through the ray method, which reduces the difficulty of constructing the occupancy grid map while ensuring the construction accuracy of the occupancy grid map.
[0059] In some embodiments, determining the probability values corresponding to the N grids at the i-th moment according to the states of the N grids may include:
[0060] When the first grid is included in the N grids, the probability value corresponding to the second grid at the i-th moment is determined to be 0, wherein the N grids are traversed in the order from the starting point to the end point of the observation ray, the first grid is the first grid whose state is the occupied state, and the second grid is the other grids located after the first grid.
[0061] It is understandable that, as mentioned above, the probability update of the occupancy grid map is essentially implemented by the cast-ray method. When the top-down angle is small, high-frequency illegal observations of the ground may be generated, which may lead to inaccurate results. Figure 3 As shown, the sensor observes grid B from P, and the corresponding observation ray is ray PB. At this time, among the multiple grids passed by ray PB, according to the order from the starting point to the end point of the observation ray, if the state of grid A is first observed to be occupied (that is, grid A is the first grid), it can be considered that there is a target in grid A. At this time, it is impossible to penetrate the target and observe other grids after grid A.
[0062] In other words, among the multiple grids in the path of ray PB, the other grids after grid A (i.e., the second grid) should not be observed at this time, that is, the state of the second grid can be considered as an unknown state at this time, and its corresponding probability value is 0. In this case, the probability values of the grids between AB on ray PB are not updated.
[0063] This can greatly reduce the frequency of ground points being described as idle, that is, reduce the probability of ground points being described as idle, and effectively improve the accuracy of the occupancy grid map of the target scene in the current time sequence.
[0064] In some embodiments, the above step 103 may include the following steps:
[0065] When the state of the third grid in the current time series and the state of the third grid in the historical time series are both occupied or idle, determining the target probability value of the third grid in the target occupied grid map according to the probability value of the third grid in the current time series and the probability value of the third grid in the historical time series;
[0066] The third grid is any grid in the occupancy grid map of the target scene.
[0067] It is understandable that the map fusion problem can involve two aspects: one is the probability update of the newly opened space, and the other is the probability update of the changed space. For the probability update of the newly opened space, it is sufficient to directly retain the probability value of the occupied grid map under the current time series, while for the probability update of the changed space, it is necessary to consider the two situations of recoverable spatial change and irreversible spatial change. For example, the movement of the mining truck is an overall movement, which will not cause damage to the space, and the impact of its movement on the space is a recoverable spatial change. After the blasting of the mining face, there will be a lot of falling rocks. During the operation of the excavator, some targets in the space where the ore is located may be removed by the operation, and some targets will naturally roll down with the operation, thus blocking the historically observable targets. Therefore, the change of the mining face is an irreversible spatial change.
[0068] In order to be compatible with the probability update of the newly opened space, the probability update of the recoverable space change, and the more complex probability update problem of the irreversible space change, in this embodiment, the probability fusion of any third grid in the occupancy grid map under the current time series and the historical time series can be divided into 9 possibilities according to its state, as shown in Table 1:
[0069]
[0070] In Table 1, the state of the third grid when the occupancy grid maps of two time series are fused is represented by two letters, the first letter represents the state of the third grid in the occupancy grid map in the current time series, and the second letter represents the state of the third grid in the occupancy grid map in the historical time series. Taking OU as an example, it indicates that the state of the third grid in the occupancy grid map in the current time series is occupied, and the state in the occupancy grid map in the historical time series is unknown.
[0071] In this embodiment, when the state of the third grid in the current time sequence and the state of the third grid in the historical time sequence are both occupied or idle, that is, in the four cases of OO, OF, FO and FF, the target probability value of the third grid in the target occupied grid map can be determined according to the probability value of the third grid in the current time sequence and the probability value of the third grid in the historical time sequence. In other words, the probability value can be directly fused according to formula (2) to calculate the target probability value of the third grid in the target occupied grid map.
[0072] In this way, the multi-time series 3D map fusion problem can be transformed into the multi-time series probability map fusion problem, which reduces the difficulty of map fusion and ensures the accuracy of map fusion.
[0073] In some embodiments, the above step 103 may include the following steps:
[0074] When the state of the third grid in the historical time series is an unknown state, the probability value of the third grid in the current time series is determined as the target probability value of the third grid in the target occupancy grid map;
[0075] The third grid is any grid in the occupancy grid map of the target scene.
[0076] In this embodiment, as mentioned above, when the state of the third grid in the historical time series is an unknown state, that is, in the three cases of OU, FU and UU, the third grid can be considered to be a newly opened space, because the occupancy grid map in the historical time series does not describe the third grid. At this time, the occupancy grid map in the current time series can be directly used to describe the fused third grid, that is, the probability value of the third grid in the current time series is determined as the target probability value of the third grid in the target occupancy grid map.
[0077] In some embodiments, the above step 103 may include the following steps:
[0078] When the state of the third grid in the current time series is an unknown state, and the state of the third grid in the historical time series is an occupied state or an idle state, the historical observation ray corresponding to the historical time series is used to traverse the third grid in the occupied grid map in the current time series;
[0079] In the case that there is a fourth grid in an occupied state among the M grids passed by the historical observation ray, determining that the target probability value of the third grid in the target occupied grid map is 0;
[0080] The third grid is any grid in the occupancy grid map of the target scene, the fourth grid is different from the third grid, and M is a positive integer.
[0081] In this embodiment, if the state of the third grid in the current time sequence is unknown, and the state of the third grid in the historical time sequence is occupied or idle, that is, in the two cases of UO and UF, it is necessary to determine why the occupation grid map in the current time sequence cannot describe the third grid, resulting in the state of the third grid in the occupation grid map in the current time sequence being unknown. This can be divided into two cases: one is that the third grid is not observed, and the other is that the third grid is attempted to be observed but is not observed.
[0082] In order to distinguish the above two situations, based on the assumption that if the occupancy grid map does not change, the same space can be observed using the same ray at any time, the recast method can be used to identify the reason why the occupancy grid map at the current time series cannot describe the third grid, that is, the historical observation ray corresponding to the historical time series can be used in the occupancy grid map at the current time series to traverse the third grid.
[0083] If there is a fourth grid in the occupied state among the M grids passed by the historical observation ray, in other words, before the historical observation ray observes the third grid, the fourth grid with the target is observed. At this time, since it is impossible to penetrate the target and observe the space behind the target, it can be considered that the third grid is blocked. At this time, the blocked space can be eliminated in the fusion result, and the target probability value of the third grid in the target occupied grid map can be determined to be 0.
[0084] In this way, the recast method is used to identify the reasons why the occupancy grid map in the current time series does not find some spaces described by the occupancy grid map in the historical time series, and then the corresponding fusion strategy is executed according to the specific reasons, which further improves the accuracy of map fusion.
[0085] In some embodiments, when the state of the third grid in the current time sequence is an unknown state, and the state of the third grid in the historical time sequence is an occupied state or an idle state, after traversing the third grid using the historical observation ray corresponding to the historical time sequence in the occupied grid map in the current time sequence, the map fusion method may further include the following steps:
[0086] When the state of the fifth grid among the M grids passed by the historical observation ray is all in an idle state, the probability value of the third grid in the historical time series is determined as the target probability value of the third grid in the target occupancy grid map;
[0087] The fifth grid is the other grids among the M grids except the third grid.
[0088] In this embodiment, as mentioned above, the recast method can be used to identify why the occupancy grid map at the current time sequence cannot describe the third grid, that is, the historical observation ray corresponding to the historical time sequence can be used in the occupancy grid map at the current time sequence to traverse the third grid.
[0089] If the states of all the fifth grids except the third grid among the M grids passed by the historical observation ray are idle, it can be considered that the third grid is not blocked. In other words, the state of the third grid is unknown because it has not been observed. At this time, the state of the third grid in the occupancy grid map under the historical time series can be retained, that is, the probability value of the third grid under the historical time series can be determined as the target probability value of the third grid in the target occupancy grid map.
[0090] In this way, the recast method is used to identify the reasons why the occupancy grid map in the current time series does not find some spaces described by the occupancy grid map in the historical time series, and then the corresponding fusion strategy is executed according to the specific reasons, which further improves the accuracy of map fusion.
[0091] Based on the map fusion method provided in the above embodiment, the present application also provides an embodiment of a map fusion device.
[0092] Figure 4 A schematic diagram of the structure of a map fusion device provided in another embodiment of the present application is shown. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.
[0093] Reference Figure 4 , the map fusion device 400 may include:
[0094] The acquisition module 401 is used to acquire the observation point cloud data of the target scene in the current time sequence;
[0095] A construction module 402 is used to construct an occupancy grid map of the target scene at the current time sequence according to the observed point cloud data at the current time sequence, wherein the occupancy grid map includes a probability value of each grid, and the probability value is determined by the state of each grid, and the state includes an occupied state, an idle state, or an unknown state;
[0096] The fusion module 403 is used to fuse the state and probability value of each grid in the occupancy grid map of the target scene in the current time series and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time series to obtain the target occupancy grid map of the target scene.
[0097] In some embodiments, building blocks 402 may include:
[0098] A first determination unit is used to determine N grids that the observation ray corresponding to the i-th moment passes through and the states of the N grids according to the observation point cloud data in the current time sequence, where N and i are positive integers;
[0099] A second determining unit, used to determine the probability values corresponding to the N grids at the i-th moment according to the states of the N grids;
[0100] The updating unit is used to update the probability values of the N grids at the current time sequence according to the probability values corresponding to the i-th moment of the N grids, so as to obtain the occupancy grid map of the target scene at the current time sequence.
[0101] In some embodiments, the second determining unit may be used to:
[0102] When the first grid is included in the N grids, the probability value corresponding to the second grid at the i-th moment is determined to be 0, wherein the N grids are traversed in the order from the starting point to the end point of the observation ray, the first grid is the first grid whose state is the occupied state, and the second grid is the other grids located after the first grid.
[0103] In some embodiments, the fusion module 403 may also be used to:
[0104] When the state of the third grid in the current time series and the state of the third grid in the historical time series are both occupied or idle, determining the target probability value of the third grid in the target occupied grid map according to the probability value of the third grid in the current time series and the probability value of the third grid in the historical time series;
[0105] The third grid is any grid in the occupancy grid map of the target scene.
[0106] In some embodiments, the fusion module 403 may also be used to:
[0107] When the state of the third grid in the historical time series is an unknown state, the probability value of the third grid in the current time series is determined as the target probability value of the third grid in the target occupancy grid map;
[0108] The third grid is any grid in the occupancy grid map of the target scene.
[0109] In some embodiments, the fusion module 403 may also be used to:
[0110] When the state of the third grid in the current time series is an unknown state, and the state of the third grid in the historical time series is an occupied state or an idle state, the historical observation ray corresponding to the historical time series is used to traverse the third grid in the occupied grid map in the current time series;
[0111] In the case that there is a fourth grid in an occupied state among the M grids passed by the historical observation ray, determining that the target probability value of the third grid in the target occupied grid map is 0;
[0112] The third grid is any grid in the occupancy grid map of the target scene, the fourth grid is different from the third grid, and M is a positive integer.
[0113] In some embodiments, the fusion module 403 may also be used to:
[0114] When the state of the fifth grid among the M grids passed by the historical observation ray is all in an idle state, the probability value of the third grid in the historical time series is determined as the target probability value of the third grid in the target occupancy grid map;
[0115] The fifth grid is the other grids among the M grids except the third grid.
[0116] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the above-mentioned map fusion method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device. Its specific functions and technical effects brought about can be found in the method embodiment part, which will not be repeated here.
[0117] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0118] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in yet another embodiment of the present application is shown.
[0119] The device may include a processor 501 and a memory 502 storing programs or instructions.
[0120] When the processor 501 executes the program, the steps in any of the above method embodiments are implemented.
[0121] Exemplarily, the program may be divided into one or more modules / units, one or more modules / units are stored in the memory 502, and executed by the processor 501 to complete the present application. One or more modules / units may be a series of program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the program in the device.
[0122] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0123] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.
[0124] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) machine-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.
[0125] The processor 501 implements any one of the methods in the above embodiments by reading and executing the program or instruction stored in the memory 502 .
[0126] In one example, the electronic device may further include a communication interface 503 and a bus 504. The processor 501, the memory 502, and the communication interface 503 are connected via the bus 504 and communicate with each other.
[0127] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0128] Bus 504 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 504 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.
[0129] In addition, in combination with the method in the above embodiment, the embodiment of the present application may provide a machine-readable storage medium for implementation. The machine-readable storage medium stores a program or instruction; when the program or instruction is executed by a processor, any one of the methods in the above embodiment is implemented. The machine-readable storage medium can be read by a machine such as a computer.
[0130] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0131] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0132] An embodiment of the present application provides a computer program product, which is stored in a machine-readable storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0133] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0134] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.
[0135] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0136] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by a computer program or instruction. These programs or instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0137] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A map fusion method, characterized in that: include: Obtain the observed point cloud data of the target scene in the current time series; Constructing an occupancy grid map of the target scene at the current time sequence according to the observed point cloud data at the current time sequence, wherein the occupancy grid map includes a probability value of each grid, and the probability value is determined by a state of each grid, and the state includes an occupied state, an idle state, or an unknown state; According to the state and probability value of each grid in the occupancy grid map of the target scene in the current time sequence, and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time sequence, a target occupancy grid map of the target scene is obtained by fusion.
2. The method according to claim 1, characterized in that The current time sequence includes multiple moments, and constructing an occupancy grid map of the target scene in the current time sequence according to the observed point cloud data in the current time sequence includes: According to the observed point cloud data in the current time sequence, determine the N grids that the observation ray corresponding to the i-th moment passes through, and the states of the N grids, where N and i are positive integers; Determine the probability value corresponding to the N grids at the i-th moment according to the states of the N grids; According to the probability values corresponding to the i-th moment of the N grids, the probability values of the N grids at the current time sequence are updated to obtain an occupied grid map of the target scene at the current time sequence.
3. The method according to claim 2, characterized in that The determining, according to the states of the N grids, probability values corresponding to the N grids at the i-th moment includes: In the case where the N grids include the first grid, it is determined that the probability value corresponding to the second grid at the i-th moment is 0, wherein the N grids are traversed in order from the starting point to the end point of the observation ray, the first grid is the first grid whose state is an occupied state, and the second grid is other grids located after the first grid.
4. The method according to claim 1, characterized in that: The target occupancy grid map of the target scene is obtained by fusing the state and probability value of each grid in the occupancy grid map of the target scene in the current time sequence and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time sequence, including: When the state of the third grid in the current time sequence and the state of the third grid in the historical time sequence are both occupied or idle, determining the target probability value of the third grid in the target occupied grid map according to the probability value of the third grid in the current time sequence and the probability value of the third grid in the historical time sequence; The third grid is any grid in the occupancy grid map of the target scene.
5. The method according to claim 1, characterized in that The target occupancy grid map of the target scene is obtained by fusing the state and probability value of each grid in the occupancy grid map of the target scene in the current time sequence and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time sequence, including: When the state of the third grid in the historical time sequence is an unknown state, determining the probability value of the third grid in the current time sequence as the target probability value of the third grid in the target occupancy grid map; The third grid is any grid in the occupancy grid map of the target scene.
6. The method according to claim 1, characterized in that The target occupancy grid map of the target scene is obtained by fusing the state and probability value of each grid in the occupancy grid map of the target scene in the current time sequence and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time sequence, including: When the state of the third grid in the current time sequence is an unknown state, and the state of the third grid in the historical time sequence is an occupied state or an idle state, the third grid is traversed by using the historical observation ray corresponding to the historical time sequence in the occupied grid map in the current time sequence; In the case that there is a fourth grid in an occupied state among the M grids passed by the historical observation ray, determining that the target probability value of the third grid in the target occupied grid map is 0; The third grid is any grid in the occupancy grid map of the target scene, the fourth grid is different from the third grid, and M is a positive integer.
7. The method according to claim 6, characterized in that When the state of the third grid in the current time sequence is an unknown state, and the state of the third grid in the historical time sequence is an occupied state or an idle state, after traversing the third grid using the historical observation ray corresponding to the historical time sequence in the occupied grid map in the current time sequence, the method further includes: When the states of the fifth grids among the M grids passed by the historical observation ray are all in an idle state, the probability value of the third grid in the historical time sequence is determined as the target probability value of the third grid in the target occupancy grid map; The fifth grid is the other grids among the M grids except the third grid.
8. A map fusion device, characterized in that: include: The acquisition module is used to obtain the observation point cloud data of the target scene in the current time series; A construction module, configured to construct an occupancy grid map of the target scene at the current time sequence according to the observed point cloud data at the current time sequence, wherein the occupancy grid map includes a probability value of each grid, and the probability value is determined by a state of each grid, and the state includes an occupied state, an idle state, or an unknown state; A fusion module is used to fuse the state and probability value of each grid in the occupancy grid map of the target scene in the current time sequence and the state and probability value of each grid in the occupancy grid map of the target scene in the historical time sequence to obtain the target occupancy grid map of the target scene.
9. An electronic device, characterized in that: The device comprises: a processor and a memory storing programs or instructions; When the processor executes the program or instruction, the method according to any one of claims 1 to 7 is implemented.
10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
11. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method according to any one of claims 1 to 7.