Data processing method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention provides a data processing method and device, electronic equipment and a computer readable medium. The method comprises the steps of determining at least one missing part of point cloud data based on a plurality of static obstacle region sets corresponding to multiple acquisition processes; determining at least one candidate region corresponding to the at least one portion among the plurality of candidate regions; determining a road boundary point cloud corresponding to the boundary of the road in the point cloud map; selecting at least one to-be-repaired area from the at least one candidate area based on the road boundary point cloud; expanding a first to-be-repaired area in the at least one to-be-repaired area to obtain a first expanded area; determining a middle area between the first expansion area and the first to-be-repaired area as a reference area corresponding to the first to-be-repaired area; and determining missing point cloud data in the first to-be-repaired area by using point cloud data of a reference area associated with the first to-be-repaired area in the point cloud map.

Description

Description of the case

[0001] The present disclosure is a divisional application of Chinese patent application CN 202011232976.3, entitled “Data processing method, device, electronic device and computer-readable medium”, filed on November 6, 2020. Technical Field

[0002] Embodiments of the present disclosure generally relate to data processing and high-precision map production, and more particularly to a data processing method, a data processing device, an electronic device, and a computer-readable medium. Background Art

[0003] High-precision maps are machine-oriented digital maps that can be used in application scenarios such as autonomous driving, robot navigation and positioning. High-precision maps play an important role in autonomous driving systems. Whether it is the environment perception unit, path planning unit, or vehicle positioning unit, they all rely on high-precision maps to work to varying degrees. Point cloud maps are one of the foundations and an important component of high-precision map production. They use dense point clouds to simulate the real environment, and special point clouds can be marked with labels to reflect special signs in the real environment.

[0004] Generally, in order to obtain a point cloud map of a target area, a laser scanning measurement method can be used to collect point cloud data of the target area. Then, the collected point cloud data can be processed and processed to generate a point cloud map of the target area. During the collection process, part of the point cloud data of the target area may not be collected due to various reasons, which results in the existence of an unrepaired area in the point cloud map that needs to repair the point cloud data. However, in the traditional solution, there are still various defects in the way of determining and processing the unrepaired area of ​​the point cloud map that need to be optimized. Summary of the invention

[0005] The embodiments of the present disclosure propose a technical solution for determining and repairing an area to be repaired in a point cloud map, and specifically provide a data processing method, a data processing device, an electronic device, and a computer-readable medium.

[0006] In a first aspect of the present disclosure, a data processing method is provided. The method includes: determining multiple candidate areas from a target area based on static obstacle information in multiple acquisition processes, and the multiple acquisition processes are used to acquire point cloud data for constructing a point cloud map of the target area. The method also includes: determining at least one area to be repaired from multiple candidate areas based on multiple parts corresponding to the multiple candidate areas in the point cloud map. The method further includes: determining the missing point cloud data in at least one area to be repaired using the point cloud data of a reference area associated with at least one area to be repaired in the point cloud map.

[0007] In a second aspect of the present disclosure, a data processing device is provided. The device includes: a candidate area determination module, configured to determine multiple candidate areas from a target area based on static obstacle information in multiple acquisition processes, and the multiple acquisition processes are used to collect point cloud data for constructing a point cloud map of the target area. The device also includes: a candidate area determination module, configured to determine multiple candidate areas from a target area based on static obstacle information in multiple acquisition processes, and the multiple acquisition processes are used to collect point cloud data for constructing a point cloud map of the target area. The device further includes: a missing point cloud data determination module, configured to determine the missing point cloud data in at least one area to be repaired using the point cloud data of a reference area associated with at least one area to be repaired in the point cloud map.

[0008] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes one or more processors and a memory. The memory is used to store computer executable instructions. The computer executable instructions are executed by one or more processors to implement the method according to the first aspect of the present disclosure.

[0009] In a fourth aspect of the present disclosure, a computer-readable medium is provided, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0010] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner.

[0012] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure may be implemented is shown.

[0013] Figure 2 A flowchart of an exemplary data processing method according to an embodiment of the present disclosure is shown.

[0014] FIG. 3A to FIG. 3D A schematic diagram showing a plurality of static obstacle area sets associated with multiple acquisition processes according to an embodiment of the present disclosure is shown.

[0015] Figure 3E A schematic diagram showing the intersection of multiple static obstacle area sets according to an embodiment of the present disclosure is shown.

[0016] Figure 4 An example process for determining one or more areas to be repaired according to an embodiment of the present disclosure is shown.

[0017] Figure 5 A schematic diagram showing multiple corresponding parts of multiple candidate areas in a point cloud map according to an embodiment of the present disclosure is shown.

[0018] Fig. 6A A schematic diagram showing the intersection of a candidate area and a road boundary line according to an embodiment of the present disclosure is shown.

[0019] Figure 6B A schematic diagram showing the intersection of a corresponding portion of a candidate area in a point cloud map with a road boundary point cloud according to an embodiment of the present disclosure.

[0020] Figure 7 A schematic diagram showing a method of using an intermediate area between an extended area and an area to be repaired as a reference area according to an embodiment of the present disclosure.

[0021] Figure 8 A block diagram of an example data processing apparatus according to an embodiment of the present disclosure is shown.

[0022] Fig. 9 A block diagram is shown of an example device that may be used to implement embodiments of the present disclosure.

[0023] Throughout the drawings, the same or similar reference numbers are used to designate the same or similar components. DETAILED DESCRIPTION

[0024] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments shown in the accompanying drawings. It should be understood that these specific embodiments are described only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way.

[0025] As pointed out above, in the traditional scheme, there are still various defects in the way of determining and processing the area to be repaired of the point cloud map that need to be optimized. Specifically, in the process of collecting point cloud data of the target area, there may be static obstacles in the target area. Such static obstacles may block the area where the point cloud data needs to be collected, resulting in the inability to collect the desired point cloud data. For example, in the point cloud data collection for the road surface, facilities that are stationary relative to the road and vehicles parked on the road surface may block the road surface, resulting in the inability to obtain the point cloud data and real road surface information of the road surface. In the case of static obstacle occlusion, the point cloud data collected by laser scanning is actually the point cloud data of the static obstacle, rather than the point cloud data of the desired area. Therefore, in the subsequent processing and processing of the point cloud data, it may be necessary to remove the point cloud data of the static obstacle from the collected point cloud data, which will result in "holes" in the subsequently generated point cloud map, that is, the point cloud data of the desired area is missing in the point cloud map.

[0026] Incomplete point cloud maps may reduce the performance of various applications that use point cloud maps, and in some cases may make the purpose of these applications impossible to achieve. Therefore, in the process of making point cloud maps, it may be necessary to repair the point cloud data of the area to be repaired in the point cloud map. For example, for the point cloud map of the road environment, due to the characteristics of the road conditions, in the event of occlusion by static obstacles, the point cloud data obtained by lidar scanning will not be able to record the height information of the actual road surface. When applied to autonomous driving, the road height information in the point cloud map is crucial for detecting road obstacles that affect vehicle driving. Therefore, in the process of making point cloud maps of road environments, it may be necessary to deal with the situation where static obstacles (for example, vehicles parked on the roadside) block the road surface.

[0027] In traditional solutions, since the collected point cloud data involves a large geographical range, the operator may need to manually mark the edges of the area to be repaired in the point cloud map. Then, for the marked area to be repaired, the computing device can generate patch point cloud data for adding to the point cloud map. However, such traditional solutions have shortcomings in processing efficiency, accuracy, and automation level.

[0028] In view of the above-mentioned problems and other potential problems existing in the traditional solutions, the embodiments of the present disclosure provide a technical solution for determining and repairing the area to be repaired in the point cloud map. In this technical solution, the computing device can determine multiple candidate areas of the area to be repaired from the target area based on the static obstacle information in multiple acquisition processes. Then, based on the multiple parts in the point cloud map corresponding to the multiple candidate areas, the computing device can determine one or more areas to be repaired from the candidate areas. Then, the computing device can use the point cloud data of the reference area associated with the area to be repaired in the point cloud map to determine the missing point cloud data in the area to be repaired. In the technical solution of the present disclosure, the computing device can automatically use the static obstacle information to determine the area to be repaired in the point cloud map, and then can supplement the point cloud data of the area to be repaired based on the point cloud data of the reference area, thereby improving the efficiency, accuracy and automation level of point cloud map repair.

[0029] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented. Figure 1 As shown, the example environment 100 may include a collection device 110 that can collect point cloud data 121 of a target area 105 to create a point cloud map 123 of the target area 105. For example, the collection device 110 may use a laser radar measurement device to emit lasers 115 to various objects in the target area 105, and then perform relevant measurements and calculations based on the received lasers reflected from the objects, thereby obtaining point cloud data 121 associated with the target area 105. Figure 1 In the example, the acquisition device 110 is depicted as a collection vehicle equipped with a laser radar measurement device, but this depiction is only schematic and is not intended to limit the scope of the present disclosure in any way. In other embodiments, the acquisition device 110 may have any other possible form. For example, the acquisition device 110 may be other vehicles (such as civilian motor vehicles, electric vehicles, aircraft, etc.) carrying a laser radar measurement device or a laser radar measurement device carried manually. In addition, the laser radar measurement device may also be replaced by any other type of point cloud data acquisition device. In other words, the acquisition device 110 of the embodiment of the present disclosure may be any device with point cloud data acquisition capabilities.

[0030] As used herein, the data collected by the acquisition device 110 through the laser radar measurement device may be referred to as "point cloud data". Generally, "point cloud data" may refer to the data information of each point on the surface of an object or object returned when a laser beam is irradiated on the surface of the object or object, which may include the coordinate information of each point (for example, the coordinate value in a three-dimensional coordinate system), and may also include the laser reflection intensity (also called "reflection value"). Therefore, in the context of the present disclosure, "point cloud data" may generally refer to data associated with points in space, for example, point coordinate data in space. In some cases, "point cloud data" may also include other aspects of information associated with points in space, such as color information or reflection intensity information. In an embodiment of the present disclosure, "point cloud data" may also refer to any spatial point data that can be used to make a point cloud map or a high-precision map.

[0031] In the context of this disclosure, high-precision Figure 1 Generally refers to an electronic map with high-precision data. For example, the high precision here refers to the higher absolute coordinate accuracy of the high-precision electronic map. Absolute coordinate accuracy refers to the accuracy between a target on the map and things in the real outside world. For another example, the absolute accuracy of a high-precision map is generally at the sub-meter level, that is, the accuracy within 1 meter, and the lateral relative accuracy (for example, the relative position accuracy of lanes and lanes, lanes and lane lines) is often even higher. On the other hand, compared with conventional maps, high-precision maps contain richer and more detailed road traffic information elements. In addition, in some embodiments, the high-precision map not only has high-precision coordinates, but also has accurate road shapes, and the slope, curvature, heading, elevation, and roll data of each lane are also included. In some embodiments, the high-precision map can not only depict the road, but also depict how many lanes there are on a road to truly reflect the actual style of the road.

[0032] exist Figure 1 In the example of FIG. 1 , the target area 105 for collecting point cloud data by the collection device 110 is an area including the road 101. Figure 1 As shown, the road 101 may be defined by road boundary lines 102 and 104, and may be divided into three lanes by lane lines 106 and 108. The acquisition device 110 is depicted as traveling in the middle lane and emitting a laser 115 to acquire point cloud data of the target area 105. However, it should be understood that such a depiction is merely exemplary, and the embodiments of the present disclosure are not limited to the acquisition device 110 being at a specific position in the target area 105, but are equally applicable to the situation where the acquisition device 110 is at any possible position. In addition, it should be noted that Figure 1The depiction of road 101 as having three lanes is merely illustrative and is not intended to limit the scope of the present disclosure in any way. In other embodiments, road 101 may have any number of lanes and road styles.

[0033] exist Figure 1 In the example of , vegetation is also arranged outside the road boundary lines 102 and 104, such as trees 112-1 to 112-6. In addition, traffic auxiliary facilities, such as traffic lights 114, are also arranged outside the road boundary line 102. In addition, during the period when the acquisition device 110 collects point cloud data of the target area 105, the vehicles 116 and 118 are parked on the road 101. That is, the vehicles 116 and 118 are not in a driving state, but remain stationary relative to the road 101. It will be understood that Figure 1 The trees 112-1 to 112-6, the traffic light 114, and the vehicles 116 and 118 depicted in the figure are only for illustration and are not intended to limit the scope of the present disclosure in any way. The target area 105 of the embodiment of the present disclosure may be a road environment including any other objects or facilities, or may be other environments different from the road environment. Figure 1 The specific numbers of various objects or elements depicted in are merely illustrative, and embodiments of the present disclosure are equally applicable to a target area 105 having any number of similar objects or elements.

[0034] like Figure 1 As shown, in order to obtain the point cloud map 123 of the target area 105, the acquisition device 110 can provide the collected point cloud data 121 of the target area 105 to the computing device 120. Then, the computing device 120 can process and process the point cloud data 121 to generate the point cloud map 123 of the target area 105. For example, the computing device 120 can obtain the three-dimensionally reconstructed point cloud map 123 from the point cloud data 121 through a simultaneous localization and mapping (SLAM) method. Of course, the embodiments of the present disclosure are not limited thereto, and the computing device 120 can also use any other available method to generate the point cloud map 123 from the point cloud data 121.

[0035] As mentioned above, when the acquisition device 110 is collecting the point cloud data 121 of the target area 105, static obstacles in the target area 105 may block the area that the acquisition device 110 wishes to collect (for example, the point cloud data of the road surface of the road 101), causing the acquisition device 110 to be unable to collect the point cloud data of the desired area. As used herein, a "static obstacle" may refer to an obstacle that remains stationary relative to the target area 105 when the acquisition device 110 is collecting the point cloud data of the target area 105. This means that a static obstacle may always block the same area. In this regard, a static obstacle may be distinguished from a "dynamic obstacle" that is in motion relative to the target area 105, because as the dynamic obstacle moves relative to the target area 105, the area that was previously blocked by the dynamic obstacle may no longer be blocked by the dynamic obstacle, so that the point cloud data of the area can be collected by the acquisition device 110 (such as through multiple scanning measurements). As an illustrative example, in Figure 1 In the figure, the traffic light 114, the trees 112-1 to 112-6, and the vehicles 116 and 118 can be considered as static obstacles relative to the road 101. In contrast, if the vehicles 116 and 118 are in a driving state on the road 101, they can be considered as dynamic obstacles.

[0036] In some cases, in order to comprehensively collect the point cloud data 121 of the target area 105, or to reduce the impact of dynamic obstacles on a single collection process, the collection device 110 may perform multiple collection processes of point cloud data on the target area 105. In each collection process, the collection device 110 may detect static obstacles in the target area 105, such as by using an environmental perception component or module to detect static obstacles to generate static obstacle information 125. For example, the static obstacle information 125 may include size information and position information of the static obstacle. For another example, the static obstacle information 125 may also include shape information and type information of the static obstacle, and so on. More generally, the static obstacle information 125 may include any information related to the static obstacle. In some embodiments, the static obstacle information 125 may be represented in the form of a detection box surrounding the static obstacle, for example, the detection box may have a cube shape. Of course, in other embodiments, the static obstacle information 125 may be represented in any other appropriate form.

[0037] like Figure 1 As shown, in addition to the point cloud data 121, the acquisition device 110 can also provide the detected static obstacle information 125 to the computing device 120, so that the computing device 120 can perform repair on the point cloud map 123 based on the static obstacle information 125 and the point cloud map 123 to obtain a repaired point cloud map 127. It should be noted that although Figure 1 In the example, the acquisition device 110 provides the static obstacle information 125 to the computing device 120 . However, in other embodiments, the computing device 120 may also obtain the static obstacle information 125 from other static obstacle detection devices other than the acquisition device 110 .

[0038] In some embodiments, in the process of generating a point cloud map 123 based on the point cloud data 121 by the computing device 120, the static obstacle information 125 can be used to remove the point cloud data of the static obstacles in the point cloud data 121 frame by frame. In the context of the present disclosure, "frame" can refer to the point cloud data or point cloud frame formed by the acquisition device 110 performing a point cloud data acquisition process on the target area 105, which can correspond to the point cloud data or point cloud frame formed by the laser radar measurement device completing a 360-degree scan. As explained above, since the point cloud data of the static obstacle is not the point cloud data expected to be collected, the computing device 120 can remove the point cloud data of the static obstacle frame by frame in the point cloud data 121 based on the static obstacle information 125. However, as a result, the point cloud data 121 will lack the point cloud data of the area blocked by the static obstacle, so that the part corresponding to the static obstacle in the generated point cloud map 123 may not have point cloud data, but form a point cloud "hole". Therefore, in an embodiment of the present disclosure, the computing device 120 may also complete the missing point cloud data in the point cloud map 123 based on the static obstacle information 125 and the point cloud map 123 , thereby obtaining a repaired point cloud map 127 .

[0039] In some embodiments, computing device 120 may include any device capable of implementing computing functions and / or control functions, which may be any type of fixed computing device, mobile computing device, or portable computing device, including, but not limited to, a dedicated computer, a general-purpose computer, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a multimedia computer, a mobile phone, a general-purpose processor, a microprocessor, a microcontroller, or a state machine. Computing device 120 may be implemented as an individual computing device or a combination of computing devices, for example, a combination of a digital signal processor (DSP) and a microprocessor, a plurality of microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration.

[0040] It should be noted that, in some embodiments, the computing device 120 may be a computing device in the acquisition device 110. In other words, the generation and repair of the point cloud map 123 may be completed by the acquisition device 110. In other embodiments, the computing device 120 may also be a computing device for producing a point cloud map or a high-precision map, which may be set at a location far away from the acquisition device 110. For example, the computing device 120 may be a computing device on the cloud or server side. In this case, the acquisition device 110 may transmit the point cloud data 121 and the static obstacle information 125 to the remote computing device 120. In addition, it should be noted that the various processes or operations performed by the computing device 120 described herein may also be completed by multiple computing devices, that is, each computing device may implement a part of these processes or operations, and these computing devices may be located in different geographical locations or belong to different entities. In addition, in the context of the present disclosure, the computing device 120 may also be referred to as an electronic device 120, and these two terms may be used interchangeably herein.

[0041] Furthermore, it should be understood that Figure 1 Only objects, units, elements, or components related to the embodiments of the present disclosure in the example environment 100 are schematically shown. In practice, the example environment 100 may also include other possible objects, units, elements, or components, etc. In addition, Figure 1 The specific number of objects, units, elements, or components shown in the example environment 100 is merely illustrative and is not intended to limit the scope of the present disclosure in any way. In other embodiments, the example environment 100 may include any suitable number of objects, units, elements, or components, etc. Therefore, the embodiments of the present disclosure are not limited to Figure 1 The specific scenario described here is not specific to the context, but is generally applicable to any technical environment for patching point cloud maps. Figure 2 An exemplary data processing method of an embodiment of the present disclosure is described below.

[0042] Figure 2 2 shows a flow chart of an example data processing method 200 according to an embodiment of the present disclosure. In some embodiments, the example method 200 may be performed by Figure 1 The example method 200 may be implemented by the computing device 120 of the example environment 100, for example, by a processor or a processing unit of the computing device 120. In other embodiments, all or part of the example method 200 may also be implemented by other electronic devices independent of the example environment 100, or may be implemented by other devices or units in the example environment 100. For ease of description, reference will be made to Figure 1 The example method 200 is described by taking the computing device 120 executing the example method 200 as an example.

[0043] exist Figure 1In the example of FIG. 1 , it is assumed that the acquisition device 110 performs multiple acquisition processes to acquire point cloud data 121 for constructing a point cloud map 123 of the target area 105. In this case, Figure 2 At box 210, the computing device 120 may determine multiple candidate areas from the target area 105 based on the static obstacle information 125 of the acquisition device 110 during multiple acquisition processes. As used herein, a "candidate area" may refer to an area in the point cloud map 123 where point cloud data may need to be repaired, that is, a candidate for the area to be repaired in the point cloud map 123. Therefore, the computing device 120 may further determine the area to be repaired in the point cloud map 123 from multiple candidate areas in subsequent processing. It should be noted that the various areas mentioned in the context of the present disclosure refer to areas in the target area 105 in a strict sense, but since the objects, objects, positions or ranges in the point cloud map 123 of the target area 105 have a corresponding relationship with the real objects, objects, positions or ranges in the target area 105, the various areas in this article are sometimes also used to refer to areas in the point cloud map 123 without causing confusion.

[0044] It will be understood that since the acquisition device 110 performs multiple acquisition processes of the point cloud data on the target area 105, the static obstacles in different acquisition processes may be different. Figure 1 In the example of , during a certain acquisition process, one or both of vehicle 116 or vehicle 118 may be a static obstacle. However, during another different acquisition process, one or both of vehicle 116 or vehicle 118 may be driving in target area 105, or may have been driven out of target area 105, and are no longer static obstacles. Therefore, in some embodiments, computing device 120 may determine the position and size of static obstacles in each acquisition process of acquisition device 110 from static obstacle information 125, that is, the area corresponding to the static obstacle. Then, computing device 120 may determine the candidate area by combining the static obstacle areas in multiple acquisition processes. FIG. 3A to FIG. 3D To further illustrate an example of determining candidate regions for the area to be repaired based on the static obstacle region.

[0045] FIG. 3A to FIG. 3D FIG. 3 is a schematic diagram showing a plurality of static obstacle area sets 310 to 340 associated with multiple acquisition processes according to an embodiment of the present disclosure. Specifically, Figure 3A 1 shows a schematic diagram of a first static obstacle region set 310, which is associated with a first acquisition process of the acquisition device 110 for the target region 105. Similarly, Figure 3BA schematic diagram of a second static obstacle region set 320 is shown. The second static obstacle region set 320 is associated with a second acquisition process of the acquisition device 110 for the target region 105 . Figure 3C A schematic diagram of a third static obstacle area set 330 is shown. The third static obstacle area set 330 is associated with a third acquisition process of the acquisition device 110 for the target area 105 . Figure 3D A schematic diagram of a fourth static obstacle area set 340 is shown. The fourth static obstacle area set 340 is associated with a fourth acquisition process of the acquisition device 110 for the target area 105 .

[0046] It should be noted that although FIG. 3A to FIG. 3D The static obstacle area is depicted as a rectangular shape, but this depiction is only for illustration and is not intended to limit the scope of the present disclosure in any way. In other embodiments, the static obstacle area corresponding to the static obstacle in the target area 105 may also be any other shape, such as a circular shape, an elliptical shape, a shape defined by the outline of the static obstacle, and so on. It should also be noted that in actual processing, the static obstacle area corresponding to the static obstacle in the target area 105 may be a three-dimensional spatial area, because the static obstacle itself is generally an object or object occupying a three-dimensional space. Therefore, in some embodiments, FIG. 3A to FIG. 3D The various static obstacle areas in can be considered as projections of the three-dimensional static obstacle areas relative to the ground. FIG. 3A to FIG. 3D The various two-dimensional static obstacle areas in the image can be considered as a schematic representation. It should also be noted that the specific number (4 times) of acquisition processes performed by the acquisition device 110 described here is only schematic, and the embodiments of the present disclosure are equally applicable to any number of acquisition processes of the acquisition device 110.

[0047] like Figure 3AAs shown, in the first collection process of the collection device 110 for the target area 105, the traffic light 114, the trees 112-1 to 112-6, and the vehicles 116 and 118 parked on the roadside are static obstacles. Therefore, based on the static obstacle information 125, the computing device 120 can determine the first static obstacle area set 310 associated with the first collection process. Specifically, the first static obstacle area set 310 can include the static obstacle area 301 corresponding to the traffic light 114, the static obstacle area 303, the static obstacle area 305, the static obstacle area 307, the static obstacle area 311, the static obstacle area 315 and the static obstacle area 317 corresponding to the trees 112-1 to 112-6, and the static obstacle area 309 and the static obstacle area 313 corresponding to the vehicle 116 and the vehicle 118, respectively. The computing device 120 may remove the point cloud data in these areas from the point cloud data 121 , because these point cloud data are actually point cloud data of static obstacles, while point cloud data of the desired area is desired to be collected.

[0048] like Figure 3B As shown, relative to the first acquisition process, in the second acquisition process of the acquisition device 110 for the target area 105, the vehicle 118 parked on the roadside has been driven away from the target area 105, so the traffic light 114, the trees 112-1 to 112-6, and the vehicle 116 parked on the roadside are static obstacles. Therefore, based on the static obstacle information 125, the computing device 120 can determine the second static obstacle area set 320 associated with the second acquisition process. Specifically, the second static obstacle area set 320 can include the static obstacle area 301 corresponding to the traffic light 114, the static obstacle areas 303, 305, 307, 311, 315, and 317 corresponding to the trees 112-1 to 112-6, respectively, and the static obstacle area 309 corresponding to the vehicle 116. The computing device 120 may remove the point cloud data in these areas from the point cloud data 121 , because these point cloud data are actually point cloud data of static obstacles, while point cloud data of the desired area is desired to be collected.

[0049] like Figure 3CAs shown, relative to the second acquisition process, in the third acquisition process of the acquisition device 110 for the target area 105, a new static obstacle appears in the target area 105. For example, the new static obstacle may be a vehicle that is temporarily parked in the target area 105 due to a breakdown or waiting for a red light. In this case, the traffic light 114, the trees 112-1 to 112-6, the vehicle 116 parked on the roadside, and the temporarily parked vehicle are static obstacles. Therefore, based on the static obstacle information 125, the computing device 120 can determine the third static obstacle area set 330 associated with the third acquisition process. Specifically, the third static obstacle area set 330 may include a static obstacle area 301 corresponding to the traffic light 114, static obstacle areas 303, 305, 307, 311, 315, and 317 corresponding to the trees 112-1 to 112-6, respectively, a static obstacle area 309 corresponding to the vehicle 116, and a static obstacle area 319 corresponding to a temporarily parked vehicle. The computing device 120 may remove the point cloud data in these areas from the point cloud data 121, because these point cloud data are actually point cloud data of static obstacles, and the point cloud data of the area desired to be collected.

[0050] like Figure 3D As shown, relative to the third collection process, in the fourth collection process of the collection device 110 for the target area 105, the temporarily parked vehicle has left the target area 105, and another new static obstacle appears in the target area 105. For example, the other new static obstacle may be a guardrail facility temporarily set in the road 101 due to road maintenance. In this case, the traffic light 114, the trees 112-1 to 112-6, the vehicle 116 parked on the roadside, and the guardrail facility are static obstacles. Therefore, based on the static obstacle information 125, the computing device 120 can determine the static obstacle area set 340 associated with the fourth collection process. Specifically, the fourth static obstacle area set 340 may include a static obstacle area 301 corresponding to the traffic light 114, static obstacle areas 303, 305, 307, 311, 315, and 317 corresponding to the trees 112-1 to 112-6, respectively, a static obstacle area 309 corresponding to the vehicle 116, and a static obstacle area 321 corresponding to the guardrail facility. The computing device 120 may remove the point cloud data in these areas from the point cloud data 121, because these point cloud data are actually point cloud data of static obstacles, and the point cloud data of the area desired to be collected.

[0051] In some embodiments, when determining the candidate area of ​​the area to be repaired in the point cloud map 123, the computing device 120 may determine the union of the first static obstacle area set 310, the second static obstacle area set 320, the third static obstacle area set 330 and the fourth static obstacle area set 340 as the candidate area of ​​the area to be repaired. For example, the union of the first to fourth static obstacle area sets 310 to 340 may include static obstacle areas 301 to 321, which may be determined as candidate areas of the area to be repaired. In this way, the static obstacle areas in each acquisition process of the acquisition device 110 for the target area 105 are all taken as candidate areas, so that the computing device 120 can ensure that the situation of missing point cloud data caused by static obstacles will not be missed. In addition, since this method only involves the merging of multiple static obstacle area sets and does not involve the comparison between multiple static obstacle area sets, the computing overhead of the computing device 120 is low.

[0052] On the other hand, it can be noted that if a static obstacle does not exist in the target area 105 in each acquisition process, then for the point cloud data that is temporarily blocked by the static obstacle and not collected in a certain acquisition process, the acquisition device 110 can still collect these point cloud data through another acquisition process. For example, although the vehicle 118 may cause the acquisition device 110 to fail to collect the desired point cloud data in the static obstacle area 313 in the first acquisition process, the computing device 120 can still collect the desired point cloud data in the static obstacle area 313 in the second to fourth acquisition processes. In other words, in the point cloud map 123 generated based on the point cloud data 121 of multiple acquisition processes, the desired point cloud data in the static obstacle area 313 may already exist. Based on such considerations, in some embodiments, when determining the candidate area of ​​the area to be repaired in the point cloud map 123, the computing device 120 can obtain one or more candidate areas based on the intersection of multiple static obstacle area sets 310 to 340. In this way, the computing device 120 can effectively exclude areas where point cloud data already exists from among candidate areas used to determine the area to be repaired, thereby simplifying subsequent processing by the computing device 120 .

[0053] Figure 3E FIG. 3 is a schematic diagram showing an intersection 350 of multiple static obstacle area sets 310 to 340 according to an embodiment of the present disclosure. Figure 3EAs shown, the computing device 120 may determine an intersection 350 of a first static obstacle region set 310, a second static obstacle region set 320, a third static obstacle region set 330, and a fourth static obstacle region set 340. Specifically, the intersection 350 may include a static obstacle region 301 corresponding to the traffic light 114, static obstacle regions 303, 305, 307, 311, 315, and 317 corresponding to trees 112-1 to 112-6, respectively, and a static obstacle region 309 corresponding to the vehicle 116. It will be noted that the intersection 350 may not include a static obstacle region 313 corresponding to the vehicle 118, a static obstacle region 319 corresponding to a temporarily parked vehicle, and a static obstacle region 321 corresponding to a guardrail facility, because the desired collected point cloud data in these static obstacle regions may be obtained by integrating point cloud data of multiple collection processes.

[0054] Return to reference Figure 2 , at box 220, after determining multiple candidate areas for the area to be repaired, the computing device 120 can determine one or more areas to be repaired from the multiple candidate areas based on the multiple parts in the point cloud map 123 corresponding to the multiple candidate areas. That is, the computing device 120 can determine whether the candidate area is the area to be repaired that requires repairing the point cloud data based on the actual situation of the corresponding part in the point cloud map 123. For example, in some embodiments, the computing device 120 can determine whether the candidate area is the area to be repaired based on whether there is point cloud data in the corresponding part of the candidate area in the point cloud map 123. For example, the computing device 120 can count the point cloud data of the candidate area in the corresponding part of the point cloud map 123. If the point cloud data in the corresponding part is not missing, the computing device 120 can filter out the candidate area, that is, determine that the candidate area is not the area to be repaired. This will be referenced later. Figure 4 and Figure 5 Such embodiments are further described below.

[0055] In other embodiments, point cloud data may exist in the corresponding part of the candidate area in the point cloud map 123, but the computing device 120 may determine that the point cloud data in the corresponding part is insufficient, which may affect the subsequent application of the point cloud map 123. For example, the lack of point cloud data may be because the candidate area is collected less times than other candidate areas that are not affected by static obstacles. In this case, the computing device 120 may also determine the candidate area with insufficient point cloud data as the area to be repaired. In another embodiment, point cloud data may exist in the corresponding part of the candidate area in the point cloud map 123, but the computing device 120 may determine that the point cloud data in the corresponding part is actually the point cloud data of the static obstacle, rather than the point cloud data intended to be collected. For example, this may be because the static obstacle blocking the area is not detected, so the point cloud data of the corresponding part of the candidate area in the point cloud map 123 is not removed. In this case, the computing device 120 may also determine the candidate area as the area to be repaired.

[0056] At block 230, after determining one or more to-be-repaired areas in the target area 105 that need to be repaired with point cloud data, the computing device 120 may determine the missing point cloud data in the to-be-repaired areas using the point cloud data of the reference area associated with the to-be-repaired areas in the point cloud map 123. As used herein, "reference area" may generally refer to an area that has similarities with the to-be-repaired area in certain attributes, so the point cloud data in the reference area may be used to determine the missing point cloud data in the to-be-repaired areas.

[0057] For example, the reference area may be an area near or around the area to be repaired, so there is a similarity in geographical location between the reference area and the area to be repaired. For another example, if the area to be repaired is part of road 101, then the reference area and the area to be repaired may be basically located in the direction of the width of the road, which means that the positions of the two in the direction of the length of the road are similar. Because in general, road 101 is basically horizontal in the direction of its width, and there will be no large inclination or height fluctuations, so if the reference area and the area to be repaired have similar positions in the direction of the length of the road, then the point cloud data in the reference area can be used to determine the missing point cloud data in the area to be repaired. In other embodiments, the computing device 120 may determine the reference area of ​​the area to be repaired by expanding the area to be repaired, which will be referred to later. Figure 7 Such embodiments are further described below.

[0058] It should be noted that the computing device 120 can use any appropriate method to derive the missing point cloud data in the area to be repaired from the point cloud data in the reference area. For example, in a scenario where both the reference area and the area to be repaired are road areas, the computing device 120 can directly use the height data in the point cloud data in the reference area (for example, the average height of the point cloud) as the height data of the point cloud data in the area to be repaired. Then, the computing device 120 can linearly fit the longitude and latitude data in the point cloud data in the area to be repaired based on the longitude and latitude coordinates in the point cloud data in the reference area. In other embodiments, the computing device 120 can also use a repair algorithm based on surface reconstruction, a repair algorithm based on the idea of ​​local expansion, etc. to derive the missing point cloud data in the area to be repaired based on the point cloud data in the reference area.

[0059] In other embodiments, the computing device 120 may use the point cloud data of the reference area to perform plane fitting to obtain the missing point cloud data in the area to be repaired. In this way, the computing device 120 can use a low-complexity algorithm to reasonably fit the missing point cloud data. Specifically, in a scene where both the reference area and the area to be repaired are road areas, the computing device 120 can use the point cloud data of the reference area to perform plane fitting, thereby obtaining an approximate plane equation description of the road surface. Then, the computing device 120 can use the obtained plane equation description to calculate the missing point cloud data in the area to be repaired, such as the height data of the point cloud. In some embodiments, the plane fitting process needs to consider the robustness of the outliers, so the random sampling consensus (RANSAC) algorithm can be used for plane fitting. For points that exceed a certain threshold from the fitted plane, they can be considered as outliers or outliers (for example, they may be caused by residual static obstacle traces) and deleted.

[0060] Through the example method 200, the computing device 120 can automatically use the static obstacle information 125 to determine the area to be repaired in the point cloud map 123, so the efficiency and accuracy of determining the area to be repaired can be improved, while avoiding the operator's manual marking of the area to be repaired in the point cloud map 123. In addition, since the computing device 120 uses the point cloud data of the existing reference area in the point cloud map 123 to repair the missing point cloud data, the computing device 120 can quickly, accurately and automatically determine the missing point cloud data in the area to be repaired. In summary, the example method 200 can improve the efficiency, accuracy and automation level of point cloud map repair.

[0061] As mentioned above in describing block 220 of the example method 200, in some embodiments, the computing device 120 may determine whether the candidate area is a to-be-repaired area based on whether point cloud data exists for a corresponding portion of the candidate area in the point cloud map 123. Figure 4 and Figure 5 Such embodiments are further described below.

[0062] Figure 4 FIG. 4 shows an example process 400 for determining one or more areas to be repaired according to an embodiment of the present disclosure. In some embodiments, the example process 400 may be performed by Figure 1 The example process 400 may be implemented by the computing device 120 of the example environment 100, for example, by a processor or a processing unit of the computing device 120. In other embodiments, all or part of the example process 400 may also be implemented by other electronic devices independent of the example environment 100, or may be implemented by other devices or units in the example environment 100.

[0063] Figure 5 FIG. 1 is a schematic diagram showing a plurality of corresponding parts 501 to 521 of a plurality of candidate areas 301 to 321 in a point cloud map 123 according to an embodiment of the present disclosure. Figure 5 As shown, the point cloud map 123 may be composed of a plurality of points including the point 510. Figure 5 In the example of , it is assumed that the computing device 120 has taken the union of the first static obstacle area set 310, the second static obstacle area set 320, the third static obstacle area set 330 and the fourth static obstacle area set 340 as the candidate area for the area to be repaired, so FIG. 3A to FIG. 3D The static obstacle regions 301 to 321 depicted in FIG. 1 are all candidate regions of the region to be repaired. In this case, in the point cloud map 123 generated based on the point cloud data 121 , the computing device 120 may determine a plurality of parts 501 to 521 corresponding to the plurality of candidate regions 301 to 321 .

[0064] In addition, it should be noted that although Figure 5 The multiple parts 501 to 521 are depicted as rectangular shapes, but this depiction is only for illustration and is not intended to limit the scope of the present disclosure in any way. In other embodiments, the multiple parts 501 to 521 in the point cloud map 123 may also be any other shapes, such as a circular shape, an elliptical shape, a shape defined by the outline of a static obstacle, and so on. It should also be noted that in actual processing, the multiple parts 501 to 521 in the point cloud map 123 may be three-dimensional spatial areas, because the static obstacles themselves are generally objects or objects occupying a three-dimensional space. Therefore, in some embodiments, Figure 5 The multiple parts 501 to 521 in the image may be considered as projections of the three-dimensional area relative to the ground. Figure 5 The various two-dimensional regions in can be considered as a schematic representation. It should be further explained that Figure 5The specific distribution of the point cloud depicted in FIG. 1 is only for illustration to show whether a point cloud exists in a specific area, and is not intended to limit the scope of the present disclosure in any way. In a practical scenario, the distribution of the point cloud should actually have a shape defined by the outer contour of an object or body.

[0065] Also refer to Figure 4 and Figure 5 ,exist Figure 4 At box 410 of , the computing device 120 may determine one or more parts of the point cloud map 123 where point cloud data is missing from the multiple parts 501 to 521. For example, the computing device 120 may determine whether the point cloud density in a certain part is lower than a threshold density. If the point cloud density in the part is lower than the threshold density, it can be considered that the point cloud data in the part is missing. On the contrary, if the point cloud density in the part is not lower than the threshold density, it can be considered that point cloud data exists in the part. In some embodiments, the threshold density can be determined based on the specific technical environment and accuracy requirements. As an example, in Figure 5 In the depicted scene, computing device 120 may determine that portion 501, portion 503, portion 505, portion 507, portion 509, portion 511, portion 515, and portion 517 are portions where point cloud data is missing. In contrast, computing device 120 may determine that portion 513, portion 519, and portion 521 are portions where point cloud data exists.

[0066] At block 420 , after determining one or more portions of the point cloud data in the point cloud map 123 that are missing, the computing device 120 may determine a candidate region corresponding to the one or more portions from among the plurality of candidate regions 301 to 321 . Figure 5 In the depicted example, since part 501, part 503, part 505, part 507, part 509, part 511, part 515, and part 517 are missing parts of the point cloud data, the computing device 120 can determine candidate areas corresponding to these parts from the candidate areas 301 to 321, that is, candidate area 301, candidate area 303, candidate area 305, candidate area 307, candidate area 309, candidate area 311, candidate area 315, and candidate area 317.

[0067] At block 430, after determining one or more candidate regions corresponding to the missing portion of the point cloud data, the computing device 120 may obtain one or more regions to be repaired based on the one or more candidate regions. In some embodiments, the computing device 120 may determine the one or more candidate regions as the regions to be repaired. Figure 5In the example of , the computing device 120 may determine the candidate area 301, the candidate area 303, the candidate area 305, the candidate area 307, the candidate area 309, the candidate area 311, the candidate area 315 and the candidate area 317 as the area to be repaired. In other embodiments, the computing device 120 may perform further selection or filtering on the one or more candidate areas to determine the area to be repaired that needs to repair the point cloud data. Such embodiments will be described in detail below.

[0068] Through the example process 400, based on whether point cloud data exists in the corresponding part of the candidate area in the point cloud map 123, the computing device 120 can exclude the candidate area where point cloud data already exists in the point cloud map 123, thereby eliminating the possibility of performing point cloud data patching on the candidate area for which point cloud data has already been collected, thereby simplifying the subsequent processing of the computing device 120, saving the processing resources of the computing device 120, and avoiding the reduction in the accuracy of the point cloud map 123 caused by replacing the actually collected point cloud data with the fitted point cloud data.

[0069] As mentioned above, the computing device 120 may further select one or more candidate areas obtained in the box 430 of the example process 440 to determine the area to be repaired in the point cloud map 123 where the point cloud data needs to be repaired. Specifically, in some embodiments, especially when the target area 105 includes the road 101, since the repaired point cloud map 127 generated by the computing device 120 may be used for the automatic driving of the vehicle later, and the vehicle is usually traveling within the road 101, the computing device 120 may pay more attention to the road surface point cloud within the road 101. In this case, if the candidate area is determined to be missing point cloud data in the corresponding part in the point cloud map 123, the computing device 120 may further determine whether the candidate area needs to be repaired based on the positional relationship between the candidate area and the road 101. In this way, the computing device 120 can select the area to be repaired more specifically, that is, improve the effectiveness of the determined area to be repaired. At the same time, since the possibility of excluding a part of the candidate area that does not necessarily need to repair the point cloud data as the area to be repaired is excluded, the subsequent processing overhead of the computing device 120 can be reduced.

[0070] Specifically, in some embodiments, in order to select one or more areas to be repaired from the one or more candidate areas, the computing device 120 may determine which one or more candidate areas of the one or more candidate areas are within the road 101 based on the road boundary point cloud corresponding to the road boundary line 102 or 104 in the point cloud map 123. Figure 5In the example of FIG. 5 , based on the positional relationship between the road boundary point cloud 520 and the portion 501 , the portion 503 , the portion 505 , the portion 507 , and the portion 509 , the computing device 120 may determine FIG. 3A to FIG. 3D The candidate area 309 depicted in FIG. 1 is within the road 101, while the candidate areas 301, 303, 305, and 307 are outside the road 101. Similarly, based on the positional relationship between the road boundary point cloud 540 and the portion 511, the portion 515, and the portion 517, the computing device 120 can determine FIG. 3A to FIG. 3D The candidate areas 311, 315, and 317 depicted in FIG. 1 are outside the road 101. In the following, for the convenience of description, the candidate area 309 located within the road 101 may also be referred to as the first candidate area. Then, the computing device 120 may determine the first candidate area 309 located within the road 101 as the area to be repaired that needs to repair the point cloud data.

[0071] In some cases, the candidate area may be neither completely within the road 101 nor completely outside the road 101, but may be partially within the road 101 and partially outside the road 101. For example, the corresponding portion of the candidate area in the point cloud map 123 may intersect with the road boundary point cloud 520 or 540. In such a case, the computing device 120 may determine the portion of the candidate area within the road 101 as the area to be repaired. In this way, the computing device 120 can selectively repair the more important missing point cloud data within the road 101 in the point cloud map 123, avoiding the repair of the less meaningful point cloud data outside the road 101 in the point cloud map 123, thereby simplifying the processing of the computing device 120 and saving the processing overhead of the computing device 120. Fig. 6A and Figure 6B Such an embodiment will be described in detail.

[0072] Fig. 6A FIG. 6 is a schematic diagram showing the intersection of a candidate area 610 and a road boundary line 102 according to an embodiment of the present disclosure. Fig. 6A In the figure, for the sake of clarity, only the road 101 and its related elements are shown, and other objects or elements are omitted. Fig. 6AAs shown, it is assumed that the candidate area 610 corresponds to a static obstacle in the target area 105. For example, the static obstacle may be a vehicle parked on the roadside. However, the vehicle may not be parked in a standardized manner, but a part of it is located inside the road 101 and the other part is located outside the road 101. Therefore, the candidate area 610 corresponding to the vehicle intersects with the road boundary line 102 of the road 101, so that the candidate area 610 has a sub-area 612 located inside the road 101 and a sub-area 614 located outside the road 101. In the following, for the convenience of description, the candidate area 610 intersecting with the road boundary line 102 of the road 101 may also be referred to as a second candidate area.

[0073] Figure 6B FIG. 1 is a schematic diagram showing the intersection of a corresponding portion 620 of a candidate area 610 in a point cloud map 123 and a road boundary point cloud 520 according to an embodiment of the present disclosure. Figure 6B In the figure, for the sake of clarity, only the road boundary point clouds 520 and 540 are shown, and the point clouds of other objects or elements are omitted. Figure 6B As shown, based on the road boundary point cloud 520, for example, based on the positional relationship between the road boundary point cloud 520 and the portion 620, the computing device 120 may determine that the candidate area 610 intersects with the road boundary line 102 of the road 101. For example, in the point cloud map 123, the computing device 120 may determine that the road boundary point cloud 520 just passes through the portion 610, so that the portion 610 has a sub-portion 622 located within the road boundary point cloud 520 and a sub-portion 624 located outside the road boundary point cloud 520. Therefore, returning to reference Fig. 6A , the computing device 120 can correspondingly determine the second candidate sub-region 612 of the second candidate region 610 within the road 101. Then, the computing device 120 can determine the second candidate sub-region 612 as the area to be repaired that needs to repair the point cloud data. In this way, the computing device 120 can selectively repair the more important missing point cloud data located within the road 101 in the point cloud map 123, avoiding the repair of the less significant point cloud data located outside the road 101 in the point cloud map 123, thereby simplifying the processing of the computing device 120 and saving the processing overhead of the computing device 120.

[0074] As mentioned above in describing block 230 of the example method 200, in some embodiments, the computing device 120 may determine the reference area of ​​the area to be repaired by extending the area to be repaired. Figure 7 Such embodiments are further described below.

[0075] Figure 7 FIG. 7 is a schematic diagram showing a middle area 715 between the extended area 720 and the area to be repaired 710 as a reference area according to an embodiment of the present disclosure. Figure 7 In the example of FIG. 1 , a real point cloud map 700 is shown, which is collected by a collection device 110 (eg, a drone carrying a laser radar measurement device) for a real road. Figure 7 As shown, there is a to-be-repaired area 710 with missing point cloud data near the road boundary point cloud 730. In order to determine the reference area of ​​the to-be-repaired area 710, the computing device 120 may expand the to-be-repaired area 710 to obtain an expanded area 720. For example, the computing device 120 may select an appropriate expansion distance, such as several tens of centimeters, and move each edge of the to-be-repaired area 710 outward by the expansion distance.

[0076] exist Figure 7 , the computing device 120 can expand the other three edges of the area to be repaired 710 except the edge that coincides with the road boundary point cloud 730 to appropriately cover the area around the area to be repaired 710 where point cloud data exists. This is because, even if the edge that coincides with the road boundary point cloud 730 is expanded outward, the extended area 720 cannot cover a larger area where valid point cloud data exists. Of course, if the area to be repaired is located in the middle of the road, the area to be repaired can also be expanded in all directions. After obtaining the extended area 720, the computing device 120 can determine the middle area 715 between the extended area 720 and the area to be repaired 710 as a reference area to determine the missing point cloud data in the area to be repaired 710. In this way, the computing device 120 can use the point cloud data closest to the area to be repaired to repair the missing point cloud data in the area to be repaired, thereby improving the accuracy of the repaired point cloud map 127.

[0077] Figure 8 800 according to an embodiment of the present disclosure. In some embodiments, the apparatus 800 may be included in Figure 1 The computing device 120 of the embodiment of the present invention may be implemented in or as the computing device 120.

[0078] like Figure 8 As shown, the example device 800 may include a candidate area determination module 810, a to-be-repaired area determination module 820, and a missing point cloud data determination module 830. The candidate area determination module 810 may be configured to determine multiple candidate areas from the target area based on static obstacle information in multiple acquisition processes, and the multiple acquisition processes are used to acquire point cloud data for constructing a point cloud map of the target area. The to-be-repaired area determination module 820 may be configured to determine at least one to-be-repaired area from multiple candidate areas based on multiple parts corresponding to the multiple candidate areas in the point cloud map. The missing point cloud data determination module 830 may be configured to determine the missing point cloud data in at least one to-be-repaired area using the point cloud data of a reference area associated with at least one to-be-repaired area in the point cloud map.

[0079] In some embodiments, the module for determining the area to be repaired 820 may include: a module for determining a portion without point cloud data, configured to determine at least one portion where point cloud data is missing from multiple portions; a module for determining a candidate area without point cloud data, configured to determine at least one candidate area corresponding to at least one portion from multiple candidate areas; and a module for obtaining the area to be repaired, configured to obtain at least one area to be repaired based on at least one candidate area.

[0080] In some embodiments, the module for obtaining the area to be repaired may include: a road boundary point cloud determination module, configured to determine a road boundary point cloud corresponding to the boundary of the road in a point cloud map; and a module for selecting the area to be repaired, configured to select at least one area to be repaired from at least one candidate area based on the road boundary point cloud.

[0081] In some embodiments, the module for selecting the area to be repaired may include: a module for determining a candidate area within a first road, configured to determine, based on a road boundary point cloud, that a first candidate area among at least one candidate area is within a road; and a module for determining the area to be repaired within the first road, configured to determine the first candidate area as the area to be repaired among at least one area to be repaired.

[0082] In some embodiments, the module for selecting the area to be repaired may include: a module for determining a candidate area within a second road, configured to determine a second candidate sub-area of ​​the second candidate area within the road if, based on the road boundary point cloud, it is determined that a second candidate area among at least one candidate area intersects with the boundary of the road; and a module for determining the area to be repaired within the second road, configured to determine the second candidate sub-area as the area to be repaired among at least one area to be repaired.

[0083] In some embodiments, the candidate area determination module 810 may include: a static obstacle area set determination module, configured to determine multiple static obstacle area sets corresponding to multiple acquisition processes respectively based on static obstacle information; and a candidate area acquisition module, configured to obtain multiple candidate areas based on the intersection of multiple static obstacle area sets.

[0084] In some embodiments, the example device 800 may also include: a first extended area acquisition module, configured to expand a first area to be repaired in at least one area to be repaired to obtain a first extended area; and a first reference area determination module, configured to determine an intermediate area between the first extended area and the first area to be repaired as a first reference area in the reference area corresponding to the first area to be repaired.

[0085] In some embodiments, the missing point cloud data determination module 830 may include: a plane fitting module configured to perform plane fitting using the point cloud data of the reference area to obtain the missing point cloud data.

[0086] Fig. 9 900 is a block diagram of an example device that can be used to implement embodiments of the present disclosure. Fig. 9 As shown in , the example device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only storage device (ROM) 902 or loaded from a storage unit 908 to a random access storage device (RAM) 903. In the RAM 903, various programs and data required for the operation of the example device 900 can also be stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0087] Multiple components in the example device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the example device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0088] The various processes and processing described above, such as various example methods or example processes, may be performed by the processing unit 901. For example, in some embodiments, the various example methods or example processes may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the example device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the CPU 901, one or more steps of the various example methods or example processes described above may be performed.

[0089] As used herein, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included herein.

[0090] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" may include computing, calculating, processing, deriving, investigating, searching (e.g., searching in a table, a database, or another data structure), ascertaining, etc. In addition, "determine" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. In addition, "determine" may include resolving, selecting, choosing, establishing, etc.

[0091] It should be noted that the embodiments of the present disclosure may be implemented by hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portion may be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods may be implemented using computer executable instructions and / or contained in a processor control code, such as a programmable memory or a data carrier such as an optical or electronic signal carrier, providing such code.

[0092] In addition, although the operation of the method of the present disclosure is described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flow chart can change the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution. It should also be noted that the features and functions of two or more devices according to the present disclosure can be embodied in one device. Conversely, the features and functions of a device described above can be further divided into being embodied by multiple devices.

[0093] Although the present disclosure has been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A data processing method, comprising: Determining multiple candidate areas from the target area based on multiple static obstacle area sets respectively corresponding to multiple acquisition processes, wherein the multiple acquisition processes are used to acquire point cloud data for constructing a point cloud map of the target area; Determine at least one portion of the point cloud data that is missing based on a plurality of portions in the point cloud map corresponding to the plurality of candidate areas; Determine, among the plurality of candidate regions, at least one candidate region corresponding to the at least one portion; Determining a road boundary point cloud corresponding to a boundary of the road in the point cloud map; Based on the road boundary point cloud, selecting the at least one area to be repaired from the at least one candidate area; Expanding a first area to be repaired in the at least one area to be repaired to obtain a first expanded area; Determine an intermediate area between the first extended area and the first area to be repaired as a reference area corresponding to the first area to be repaired; as well as The missing point cloud data in the first area to be repaired is determined by using the point cloud data of the reference area associated with the first area to be repaired in the point cloud map.

2. The method according to claim 1, wherein selecting the at least one area to be repaired comprises: Based on the road boundary point cloud, determine that a first candidate area among the at least one candidate area is within the road; as well as The first candidate region is determined as the region to be repaired in the at least one region to be repaired.

3. The method according to claim 1, wherein selecting the at least one area to be repaired comprises: If it is determined based on the road boundary point cloud that a second candidate region among the at least one candidate region intersects with the boundary of the road, then determining a second candidate subregion of the second candidate region within the road; as well as The second candidate sub-region is determined as the to-be-repaired region in the at least one to-be-repaired region.

4. The method according to claim 1, wherein the plurality of static obstacle area sets corresponding to the plurality of acquisition processes are determined based on the static obstacle information, and the plurality of candidate areas are determined from the target area based on the plurality of static obstacle area sets corresponding to the plurality of acquisition processes, comprising: The plurality of candidate regions are obtained based on an intersection of the plurality of static obstacle region sets.

5. The method according to claim 1, wherein determining the missing point cloud data in the first area to be repaired comprises: Plane fitting is performed using the point cloud data of the reference area to obtain the missing point cloud data.

6. A data processing device, comprising: A candidate area determination module is configured to determine a plurality of candidate areas from a target area based on a plurality of static obstacle area sets respectively corresponding to a plurality of acquisition processes, wherein the plurality of acquisition processes are used to acquire point cloud data for constructing a point cloud map of the target area; A module for determining an area to be repaired, configured to determine at least one missing part of the point cloud data based on a plurality of parts in the point cloud map corresponding to the plurality of candidate areas; Determine, among the plurality of candidate regions, at least one candidate region corresponding to the at least one portion; Determining a road boundary point cloud corresponding to a road boundary in the point cloud map; selecting the at least one area to be repaired from the at least one candidate area based on the road boundary point cloud; A first extended area obtaining module is configured to expand a first to-be-repaired area in the at least one to-be-repaired area to obtain a first extended area; A first reference region determining module is configured to determine an intermediate region between the first extended region and the first region to be repaired as a reference region corresponding to the first region to be repaired; as well as The missing point cloud data determination module is configured to determine the missing point cloud data in the first area to be repaired by using the point cloud data of the reference area corresponding to the first area to be repaired in the point cloud map.

7. The device according to claim 6, wherein the module for determining the area to be repaired comprises: A first candidate region determination module within the road, configured to determine, based on the road boundary point cloud, that a first candidate region among the at least one candidate region is within the road; as well as The first road region-to-be-repaired area determination module is configured to determine the first candidate area as the region to be repaired in the at least one region to be repaired.

8. The device according to claim 6, wherein the module for determining the area to be repaired comprises: A second candidate region determination module within the road, configured to determine a second candidate sub-region of the second candidate region within the road if it is determined based on the road boundary point cloud that a second candidate region among the at least one candidate region intersects the boundary of the road; as well as The module for determining the area to be repaired in the second road is configured to determine the second candidate sub-area as the area to be repaired in the at least one area to be repaired.

9. The apparatus according to claim 6, wherein the candidate region determination module comprises: A static obstacle area set determination module, configured to determine the plurality of static obstacle area sets corresponding to the plurality of acquisition processes respectively based on the static obstacle information; as well as The candidate region obtaining module is configured to obtain the multiple candidate regions based on the intersection of the multiple static obstacle region sets.

10. The device according to claim 6, wherein the missing point cloud data determination module comprises: The plane fitting module is configured to perform plane fitting using the point cloud data of the reference area to obtain the missing point cloud data.

11. An electronic device, comprising: one or more processors; as well as A memory storing computer executable instructions, which, when executed by the one or more processors, enables the electronic device to implement the method according to any one of claims 1 to 5.

12. A computer-readable medium having computer-executable instructions stored thereon, wherein the computer-executable instructions implement the method according to any one of claims 1 to 5 when executed by a processor.

Images