Map updating system, method and device

Abstract

The present application discloses a map updating system, related methods, devices and equipment. Among them, the map updating method includes: collecting road environment image data, and determining the posture data of the mobile device at the time of image collection; determining the first correspondence between the road feature observation elements of the road segment and the map elements according to the image data and posture data at the current time and multiple historical times; adjusting the observation elements at the current time according to the first correspondence; updating the map elements corresponding to the posture data at the current time according to the adjusted observation elements at the current time. By adopting this processing method, the camera sensor and the relatively low-cost positioning device equipped with a general mobile device can be used to improve the accuracy of local relative measurement, compensate for the insufficient measurement accuracy of the camera sensor and the positioning device, and realize real-time and accurate change detection of map data on the mobile device side; therefore, it can take into account both the lower cost of the mobile device and the higher map update accuracy.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and specifically to map updating systems, methods and devices, electronic devices, and mobile devices. Background Art

[0002] In scenarios such as autonomous driving, indoor robots, and drones, mobile device positioning technology based on environmental maps can be used. Taking autonomous driving as an example, this technology allows the use of high-performance equipment or complex methods to obtain maps with a preset clarity level in advance. During real-time positioning, the mobile device obtains real-time observation data through sensors, and then aligns the observation data with the map data to obtain the position data of the mobile device.

[0003] Map production can be divided into data collection, data processing, manual verification and other links. Among them, data collection requires a special data collection vehicle, which uses sensors such as lidar and cameras to obtain road environment information, and requires high-precision positioning equipment to provide vehicle posture information, so as to combine the road environment information into a map in space. The actual road is constantly changing, and the map needs to be maintained and updated after production, that is: the latest observation of the actual road surface is made by sensors, and the existing map and the latest observation data are aligned according to the vehicle posture information, so that point-to-point map transformation detection can be performed directly. At this time, a special data collection vehicle becomes an expensive option, because map maintenance and updating is a continuous and high-frequency task. In order to reduce the cost of map update equipment, the current main approach is crowdsourcing, using vehicles equipped with low-cost sensors and positioning equipment, such as general self-driving vehicles, to update maps.

[0004] However, in the process of realizing the present invention, the inventors found that the prior art has at least the following problems: 1) Low-cost sensors cannot provide accurate environmental information, resulting in problems such as false detection (observation distortion), missed detection (incomplete observation), and noise (observation noise) in the observation data during map updates; 2) Low-cost positioning equipment cannot provide accurate vehicle posture, resulting in limited positioning accuracy and poor consistency of multiple measurements during map updates. Both of the above situations make it impossible to directly judge map changes, resulting in low map accuracy. Accordingly, the device positioning technology based on the existing map update method has the problem of low positioning accuracy. Therefore, how to perform reliable map change detection is a challenging task. Summary of the invention

[0005] The present application provides a map updating system to solve the problem of low map updating accuracy in the prior art. The present application also provides a map updating method and device, a newly added building detection method and device, an electronic device, and a mobile device.

[0006] The present application provides a map updating system, comprising:

[0007] A mobile device, used to collect road environment image data and determine the position and posture data of the mobile device at the time of image collection; determine the first correspondence between the road feature observation element of the road segment and the map element according to the image data and the position and posture data at the current time and multiple historical times; adjust the road feature observation element at the current time according to at least the first correspondence; if the adjusted road feature observation element at the current time is inconsistent with the map element, send a map update request for the position and posture data at the current time to a server;

[0008] The server is used to receive the map update request and update the map elements corresponding to the posture data at the current moment according to the adjusted road feature observation elements at the current moment.

[0009] This application provides a map updating method, including:

[0010] Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection;

[0011] Determine a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and a plurality of historical moments;

[0012] Adjusting the observation element at the current moment according to at least the first corresponding relationship;

[0013] According to the adjusted observation element at the current moment, the map element corresponding to the posture data at the current moment is updated.

[0014] Optionally, adjusting the road feature element at the current moment at least according to the first corresponding relationship includes:

[0015] According to the first corresponding relationship, the road characteristic elements that are not observed at the current moment are determined.

[0016] Optionally, adjusting the road feature element at the current moment at least according to the first corresponding relationship includes:

[0017] According to the first corresponding relationship, the noise elements in the observation elements at the current moment are removed.

[0018] Optionally, also include:

[0019] Determining a second correspondence between map elements and unregistered observation elements; the unregistered observation elements include observation elements not included in the first correspondence;

[0020] At least according to the second correspondence, adjusting the unregistered observation elements at the current moment;

[0021] According to the adjusted unregistered observation elements at the current moment, the map elements corresponding to the pose data at the current moment are updated.

[0022] Optionally, adjusting the unregistered observation element at the current moment at least according to the second corresponding relationship includes:

[0023] According to the second corresponding relationship, the noise elements in the road feature elements at the current moment are removed.

[0024] Optionally, adjusting the unregistered observation element at the current moment at least according to the second corresponding relationship includes:

[0025] According to the second corresponding relationship, orthogonal processing is performed on the distorted road feature elements at the current moment.

[0026] Optionally, adjusting the unregistered observation element at the current moment at least according to the second corresponding relationship includes:

[0027] Through the local optimization graph model algorithm, the unaligned observation elements at the current moment are adjusted according to the first correspondence and the second correspondence.

[0028] Optionally, adjusting the unregistered observation elements at the current moment and the pose data at the current moment according to the first correspondence and the second correspondence by using a local optimization graph model algorithm;

[0029] According to the adjusted unregistered observation elements, the map elements corresponding to the adjusted pose data are updated.

[0030] Optionally, the first corresponding relationship is determined according to the image data and the posture data at the current moment and multiple adjacent historical moments by dynamically stitching multiple frames of observation elements, so that the registration level coefficient is greater than the first coefficient threshold and less than the second coefficient threshold; the registration level coefficient is determined according to the number of registration elements.

[0031] Optionally, the registration level coefficient is determined according to the number of registration elements and the quality of observation elements.

[0032] Optionally, the first corresponding relationship is determined by the following steps:

[0033] For each image acquisition moment, determining the road feature observation element at the moment according to the image data at the moment;

[0034] According to the posture data of the current moment and multiple adjacent historical moments, the road feature observation elements of the current moment and multiple adjacent historical moments are spliced;

[0035] Determine a first correspondence between observation elements of the spliced ​​road segment and map elements; and determine an observation quality score of the observation element;

[0036] Determine the registration level coefficient according to the number of observation elements included in the first corresponding relationship and the observation quality scores of the observation elements;

[0037] If the registration level coefficient is less than the first coefficient threshold, then add and splice the observation element of at least one historical moment; if the registration level coefficient is greater than the second coefficient threshold, then remove the observation element of at least one historical moment;

[0038] According to the spliced ​​and adjusted observation elements, the first corresponding relationship and the registration level coefficient are re-determined until the registration level coefficient is greater than or equal to the first coefficient threshold and the registration level coefficient is less than or equal to the second coefficient threshold.

[0039] Optionally, determining the observation quality score of the observation element includes:

[0040] The probability that the observed element is an element type is used as the observation quality score.

[0041] The present application also provides a map updating method, comprising:

[0042] Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection;

[0043] Determine a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and a plurality of historical moments;

[0044] Adjusting the road feature observation element at the current moment at least according to the first corresponding relationship;

[0045] If the adjusted road feature observation element at the current moment is inconsistent with the map element, a map update request for the posture data at the current moment is sent to the server.

[0046] The present application also provides a map updating method, comprising:

[0047] Receiving a map update request for a target position sent by a mobile device;

[0048] According to the adjusted road feature element corresponding to the target posture carried in the request, the map element corresponding to the target posture is updated.

[0049] The present application also provides a map updating device, comprising:

[0050] An image acquisition unit, used for acquiring road environment image data;

[0051] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0052] A multi-frame element registration unit, used for determining a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and a plurality of historical moments;

[0053] An element adjustment unit, configured to adjust the observation element at the current moment at least according to the first corresponding relationship;

[0054] The map updating unit is used to update the map elements corresponding to the posture data at the current moment according to the adjusted observation elements at the current moment.

[0055] The present application also provides a mobile device, comprising:

[0056] Image acquisition device;

[0057] Processor; and

[0058] The memory is used to store a program for implementing a map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting road environment image data and determining the posture data of the mobile device at the time of image collection; determining the first correspondence between the road feature observation elements of the road segment and the map elements based on the image data and the posture data at the current time and multiple historical times; adjusting the observation elements at the current time at least based on the first correspondence; and updating the map elements corresponding to the posture data at the current time based on the adjusted observation elements at the current time.

[0059] The present application also provides a map updating device, comprising:

[0060] An image acquisition unit, used for acquiring road environment image data;

[0061] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0062] A multi-frame element registration unit, used for determining a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and a plurality of historical moments;

[0063] An element adjustment unit, configured to adjust the road feature observation element at a current moment at least according to the first corresponding relationship;

[0064] The request sending unit is used to send a map update request for the posture data at the current moment to the server if the adjusted road feature observation element at the current moment is inconsistent with the map element.

[0065] The present application also provides a mobile device, comprising:

[0066] Image acquisition device;

[0067] Processor; and

[0068] A memory is used to store a program for implementing a map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting road environment image data and determining the posture data of the mobile device at the time of image collection; determining the first correspondence between the road feature observation elements of the road segment and the map elements based on the image data and the posture data at the current time and multiple historical times; adjusting the road feature observation elements at the current time at least based on the first correspondence; if the adjusted road feature observation elements at the current time are inconsistent with the map elements, sending a map update request for the posture data at the current time to the server.

[0069] The present application also provides a map updating device, comprising:

[0070] A request receiving unit, configured to receive a map update request for a target position sent by a mobile device;

[0071] The map updating unit is used to update the map elements corresponding to the target posture according to the adjusted road feature elements corresponding to the target posture carried in the request.

[0072] The present application also provides an electronic device, comprising:

[0073] Processor; and

[0074] The memory is used to store a program of a real map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: receiving a map update request for a target posture sent by a mobile device; and updating a map element corresponding to the target posture according to an adjusted road feature element corresponding to the target posture carried in the request.

[0075] The present application also provides a map change detection method, comprising:

[0076] Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection;

[0077] Determine a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and at least one historical moment;

[0078] Adjusting the observation element at the current moment according to at least the first corresponding relationship;

[0079] According to the adjusted observation elements and map elements, the changed road characteristic elements are determined.

[0080] The present application also provides a map change detection device, comprising:

[0081] An image acquisition unit, used for acquiring road environment image data;

[0082] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0083] A multi-frame element registration unit, used to determine a first corresponding relationship between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and at least one historical moment;

[0084] An element adjustment unit, configured to adjust the observation element at the current moment at least according to the first corresponding relationship;

[0085] The changed element determination unit is used to determine the changed road characteristic elements according to the adjusted observation elements and map elements.

[0086] The present application also provides a mobile device, comprising:

[0087] Image acquisition device;

[0088] Processor; and

[0089] The memory is used to store a program for implementing a map change detection method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting road environment image data and determining the posture data of the mobile device at the time of image collection; determining the first correspondence between the road feature observation element of the road segment and the map element based on the image data and the posture data at the current time and at least one historical time; adjusting the observation element at the current time at least based on the first correspondence; and determining the changed road feature element based on the adjusted observation element and map element.

[0090] The present application also provides a new building detection method, including:

[0091] Collect environmental image data and determine the position and posture data of the mobile device at the time of image collection;

[0092] Determine a first correspondence between an environmental feature observation element of a local area and a map element based on the image data and the position data at a current moment and at least one historical moment;

[0093] According to the first corresponding relationship, adjust the observation element at the current moment;

[0094] Determine the location of new buildings based on the adjusted observation elements and map elements.

[0095] The present application also provides a new building detection device, including:

[0096] An image acquisition unit, used for acquiring environmental image data;

[0097] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0098] A multi-frame element registration unit, used to determine a first corresponding relationship between an environmental feature observation element of a local area and a map element based on the image data and the position data at a current moment and at least one historical moment;

[0099] An element adjustment unit, used for adjusting the observation element at the current moment according to the first corresponding relationship;

[0100] A new building location determination unit is added to determine the location of new buildings based on the adjusted observation elements and map elements.

[0101] The present application also provides a mobile device, comprising:

[0102] Image acquisition device;

[0103] Processor; and

[0104] The memory is used to store a program for implementing a newly added building detection method. After the device is powered on and runs the program of the method through the processor, the following steps are performed: collecting environmental image data and determining the posture data of the mobile device at the time of image collection; determining the first correspondence between the environmental feature observation elements of the local area and the map elements based on the image data and the posture data at the current time and at least one historical time; adjusting the observation elements at the current time based on the first correspondence; and determining the location of the newly added building based on the adjusted observation elements and map elements.

[0105] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned various methods.

[0106] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned various methods.

[0107] Compared with the prior art, this application has the following advantages:

[0108] The map updating system provided in the embodiment of the present application collects road environment image data through a mobile device to determine the posture data of the mobile device at the time of image collection; determines the first correspondence between the road feature observation elements of the road segment and the map elements according to the image data and the posture data at the current time and multiple historical times; adjusts the road feature observation elements at the current time at least according to the first correspondence; if the adjusted road feature observation elements at the current time are inconsistent with the map elements, a map update request for the posture data at the current time is sent to the server; the server updates the map elements corresponding to the posture data at the current time according to the adjusted road feature observation elements at the current time; this processing method allows the use of a relatively low-cost camera equipped on a general mobile device as an environmental perception sensor and a relatively low-cost positioning device, and by improving the accuracy of local relative measurement, compensates for the insufficient measurement accuracy of the camera sensor and the positioning device, so as to realize real-time and accurate change detection of map data on the mobile device side, so as to update the map data; therefore, it can effectively take into account both the lower cost of the mobile device and the higher accuracy of map updates.

[0109] The newly added building detection method provided in the embodiment of the present application collects environmental image data and determines the posture data of the mobile device at the time of image collection; determines the first correspondence between the environmental feature observation elements of the local area and the map elements according to the image data and the posture data at the current time and at least one historical time; adjusts the observation elements at the current time according to the first correspondence; determines the location of the newly added building according to the adjusted observation elements and map elements; this processing method enables the use of a low-cost camera equipped with a general mobile device as an environmental perception sensor and a low-cost positioning device, and improves the accuracy of local relative measurement to make up for the insufficient measurement accuracy of the camera sensor and the positioning device, so as to realize the detection of newly added buildings on the mobile device side; therefore, it can effectively take into account the lower cost of mobile devices and the higher accuracy of newly added building detection. Timely and accurate detection of newly added buildings can determine the location of illegal buildings on the basis of ensuring timeliness, which is of great significance to the protection of cultivated land and the construction management of cities. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] Figure 1 is a structural schematic diagram of an embodiment of a map updating system provided by the present application;

[0111] Figure 2 is a schematic diagram of a scenario of an embodiment of a map updating system provided by the present application;

[0112] Figure 3 is a schematic diagram of a road feature map of an embodiment of a map updating system provided by the present application;

[0113] Figure 4 is a schematic diagram of device interaction in an embodiment of a map updating system provided by the present application;

[0114] Figure 5 This is a specific flow chart of a mobile device of an embodiment of a map updating system provided by the present application;

[0115] Figure 6 is a schematic diagram of fuzzy registration of an embodiment of a map updating system provided by the present application;

[0116] Figure 7 It is a local optimization graph model of an embodiment of a map updating system provided by the present application;

[0117] Figure 8 is a schematic diagram of the registration effect of an embodiment of a map updating system provided by the present application;

[0118] Fig. 9 is a flow chart of an embodiment of a map updating method provided by the present application;

[0119] Fig.10 is a specific flow chart of an embodiment of a map updating method provided by the present application;

[0120] Fig.11 is a specific flow chart of an embodiment of a map updating method provided by the present application;

[0121] Fig.12 is a schematic diagram of an embodiment of a map updating device provided by the present application;

[0122] Fig.13 It is a schematic diagram of an embodiment of a mobile device provided by the present application. DETAILED DESCRIPTION

[0123] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0124] In the present application, a map updating system, method and device, an electronic device, and a mobile device are provided. In the following embodiments, a vehicle is taken as an example to explain various solutions in detail one by one.

[0125] First embodiment

[0126] Please refer to Figure 1 , which is a structural diagram of an embodiment of a map updating system provided by the present application. A map updating system provided by the present application includes: a mobile device 1 and a server 2.

[0127] The mobile device 1 includes, but is not limited to, unmanned vehicles, mobile robots, drones and other movable devices, and may also be a manned mobile device equipped with an environmental perception sensor, etc. The mobile device 1 may use a camera as an environmental perception sensor to capture images of the surrounding environment and determine characteristic elements of the surrounding environment based on the image data. The mobile device 1 may also use a positioning device that is generally equipped in an autonomous driving vehicle to determine the position data of the mobile device in real time. It can be seen that compared to a dedicated high-performance mapping vehicle (which uses a laser radar or the like to perceive the surrounding environment), the mobile device 1 has a lower hardware cost.

[0128] The server 2 can store map data, which may include road feature data. In this embodiment, the server is equipped with a database management system, and the map data is stored in a map database. The database can store map data of multiple smaller areas separately, or it can store map data of a city or a country completely.

[0129] Please refer to Figure 2 , which is a schematic diagram of a scenario of a map updating system embodiment provided by the present application. In this embodiment, a high-performance map collecting vehicle ( Figure 2 The first mobile device in the example) uses high-performance sensors such as laser radar to obtain road environment information, and uses high-precision positioning equipment to determine vehicle posture information. The road environment information is spatially combined into a map based on the posture data, and the map data is stored in the cloud. This stage is the map generation stage. Since the actual road is constantly changing, the map needs to be maintained and updated after production. Therefore, the mobile device 1 ( Figure 2 The second mobile device in the map) performs map change detection. The mobile device 1 can make the latest observation of the actual road surface through the camera, and adjust the observation data of the latest frame of image according to the observation elements of multiple frames of images accumulated over a period of time (i.e., the observation elements of the road segment), thereby improving the accuracy of local relative measurement to make up for the insufficient accuracy of camera sensors and low-cost positioning equipment in measurement. The mobile device 1 can align the existing map and the adjusted latest observation data according to the vehicle posture information, so as to realize real-time and accurate change detection of map data on the vehicle side. When the mobile device 1 detects that the map data has changed, it sends a map update request to the server 2; the server 2 updates the map elements of the corresponding position according to the adjusted road feature observation elements of the latest frame, so as to realize the update of cloud map data. The third mobile device can be an unmanned vehicle, which performs device positioning processing according to the real-time updated map data.

[0130] As shown in Figure 3, the map may contain lane lines, lane markings, curbs, traffic lights, street lights, trees, etc. When storing the map, only the feature locations ( Figure 4 black dots in the figure), feature categories and other information to ensure the compactness of map data.

[0131] Please refer to Figure 4 , which is a device interaction diagram of an embodiment of a map updating system provided by the present application. In this embodiment, the mobile device 1 is used to collect road environment image data and determine the posture data of the mobile device at the time of image collection; according to the image data and the posture data at the current time and at least one historical time, determine the first correspondence between the road feature observation element of the road segment and the map element; according to the first correspondence, adjust the road feature observation element at the current time; if the adjusted road feature observation element at the current time is inconsistent with the map element, send a map update request for the posture data at the current time to the server; the server 2 is used to receive the map update request, and according to the adjusted road feature observation element at the current time, update the map element corresponding to the posture data at the current time.

[0132] Depend on Figure 4 It can be seen that the mobile device 1 of this embodiment can determine whether to update the map element through the following steps:

[0133] Step 1: Collect road environment image data; and determine the position data of the mobile device at the time of image collection.

[0134] In this embodiment, the image acquisition device (such as a camera) equipped by the mobile device 1 acquires the image of the environment in which the current vehicle is located. The camera here can be a binocular camera, a surround camera, etc. In specific implementation, the image acquisition frequency can be determined according to application requirements, such as acquiring 50 frames of image data per second, etc. In this embodiment, the latest acquired image data is referred to as the image data of the current frame (current moment), and the image data before the latest frame of image data is referred to as the image data of the historical frame (historical moment).

[0135] In this embodiment, the positioning device equipped by the mobile device 1 provides the posture information of the vehicle at the time of collecting each frame of image data. In specific implementation, the mobile device positioning technology based on the road feature map can be used to determine the device posture data in real time. This positioning technology allows the use of high-performance equipment or complex methods to obtain a road feature map with a preset clarity level in advance. During real-time positioning, the road environment features are detected in real time, and then the observed features are aligned with the map to obtain the posture data of the mobile device. Since this positioning technology belongs to the more mature existing technology, it will not be repeated here.

[0136] Step 2: Determine the first correspondence between the road feature observation elements of the road segment and the map elements based on the image data and the posture data at the current moment and multiple adjacent historical moments.

[0137] The mobile device 1 can adjust the observation elements of the latest frame image based on the continuous 3D road features (road feature observation elements of the road segment) of multiple frame images (such as 10 frames of images) of a section of road collected over a period of time, thereby improving the accuracy of local relative measurement, thereby compensating for the insufficient measurement accuracy of camera sensors and low-cost positioning equipment.

[0138] In this embodiment, the road feature observation elements of the road segment can be determined by the following processing procedure. First, for each image acquisition moment, a 3D observation of the environment corresponding to the moment can be provided based on multiple frames of image data acquired at the moment (such as two frames of images acquired by a binocular camera) through a stereoscopic vision method, which can also be called environmental point cloud data. At the same time, feature extraction is performed on the camera 2D image, where the features refer to the features of the corresponding map elements, such as lane lines, lane markings, road facilities, etc. Then, the 2D image and 3D observation can be fused to obtain the 3D features of the current road, i.e., the observation elements. According to the device posture data at each image acquisition moment, multiple frames of 3D road features of the current road are spliced ​​together to form a continuous 3D feature of a section of road, i.e., the road feature observation elements of the road segment, which can also be called map segment observations or cumulative observation elements.

[0139] The first correspondence is a correspondence between observation elements and map elements obtained by directly matching road feature observation elements of the road segment with map elements according to the position data. The correspondence may include only part of the accumulated observation elements, which are elements that can be directly registered with map elements. The part of the accumulated observation elements not included in the first correspondence may be referred to as elements that are not directly registered.

[0140] In one example, the 3D road features of the current road of a fixed number of frames can be spliced ​​together to form the road feature observation elements of the road segment. With this processing method, the first correspondence can be quickly determined, thereby improving the speed of map data update. However, to ensure the comprehensiveness of the current observation, it is necessary to set the fixed number of frames to a larger value (such as 50 frames), which may reduce the compactness of the current observation, so that the observation elements at the current moment are adjusted according to the observation elements at an earlier moment, thereby reducing the accuracy of the adjusted observation elements at the current moment, thereby reducing the accuracy of map data update.

[0141] In another example, the first correspondence is determined based on the image data and the pose data at the current moment and multiple adjacent historical moments by dynamically stitching multiple frames of observation elements, so that the registration level coefficient is greater than the first coefficient threshold and less than the second coefficient threshold. The registration level coefficient can be determined based on the number of registration elements. The number of registration elements can be the number of observation elements in the first correspondence. This processing method can ensure the comprehensiveness and compactness of the current observation, thereby ensuring the accuracy of map data update.

[0142] In specific implementation, the registration level coefficient can also be determined according to the number of registration elements and the quality of observation elements. This processing method can effectively improve the accuracy of the registration level coefficient, thereby ensuring the accuracy of map data update.

[0143] Please refer to Figure 5 , which is a schematic diagram of a first corresponding relationship determination process of a map updating system embodiment provided by the present application. In this embodiment, the first corresponding relationship is determined by dynamic multi-frame splicing, and the specific implementation may include the following steps:

[0144] Step 2-1: For each image acquisition moment, determine the road feature observation element at that moment based on the image data at that moment.

[0145] In this embodiment, the road feature observation elements of the road segment can be determined by the following processing procedure. First, for each image acquisition moment, a 3D observation of the environment corresponding to the moment can be provided based on multiple frames of image data acquired at the moment (such as two frames of images acquired by a binocular camera) through a stereoscopic vision method, which can also be called environmental point cloud data. At the same time, feature extraction is performed on the camera 2D image, where the features refer to the features of the corresponding map elements, such as lane lines, lane markings, road facilities, etc. Then, the 2D image and 3D observation can be fused to obtain the 3D features of the current road, i.e., the observation elements. According to the device posture data at each image acquisition moment, multiple frames of 3D road features of the current road are spliced ​​together to form a continuous 3D feature of a section of road, i.e., the road feature observation elements of the road segment, which can also be called map segment observations or cumulative observation elements.

[0146] Step 2-2: Based on the posture data of the current moment and multiple adjacent historical moments, the road feature observation elements of the current moment and multiple adjacent historical moments are spliced.

[0147] In this embodiment, a preset number of road feature observation elements at multiple moments are first spliced ​​together. The preset number can be determined according to application requirements. For example, if the preset number is 11, the road feature observation elements of 11 consecutive image acquisition moments of the current moment and the adjacent 10 historical moments are spliced ​​together. In specific implementation, the road feature observation elements of any number of moments can also be spliced ​​together first.

[0148] Step 2-3: Determine a first correspondence between the spliced ​​road segment observation element and the map element; and determine an observation quality score of the observation element.

[0149] In this embodiment, the probability that the observed element is an element type is used as the observation quality score. This probability can also be called confidence, that is, the probability that an element is a category, and this probability can be directly used as the observation quality. The observation quality can be given in the link of extracting 3D elements from the image. For example, when determining that an observed element is a lane line, the probability of obtaining that the element is a lane line is 0.53, then the observation quality score of the element can be 0.53 points. In specific implementation, other methods can also be used to determine the observation quality score.

[0150] In this embodiment, according to the posture information corresponding to each frame of the mobile device 1, the spliced ​​3D feature segment is preliminarily aligned with the map to obtain a first corresponding relationship. In specific implementation, the element alignment can directly adopt the method of finding the minimum root mean square error.

[0151] Step 2-4: Determine the registration level coefficient according to the number of observation elements included in the first correspondence and the observation quality scores of the observation elements.

[0152] In this embodiment, the registration level coefficient is determined according to the number of registration elements and the quality of observation elements, and the registration level coefficient can be calculated using the following formula:

[0153]

[0154] Where reg is the registration level coefficient; pi is the i-th element on the map; qi is the current observation corresponding to the i-th element; ai is the weight of the i-th element, that is, the observation quality score of the i-th element; I(*) is the indicator function, which takes 1 when the independent variable is 0, and all other values ​​of the independent variable are 0. It can be seen that the more overlapping elements between the observed elements and the map elements, the larger reg is.

[0155] Step 2-5: If the registration level coefficient is less than the first coefficient threshold, then add and splice the observation elements of at least one historical moment; if the registration level coefficient is greater than the second coefficient threshold, then clear the observation elements of at least one historical moment.

[0156] Step 2-6: According to the adjusted observation elements, re-determine the first correspondence and the registration level coefficient until the registration level coefficient is greater than or equal to the first coefficient threshold and less than or equal to the second coefficient threshold.

[0157] In this embodiment, the minimum registration level coefficient reg_min (the first coefficient threshold) is set. When reg < reg_min, it means that there are too few overlapping elements, the current observation comprehensiveness is low, and the registration result accuracy is low. Therefore, 3D elements need to be added, that is, the number of stitching frames is increased. According to the increased observation elements, the first correspondence and the registration level coefficient are re-determined, thereby realizing dynamic stitching frames until reg >= reg_min (the road segment length can ensure that the overlapping elements reach a certain number). At the same time, the maximum registration level coefficient reg_max (the second coefficient threshold) is set. When reg > reg_max, it means that there are too many overlapping elements, the current observation is not compact enough, the road segment is reduced, and the dynamic stitching frames are decreased. According to the decreased observation elements, the first correspondence and the registration level coefficient are re-determined until reg <= reg_max. In this way, the comprehensiveness (reg > reg_min) and compactness (reg < reg_max) of the current observation are ensured. Finally, if the registration level coefficient is greater than or equal to the first coefficient threshold and less than or equal to the second coefficient threshold, it means that the dynamic multi-frame stitching is completed and the first correspondence is adjusted.

[0158] Based on the image data and the pose data at the current moment and at least one historical moment, determine the road feature observation elements of the road segment and the first correspondence with the map elements.

[0159] Step 3: Adjust the observation elements at the current moment according to the first correspondence.

[0160] Map change detection is performed each time an image is captured during the vehicle's driving. Since the image acquisition device may cause the image of the current frame to be inaccurate, the detection of the current frame depends on the data of multiple adjacent historical frames before to form a map observation segment, and the observation elements of the current frame are adjusted according to the registered first correspondence. That is to say, the observation elements of the current frame are calibrated using the local relative measurement results. Among them, the local relative measurement can be the relative position relationship between elements in the observation.

[0161] In one example, step 3 can be implemented as follows: according to the first correspondence, determine the road feature elements that are not observed at the current moment. The first correspondence includes the observed elements of the map segment, and the accumulated observed elements of a section of the road may include the observed elements that are missed by the current frame image. This processing method can complete the observed elements of the current frame image; therefore, the accuracy of map element change detection can be improved.

[0162] In another example, step 3 can be implemented as follows: according to the first correspondence, the noise elements in the observed elements at the current moment are removed. The first correspondence includes the observed elements of the map segment, and the noise elements in the current frame image can be determined according to the observed elements of a section of road. By adopting this processing method, the noise elements in the current frame image can be removed; therefore, the accuracy of map element change detection can be improved.

[0163] Depend on Figure 6 It can be seen that the first corresponding relationship is formed after the above preliminary alignment, but there are still a considerable number of observed elements that do not overlap with the elements on the map. For example, due to the inaccurate positioning caused by the relatively low-cost positioning device of the mobile device 1, the map elements and the observed elements are misaligned near the T-junction. In order to solve this problem, this embodiment also performs fuzzy alignment on the unaligned observed elements in order to further optimize and adjust these observed elements. In specific implementation, the mobile device 1 may also perform the following steps:

[0164] Step 4': Determine a second correspondence between the map elements and the unregistered observation elements.

[0165] The unregistered observation elements include observation elements not included in the first correspondence relationship. The observation elements in the first correspondence relationship are removed from the multi-frame accumulated observation elements, and the remaining observation elements include unregistered observation elements.

[0166] In this embodiment, the second correspondence between the map element and the unregistered observation element can be determined based on the positioning error range. The positioning error range gives the possible deviation range of the positioning posture, which can be given by the positioning device and represents the uncertainty of positioning. This value can be determined by the noise of the positioning input quantity, which is obtained after the noise transfer calculation of the positioning calculation process. The positioning error range is a dynamic value, which can be given by the positioning device in real time.

[0167] Depend on Figure 6 It can be seen that for the unregistered observation elements, the radius b (the positioning error range) can be used as the range to search for map elements that are consistent with the observation elements on the map and record the corresponding relationship. After fuzzy registration, more observation elements establish corresponding relationships with map elements, forming a second corresponding relationship.

[0168] Step 5': According to the second corresponding relationship, adjust the unregistered observation elements at the current moment.

[0169] In one example, step 5' can be implemented as follows: according to the second correspondence, the noise data in the road feature element at the current moment is removed. The second correspondence includes the observed elements of the map segment, and the noise elements in the current frame image can be determined according to the observed elements of a section of the road. By adopting this processing method, the noise elements in the current frame image can be further removed; therefore, the accuracy of map element change detection can be improved.

[0170] In one example, step 5' may be implemented in the following manner: according to the second corresponding relationship, an orthogonal process is performed on the distorted road feature elements at the current moment.

[0171] In specific implementation, the unregistered observation elements at the current moment, such as inaccurately positioned observation elements, distorted observation elements, etc., can be adjusted according to the second correspondence through a local optimization graph model algorithm. In this embodiment, local optimization processing is performed on the observation elements in the second correspondence and the pose data corresponding to the observations. Figure 7 The figure shows the local optimization graph model of this embodiment. The landmark node is the current observation element or map element; the pose node is the pose corresponding to each frame observation provided by the positioning device; the dotted line is the observation model, which is given by the observation model of the binocular camera or the surround camera; the solid line is the landmark relative observation model. This embodiment fixes the observation elements in the first corresponding relationship established in the preliminary registration stage, and optimizes the observation elements in the second corresponding relationship established in the fuzzy registration stage; the dotted line is the motion model, which is given by the positioning device.

[0172] for Figure 7 The local optimization graph model shown can be optimized using the following cost function:

[0173] J=M+V+R

[0174] Among them, J is the cost function; M is the cost term given by the motion model; V is the cost term given by the observation model; R is the cost term given by the road sign relative to the observation model. The two terms M and V vary depending on the configuration of the positioning device and the camera, which are discussed in the relevant literature and will not be repeated here. The optimized parameters are the vehicle posture and the observed map elements.

[0175] The R term reflects the cost of registering the landmark nodes with the map nodes. In specific implementation, the following formula can be used:

[0176]

[0177] Among them, pi is the i-th element on the map; qi is the current observation at the position of the i-th element; ai is the weight of the i-th element; K(*) is the indicator function, which takes 0 when the independent variable is 0, and takes 1 for other values.

[0178] In specific implementation, the cost term V given by the observation model can be expressed as follows:

[0179]

[0180] Among them, k is the frame number, h() is the observation equation, x k is the k-th frame pose, y j is the jth landmark pose, ∑ vk (z kj -h(x k ,y j )) is the observation information matrix.

[0181] In specific implementation, the cost term M given by the motion model can be expressed as follows:

[0182]

[0183] Where k is the frame number, f() is the motion equation, and x k is the k-th frame pose, u k is the motion sensor reading, ∑ Mk (x k-1 -f(x k ,u k )) is the motion information matrix.

[0184] Step 4: If the adjusted road feature observation element at the current moment is inconsistent with the map element, a map update request for the posture data at the current moment is sent to the server.

[0185] The system provided in this embodiment can accurately align the adjusted road feature observation elements at the current moment with the map elements after optimizing the observation elements and optimizing the posture, and directly perform point-to-point change detection. If it is determined that the adjusted road feature observation elements at the current moment are inconsistent with the map elements, it means that the map elements have changed, and a map update request for the posture data at the current moment is sent to the server to update the map elements corresponding to the posture data. The map update request may include the posture data at the current moment, and may also include the adjusted road feature observation elements at the current moment.

[0186] Depend on Figure 4 It can be seen that after receiving the map update request, the server 2 of this embodiment can update the map elements corresponding to the posture data according to the adjusted road feature observation elements carried in the request.

[0187] like Figure 8 As shown in the figure, the upper part is the ideal situation when using a professional data collection vehicle, a is a map diagram, and b is the observation of the actual road surface through high-performance sensors such as laser radar. Through the vehicle posture information, a and b can be registered, so that point-to-point transformation detection can be directly performed (whether the map has changed). The lower part is the situation when (low) cost camera sensors and positioning devices are used. D is the situation when the positioning posture is inaccurate, and there is a misalignment between the map and the observation; e is the situation when there is noise or distortion in the observation; f is the situation when the observation is incomplete. The situations shown in d, e, and f cannot directly judge the map change. For example, in the case of e, the two points below are aligned, but the point above is not aligned. The above fuzzy registration can establish a corresponding relationship for the point above, and then stretch it through optimization to make the point above aligned. In summary, the system can solve the problems of incomplete observation and noise by splicing observation elements in multiple frames, and can solve the problems of inaccurate positioning, observation distortion and noise by fuzzy registration; and finally, it needs to be adjusted through the optimization model.

[0188] As can be seen from the above embodiments, the map updating system provided by the embodiments of the present application collects road environment image data through a mobile device to determine the posture data of the mobile device at the time of image collection; determines the first correspondence between the road feature observation elements of the road segment and the map elements according to the image data and the posture data at the current time and multiple adjacent historical times; adjusts the road feature observation elements at the current time at least according to the first correspondence; if the adjusted road feature observation elements at the current time are inconsistent with the map elements, a map update request for the posture data at the current time is sent to the server; the server updates the map elements corresponding to the posture data at the current time according to the adjusted road feature observation elements at the current time; this processing method enables the use of a relatively low-cost camera equipped with a general mobile device as an environmental perception sensor, and a relatively low-cost positioning device, to improve the accuracy of local relative measurement, and to compensate for the insufficient measurement accuracy of the camera sensor and the positioning device, so as to realize real-time and accurate change detection of map data on the mobile device side, so as to update the map data; therefore, it can effectively take into account both the lower cost of the mobile device and the higher accuracy of map updates.

[0189] Second embodiment

[0190] In the above embodiment, a map updating system is provided, and correspondingly, the present application also provides a map updating method, which corresponds to the mobile device side embodiment of the above method.

[0191] Please refer to Fig. 9, which is a flow chart of an embodiment of the map updating method of the present application, the execution subject of the method includes a map updating device, which can be deployed on a mobile device 1. Since the method embodiment is basically similar to the system embodiment 1, the description is relatively simple, and the relevant parts can refer to the partial description of the system embodiment 1. The method embodiment described below is only illustrative.

[0192] A map updating method of this embodiment includes:

[0193] Step S901: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection.

[0194] Step S903: Determine a first correspondence between a road feature observation element of a road segment and a map element based on the image data and the position data at the current moment and a plurality of adjacent historical moments.

[0195] In this embodiment, the first corresponding relationship is determined based on the image data and the posture data at the current moment and multiple adjacent historical moments by dynamically stitching multiple frames of observation elements, so that the registration level coefficient is greater than the first coefficient threshold and less than the second coefficient threshold; the registration level coefficient is determined based on the number of registration elements.

[0196] During specific implementation, the registration level coefficient may also be determined according to the number of registration elements and the quality of observation elements.

[0197] Please refer to Fig.10 , which is a specific flow chart of step S903 of an embodiment of the map updating method of the present application. In this embodiment, the first corresponding relationship is determined by the following steps:

[0198] Step S9031: for each image acquisition moment, determining the road feature observation element at the moment according to the image data at the moment;

[0199] Step S9033: splicing the road feature observation elements of the current moment and multiple adjacent historical moments according to the posture data of the current moment and multiple adjacent historical moments;

[0200] Step S9035: determining a first correspondence between the spliced ​​road segment observation element and the map element; and determining an observation quality score of the observation element;

[0201] The determination of the observation quality score of the observation element may be implemented in the following manner: the probability that the observation element is an element type is used as the observation quality score.

[0202] Step S9037: determining a registration level coefficient according to the number of observation elements included in the first corresponding relationship and the observation quality scores of the observation elements;

[0203] Step S9038: if the registration level coefficient is less than the first coefficient threshold, then add and stitch at least one observation element of a historical moment; if the registration level coefficient is greater than the second coefficient threshold, then remove at least one observation element of a historical moment;

[0204] Step S9039: Re-determine the first corresponding relationship and the registration level coefficient according to the spliced ​​and adjusted observation elements until the registration level coefficient is greater than or equal to the first coefficient threshold and the registration level coefficient is less than or equal to the second coefficient threshold.

[0205] Step S905: Adjust the observation element at the current moment at least according to the first corresponding relationship.

[0206] In one example, step S905 may include the following sub-steps: determining, according to the first corresponding relationship, road feature elements that are not observed at the current moment.

[0207] In an example, step S905 may further include the following sub-steps: according to the first corresponding relationship, clearing noise elements in the observation elements at the current moment.

[0208] Step S907: According to the adjusted observation elements at the current moment, the map elements corresponding to the posture data at the current moment are updated.

[0209] Please refer to Fig.11 , which is a specific flow chart of an embodiment of the map updating method of the present application. In this embodiment, the method may also include the following steps:

[0210] Step S1101: determining a second correspondence between map elements and unregistered observation elements; the unregistered observation elements include observation elements not included in the first correspondence;

[0211] Step S1103: adjusting the unregistered observation elements at the current moment at least according to the second corresponding relationship;

[0212] In one example, step S1103 may include the following sub-steps: according to the second corresponding relationship, clearing noise elements in the road feature elements at the current moment.

[0213] In one example, step S1103 may further include the following sub-step: performing orthogonal processing on the distorted road feature elements at the current moment according to the second corresponding relationship.

[0214] In specific implementation, step S1103 may be implemented in the following manner: adjusting the unaligned observation elements at the current moment according to the first corresponding relationship and the second corresponding relationship through a local optimization graph model algorithm.

[0215] Step S1105: According to the adjusted unregistered observation elements at the current moment, the map elements corresponding to the pose data at the current moment are updated.

[0216] In this embodiment, the local optimization graph model algorithm is used to adjust the unaligned observation elements at the current moment and the pose data at the current moment according to the first correspondence and the second correspondence; accordingly, step S1105 can be implemented in the following manner: based on the adjusted unaligned observation elements, the map elements corresponding to the adjusted pose data are updated.

[0217] It can be seen from the above embodiments that the map updating method provided in the embodiments of the present application collects road environment image data and determines the posture data of the mobile device at the time of image collection; determines the first correspondence between the road feature observation elements of the road segment and the map elements according to the image data and the posture data at the current time and multiple adjacent historical times; adjusts the observation elements at the current time at least according to the first correspondence; updates the map elements corresponding to the posture data at the current time according to the adjusted observation elements at the current time; this processing method enables the use of a relatively low-cost camera equipped on a general mobile device as an environmental perception sensor and a relatively low-cost positioning device, and compensates for the insufficient measurement accuracy of the camera sensor and the positioning device by improving the accuracy of local relative measurement, so as to realize real-time and accurate change detection of map data on the mobile device side, and update the map data; therefore, it can effectively take into account both the lower cost of mobile devices and the higher map update accuracy.

[0218] Third embodiment

[0219] Please refer to Fig.12 , which is a schematic diagram of an embodiment of a map updating device of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0220] A map updating device of this embodiment includes:

[0221] An image acquisition unit 1201 is used to acquire road environment image data;

[0222] A positioning unit 1203, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0223] A multi-frame element registration unit 1205, configured to determine a first correspondence between a road feature observation element of a road segment and a map element based on the image data and the position data at a current moment and a plurality of adjacent historical moments;

[0224] An element adjustment unit 1207, configured to adjust the observation element at the current moment at least according to the first corresponding relationship;

[0225] The map updating unit 1209 is used to update the map elements corresponding to the position and posture data at the current moment according to the adjusted observation elements at the current moment.

[0226] Fourth embodiment

[0227] Please refer to Fig.13 , which is a schematic diagram of an embodiment of a mobile device of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0228] A mobile device of this embodiment includes: an image acquisition device 1300, a processor 1301 and a memory 1302; the memory is used to store a program for implementing a map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: acquiring road environment image data and determining the posture data of the mobile device at the time of image acquisition; determining a first correspondence between a road feature observation element of a road segment and a map element based on the image data and the posture data at the current time and multiple adjacent historical times; adjusting the observation element at the current time at least based on the first correspondence; and updating the map element corresponding to the posture data at the current time based on the adjusted observation element at the current time.

[0229] Fifth embodiment

[0230] In the above embodiment, a map updating system is provided. Correspondingly, the present application also provides a map updating method. The method corresponds to the above method embodiment, and its execution subject is a mobile device 1. Since the method embodiment is basically similar to the method embodiment 1, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment 1. The method embodiment described below is only illustrative.

[0231] A map updating method of this embodiment includes:

[0232] Step 1: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection;

[0233] Step 2: determining a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at the current moment and multiple historical moments;

[0234] Step 3: adjusting the road feature observation element at the current moment according to at least the first corresponding relationship;

[0235] Step 4: If the adjusted road feature observation element at the current moment is inconsistent with the map element, a map update request for the posture data at the current moment is sent to the server.

[0236] Sixth embodiment

[0237] The present application also provides a map updating device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0238] A map updating device of this embodiment includes:

[0239] An image acquisition unit, used for acquiring road environment image data;

[0240] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0241] A multi-frame element registration unit, used for determining a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and a plurality of historical moments;

[0242] An element adjustment unit, configured to adjust the road feature observation element at a current moment at least according to the first corresponding relationship;

[0243] The request sending unit is used to send a map update request for the posture data at the current moment to the server if the adjusted road feature observation element at the current moment is inconsistent with the map element.

[0244] Seventh embodiment

[0245] The present application also provides a mobile device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0246] A mobile device of the present embodiment includes: an image acquisition device, a processor and a memory; the memory is used to store a program for implementing a map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: acquiring road environment image data and determining the posture data of the mobile device at the time of image acquisition; determining a first correspondence between a road feature observation element of a road segment and a map element based on the image data and the posture data at the current time and multiple historical times; adjusting the road feature observation element at the current time based on at least the first correspondence; if the adjusted road feature observation element at the current time is inconsistent with the map element, sending a map update request for the posture data at the current time to the server.

[0247] Eighth embodiment

[0248] In the above embodiment, a map updating system is provided. Correspondingly, the present application also provides a map updating method. The method corresponds to the above method embodiment, and its execution subject includes but is not limited to the server 2. Since the method embodiment is basically similar to the method embodiment 1, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment 1. The method embodiment described below is only illustrative.

[0249] A map updating method of this embodiment includes:

[0250] Step 1: Receive a map update request for a target pose from a mobile device;

[0251] Step 2: Update the map element corresponding to the target posture according to the adjusted road feature element corresponding to the target posture carried in the request.

[0252] Ninth embodiment

[0253] The present application also provides a map updating device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0254] A map updating device of this embodiment includes:

[0255] A request receiving unit, configured to receive a map update request for a target position sent by a mobile device;

[0256] The map updating unit is used to update the map elements corresponding to the target posture according to the adjusted road feature elements corresponding to the target posture carried in the request.

[0257] Tenth embodiment

[0258] The present application also provides an electronic device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0259] An electronic device of this embodiment includes: a processor and a memory; the memory is used to store a program for implementing a map updating method. After the device is powered on and the program of the method is run by the processor, the following steps are performed: receiving a map update request for a target posture sent by a mobile device; and updating the map elements corresponding to the target posture according to the adjusted road feature elements corresponding to the target posture carried in the request.

[0260] Eleventh Embodiment

[0261] In the above embodiment, a map updating system is provided. Correspondingly, the present application also provides a map change detection method. The method corresponds to the above method embodiment, and its execution subject is a mobile device 1. Since the method embodiment is basically similar to the method embodiment 1, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment 1. The method embodiment described below is only illustrative.

[0262] A map change detection method of this embodiment includes:

[0263] Step 1: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection;

[0264] Step 2: determining a first correspondence between a road feature observation element of a road segment and a map element according to the image data and the position data at the current moment and multiple historical moments;

[0265] Step 3: adjusting the road feature observation element at the current moment according to at least the first corresponding relationship;

[0266] Step 4: Determine the changed road feature elements based on the adjusted observation elements and map elements.

[0267] Twelfth Embodiment

[0268] The present application also provides a map change detection device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0269] A map change detection device of this embodiment includes:

[0270] An image acquisition unit, used for acquiring road environment image data;

[0271] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0272] A multi-frame element registration unit, used to determine a first corresponding relationship between a road feature observation element of a road segment and a map element according to the image data and the position data at a current moment and at least one historical moment;

[0273] An element adjustment unit, configured to adjust the observation element at the current moment at least according to the first corresponding relationship;

[0274] The changed element determination unit is used to determine the changed road characteristic elements according to the adjusted observation elements and map elements.

[0275] Thirteenth Embodiment

[0276] The present application also provides a mobile device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0277] A mobile device of this embodiment includes: an image acquisition device, a processor and a memory; the memory is used to store a program for implementing a map change detection method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: acquiring road environment image data and determining the posture data of the mobile device at the time of image acquisition; determining a first correspondence between a road feature observation element of a road segment and a map element based on the image data and the posture data at the current time and at least one historical time; adjusting the observation element at the current time based on at least the first correspondence; and determining the changed road feature element based on the adjusted observation element and map element.

[0278] Fourteenth Embodiment

[0279] In the above-mentioned embodiment, a map updating system is provided, and correspondingly, the present application also provides a new building detection method.

[0280] Under the existing technology, remote sensing data is obtained by observing the area to be inspected through remote sensing technology, and new buildings are detected based on remote sensing images. At present, satellite data can be obtained mainly from remote sensing technology platforms, and remote sensing instruments and information are received, processed and analyzed. However, the inventors of this application have found that the existing technology has at least a problem of high cost of new building detection equipment.

[0281] The new building detection method provided by the present application corresponds to the embodiment of the above method, and its execution subject includes but is not limited to a drone. Since the method embodiment is basically similar to the method embodiment 1, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment 1. The method embodiment described below is only illustrative.

[0282] A new building detection method of this embodiment includes:

[0283] Step 1: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection.

[0284] In this embodiment, the mobile device is a drone, which can collect road environment image data through a low-profile camera sensor and determine the position data of the drone at the time of image collection through a low-profile positioning device.

[0285] Step 2: Determine the first correspondence between the environmental feature observation elements of the local area and the map elements based on the image data and the posture data at the current moment and at least one historical moment.

[0286] The map elements can be generated by satellite remote sensing technology. Generating city maps by satellite remote sensing technology is a relatively mature existing technology and will not be described in detail here.

[0287] Step 3: Adjust the road feature observation element at the current moment at least according to the first corresponding relationship.

[0288] Step 4: Determine the location of the new building based on the adjusted observation elements and map elements.

[0289] If the adjusted observation elements include observation elements that are not included in the map elements corresponding to the same posture, the observation elements can be used as newly added buildings, thereby determining the location of the newly added buildings.

[0290] It can be seen from the above embodiments that the newly added building detection method provided by the embodiment of the present application collects environmental image data and determines the posture data of the mobile device at the time of image collection; determines the first correspondence between the environmental feature observation elements of the local area and the map elements according to the image data and the posture data at the current moment and at least one historical moment; adjusts the observation elements at the current moment according to the first correspondence; determines the location of the newly added building according to the adjusted observation elements and map elements; this processing method enables the use of a low-cost camera equipped with a general mobile device as an environmental perception sensor, and a low-cost positioning device, by improving the accuracy of local relative measurement, to compensate for the insufficient measurement accuracy of the camera sensor and the positioning device, so as to realize the detection of newly added buildings on the mobile device side; therefore, it can effectively take into account the lower cost of mobile devices and the higher accuracy of newly added building detection. Timely and accurate detection of newly added buildings can ensure the location of illegal buildings on the basis of timeliness, which is of great significance to the protection of cultivated land and the construction management of cities.

[0291] Fifteenth Embodiment

[0292] The present application also provides a newly added building detection device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0293] A new building detection device of this embodiment includes:

[0294] An image acquisition unit, used for acquiring environmental image data;

[0295] A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition;

[0296] A multi-frame element registration unit, used to determine a first corresponding relationship between an environmental feature observation element of a local area and a map element based on the image data and the position data at a current moment and at least one historical moment;

[0297] An element adjustment unit, used for adjusting the observation element at the current moment according to the first corresponding relationship;

[0298] A new building location determination unit is added to determine the location of new buildings based on the adjusted observation elements and map elements.

[0299] Sixteenth Embodiment

[0300] The present application also provides a mobile device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0301] A mobile device of this embodiment includes: an image acquisition device, a processor and a memory; the memory is used to store a program for implementing a newly added building detection method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting environmental image data and determining the posture data of the mobile device at the time of image acquisition; determining the first corresponding relationship between the environmental feature observation elements of the local area and the map elements based on the image data and the posture data at the current time and at least one historical time; adjusting the observation elements at the current time based on the first corresponding relationship; determining the location of the newly added building based on the adjusted observation elements and map elements.

[0302] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0303] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0304] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0305] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0306] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A map updating system, It is characterized in that include: A mobile device is used to collect road environment image data and determine the position and posture data of the mobile device at the image collection time; for each image collection time, determine the road feature observation element at the time according to the image data at the time; splice the road feature observation elements at the current time and multiple adjacent historical moments according to the position and posture data at the current time and multiple adjacent historical moments; Determine a first correspondence between the spliced ​​road feature observation element and the map element; adjust the road feature observation element at the current moment at least according to the first correspondence; If the adjusted road feature observation element at the current moment is inconsistent with the map element, a map update request for the posture data at the current moment is sent to the server; The server is used to receive the map update request and update the map elements corresponding to the posture data at the current moment according to the adjusted road feature observation elements at the current moment.

2. A map updating method, It is characterized in that include: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the current moment and multiple adjacent historical moments according to the posture data at the current moment and multiple adjacent historical moments; Determine the first correspondence between the spliced ​​road feature observation elements and the map elements; Adjusting the observation element at the current moment according to at least the first corresponding relationship; According to the adjusted observation element at the current moment, the map element corresponding to the posture data at the current moment is updated.

3. The method according to claim 2, It is characterized in that The adjusting the road characteristic element at the current moment at least according to the first corresponding relationship includes: According to the first corresponding relationship, the road characteristic elements that are not observed at the current moment are determined.

4. The method according to claim 2, It is characterized in that The adjusting the road characteristic element at the current moment at least according to the first corresponding relationship includes: According to the first corresponding relationship, the noise elements in the observation elements at the current moment are removed.

5. The method according to claim 2, It is characterized in that Also includes: determining a second correspondence between the map element and the unregistered observation element; The unregistered observation elements include observation elements not included in the first correspondence; At least according to the second correspondence, adjusting the unregistered observation elements at the current moment; According to the adjusted unregistered observation elements at the current moment, the map elements corresponding to the pose data at the current moment are updated.

6. The method according to claim 5, It is characterized in that The step of adjusting the unregistered observation elements at the current moment at least according to the second corresponding relationship includes: According to the second corresponding relationship, the noise elements in the road feature elements at the current moment are removed.

7. The method according to claim 5, It is characterized in that The step of adjusting the unregistered observation elements at the current moment at least according to the second corresponding relationship includes: According to the second corresponding relationship, orthogonal processing is performed on the distorted road feature elements at the current moment.

8. The method according to any one of claims 5 to 7, It is characterized in that The step of adjusting the unregistered observation elements at the current moment at least according to the second corresponding relationship includes: Through the local optimization graph model algorithm, the unaligned observation elements at the current moment are adjusted according to the first correspondence and the second correspondence.

9. The method according to claim 8, It is characterized in that By using a local optimization graph model algorithm, the unregistered observation elements at the current moment and the pose data at the current moment are adjusted according to the first correspondence relationship and the second correspondence relationship; According to the adjusted unregistered observation elements, the map elements corresponding to the adjusted pose data are updated.

10. The method according to claim 2, It is characterized in that By dynamically stitching together multiple frames of observation elements, the first corresponding relationship is determined based on the image data and the posture data at the current moment and multiple adjacent historical moments, so that the registration level coefficient is greater than the first coefficient threshold and less than the second coefficient threshold; the registration level coefficient is determined based on the number of registration elements.

11. The method according to claim 10, It is characterized in that The registration level coefficient is determined according to the number of registration elements and the quality of observation elements.

12. The method according to claim 11, It is characterized in that Also includes: Determine the observation quality score of the observation element; Determine the registration level coefficient according to the number of observation elements included in the first corresponding relationship and the observation quality scores of the observation elements; If the registration level coefficient is less than the first coefficient threshold, then add and splice the observation element of at least one historical moment; if the registration level coefficient is greater than the second coefficient threshold, then remove the observation element of at least one historical moment; According to the spliced ​​and adjusted observation elements, the first corresponding relationship and the registration level coefficient are re-determined until the registration level coefficient is greater than or equal to the first coefficient threshold and the registration level coefficient is less than or equal to the second coefficient threshold.

13. The method according to claim 12, It is characterized in that Determining the observation quality score of the observation element includes: The probability that the observed element is an element type is used as the observation quality score.

14. A map updating method, It is characterized in that include: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the current moment and multiple adjacent historical moments according to the posture data at the current moment and multiple adjacent historical moments; Determine the first correspondence between the spliced ​​road feature observation elements and the map elements; Adjusting the road feature observation element at the current moment at least according to the first corresponding relationship; If the adjusted road feature observation element at the current moment is inconsistent with the map element, a map update request for the posture data at the current moment is sent to the server.

15. A map updating method, It is characterized in that include: Receiving a map update request for a target position sent by a mobile device; updating the map element corresponding to the target posture according to the adjusted road feature element corresponding to the target posture carried in the request; The request is sent in the following manner: The mobile device collects road environment image data and determines the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the target moment and multiple adjacent historical moments according to the posture data at the target moment and multiple adjacent historical moments; determine the first corresponding relationship between the spliced ​​road feature observation elements and the map elements; Adjusting the road feature observation element at the target time at least according to the first correspondence; If the adjusted road feature observation element at the target time is inconsistent with the map element, a map update request for the target position at the target time is sent to the server.

16. A map updating device, It is characterized in that include: An image acquisition unit, used for acquiring road environment image data; A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition; A multi-frame element registration unit is used to determine the road feature observation elements at each image acquisition moment according to the image data at the moment; and to splice the road feature observation elements at the current moment and multiple adjacent historical moments according to the posture data at the current moment and multiple adjacent historical moments; Determine the first correspondence between the spliced ​​road feature observation elements and the map elements; An element adjustment unit, configured to adjust the observation element at the current moment at least according to the first corresponding relationship; The map updating unit is used to update the map elements corresponding to the posture data at the current moment according to the adjusted observation elements at the current moment.

17. A mobile device, It is characterized in that include: Image acquisition device; processor; as well as The memory is used to store a program for implementing the map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting road environment image data and determining the posture data of the mobile device at the image collection time; for each image collection time, determining the road feature observation element at the time according to the image data at the time; splicing the road feature observation elements at the current time and multiple adjacent historical moments according to the posture data at the current time and multiple adjacent historical moments; Determine a first correspondence between the spliced ​​road feature observation elements and the map elements; adjust the observation elements at the current moment at least according to the first correspondence; and update the map elements corresponding to the posture data at the current moment according to the adjusted observation elements at the current moment.

18. A map updating device, It is characterized in that include: An image acquisition unit, used for acquiring road environment image data; A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition; A multi-frame element registration unit is used to determine the road feature observation elements at each image acquisition moment according to the image data at the moment; and to splice the road feature observation elements at the current moment and multiple adjacent historical moments according to the posture data at the current moment and multiple adjacent historical moments; Determine the first correspondence between the spliced ​​road feature observation elements and the map elements; An element adjustment unit, configured to adjust the road feature observation element at a current moment at least according to the first corresponding relationship; The request sending unit is used to send a map update request for the posture data at the current moment to the server if the adjusted road feature observation element at the current moment is inconsistent with the map element.

19. A mobile device, It is characterized in that include: Image acquisition device; processor; as well as The memory is used to store a program for implementing the map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting road environment image data and determining the posture data of the mobile device at the image collection time; for each image collection time, determining the road feature observation element at the time according to the image data at the time; splicing the road feature observation elements at the current time and multiple adjacent historical moments according to the posture data at the current time and multiple adjacent historical moments; Determine a first correspondence between the spliced ​​road feature observation element and the map element; adjust the road feature observation element at the current moment at least according to the first correspondence; If the adjusted road feature observation element at the current moment is inconsistent with the map element, a map update request for the posture data at the current moment is sent to the server.

20. A map updating device, It is characterized in that include: A request receiving unit, configured to receive a map update request for a target position sent by a mobile device; A map updating unit, configured to update a map element corresponding to the target posture according to the adjusted road feature element corresponding to the target posture carried in the request; The request is sent in the following manner: The mobile device collects road environment image data and determines the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the target moment and multiple adjacent historical moments according to the posture data at the target moment and multiple adjacent historical moments; determine the first corresponding relationship between the spliced ​​road feature observation elements and the map elements; Adjusting the road feature observation element at the target time at least according to the first correspondence; If the adjusted road feature observation element at the target time is inconsistent with the map element, a map update request for the target position at the target time is sent to the server.

21. An electronic device, It is characterized in that include: processor; as well as The memory is used to store a program of a real map updating method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: receiving a map update request for a target posture sent by a mobile device; updating a map element corresponding to the target posture according to an adjusted road feature element corresponding to the target posture carried in the request; The request is sent in the following manner: The mobile device collects road environment image data and determines the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the target moment and multiple adjacent historical moments according to the posture data at the target moment and multiple adjacent historical moments; determine the first corresponding relationship between the spliced ​​road feature observation elements and the map elements; Adjusting the road feature observation element at the target time at least according to the first correspondence; If the adjusted road feature observation element at the target time is inconsistent with the map element, a map update request for the target position at the target time is sent to the server.

22. A map change detection method, It is characterized in that include: Collect road environment image data and determine the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the current moment and multiple adjacent historical moments according to the posture data at the current moment and at least one adjacent historical moment; Determine the first correspondence between the spliced ​​road feature observation elements and the map elements; Adjusting the observation element at the current moment according to at least the first corresponding relationship; According to the adjusted observation elements and map elements, the changed road characteristic elements are determined.

23. A map change detection device, It is characterized in that include: An image acquisition unit, used for acquiring road environment image data; A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition; A multi-frame element registration unit is used to determine the road feature observation element at each image acquisition moment according to the image data at the moment; and to splice the road feature observation elements at the current moment and at least one adjacent historical moment according to the posture data at the current moment and at least one adjacent historical moment; Determine the first correspondence between the spliced ​​road feature observation elements and the map elements; An element adjustment unit, configured to adjust the observation element at the current moment at least according to the first corresponding relationship; The changed element determination unit is used to determine the changed road characteristic elements according to the adjusted observation elements and map elements.

24. A mobile device, It is characterized in that include: Image acquisition device; processor; as well as The memory is used to store a program for implementing the map change detection method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting road environment image data and determining the posture data of the mobile device at the image collection time; for each image collection time, determining the road feature observation element at the time according to the image data at the time; splicing the road feature observation elements at the current time and multiple adjacent historical moments according to the posture data at the current time and at least one adjacent historical moment; Determine a first correspondence between the spliced ​​road feature observation element and the map element; and adjust the observation element at the current moment at least according to the first correspondence; According to the adjusted observation elements and map elements, the changed road characteristic elements are determined.

25. A new building detection method, It is characterized in that include: Collect environmental image data and determine the position and posture data of the mobile device at the time of image collection; For each image acquisition moment, determine the road feature observation element at the moment according to the image data at the moment; splice the road feature observation elements at the current moment and multiple adjacent historical moments according to the posture data at the current moment and at least one adjacent historical moment; Determine the first correspondence between the spliced ​​environmental feature observation elements and the map elements; According to the first corresponding relationship, adjust the observation element at the current moment; Determine the location of new buildings based on the adjusted observation elements and map elements.

26. A new building detection device, It is characterized in that include: An image acquisition unit, used for acquiring environmental image data; A positioning unit, used to determine the position and posture data of the mobile device at the time of image acquisition; A multi-frame element registration unit is used to determine the road feature observation element at each image acquisition moment according to the image data at the moment; and to splice the road feature observation elements at the current moment and at least one adjacent historical moment according to the posture data at the current moment and at least one adjacent historical moment; Determine the first correspondence between the spliced ​​environmental feature observation elements and the map elements; An element adjustment unit, used for adjusting the observation element at the current moment according to the first corresponding relationship; A new building location determination unit is added to determine the location of new buildings based on the adjusted observation elements and map elements.

27. A mobile device, It is characterized in that include: Image acquisition device; processor; as well as The memory is used to store a program for implementing the newly added building detection method. After the device is powered on and the program of the method is run through the processor, the following steps are performed: collecting environmental image data and determining the posture data of the mobile device at the image collection time; for each image collection time, determining the road feature observation element at the time according to the image data at the time; splicing the road feature observation elements at the current time and multiple adjacent historical moments according to the posture data at the current time and at least one adjacent historical moment; Determine a first correspondence between the spliced ​​environmental feature observation elements and the map elements; and adjust the observation elements at the current moment according to the first correspondence; Determine the location of new buildings based on the adjusted observation elements and map elements.

Images