A map update processing method, device, system and medium

Abstract

The present invention provides a method, device, system, and medium for processing map updates. Specifically, a vehicle collects a road section image of a road section to be updated on a map, and uses a preset visual perception network to segment road surface elements and non-road surface elements from the road section image. The road section image carrying the road surface elements is uploaded to the cloud, so that the cloud generates a second vectorized image corresponding to the road surface elements. The current position coordinates of the vehicle on the map are then used to determine the pixel coordinates of the non-road surface elements on the map, thereby obtaining a first vectorized image and uploading it to the cloud, so that the cloud completes the map update processing of the road section to be updated based on the second vectorized image and the first vectorized image. The processing method provided by the present invention can improve the efficiency and accuracy of map updates.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing and analysis, and in particular to a map updating processing method, device, system and medium. Background Art

[0002] High-precision maps, a fundamental element of autonomous driving, provide the basis for autonomous vehicles' driving processes. These maps accurately represent roads and surrounding scenes in electronic form. However, actual roads and surrounding scenes are subject to change. To ensure the safety of autonomous driving, high-precision maps need to be updated, a topic of intense research interest among those skilled in the art.

[0003] In existing technology, the update of high-precision maps mainly relies on crowdsourcing methods, such as using the image information of the road section to be updated collected by all vehicles with autonomous driving functions, and then performing road surface element recognition processing on the vehicle side to obtain all road surface elements of the road section to be updated. These road elements are then vectorized to obtain a vectorized map of the road section to be updated, and then the vectorized map is uploaded to the cloud and used as a high-precision map of the road section to be updated.

[0004] However, in the existing technology, the perception processing and vectorization mapping of all road elements in the updated road section are performed on the vehicle side, resulting in low processing efficiency. In addition, in the existing technology, the vectorized map is directly fed back to the driver for use, which may lead to the problem of insufficient vectorization accuracy or the current high-precision map not needing to be updated, which may lead to the use of a low-precision vector map to update a high-precision map that does not need to be updated, which may affect the driver's driving safety. Summary of the Invention

[0005] The present invention provides a map update processing method, device, system and medium to solve the problems of low map update processing efficiency and low processing accuracy in the prior art.

[0006] A first aspect of the present invention provides a method for processing a map update, the method being applied to a vehicle side, the method comprising:

[0007] Collecting a road segment image of the road segment to be updated on the map, and using a preset visual perception network to segment the road segment image into road elements with pole signs and road elements without pole signs;

[0008] Uploading the road section image carrying the pole sign road surface element to the cloud, so that the cloud generates a second vectorized image corresponding to the pole sign road surface element;

[0009] The current position coordinates of the vehicle at the vehicle end on the map are obtained, the pixel coordinates of the non-pole sign road surface elements on the road section image are determined based on the current position coordinates of the vehicle, and coordinate transformation processing is performed on the pixel coordinates to generate a corresponding first vectorized image, and the first vectorized image is uploaded to the cloud, so that the cloud completes the update processing of the map of the road section to be updated based on the second vectorized image and the first vectorized image.

[0010] In an optional specific implementation, the segmenting of the road sign road elements and the non-road sign road elements from the road segment image using a preset visual perception network includes:

[0011] Extracting image features of the road section image using a backbone network in a preset visual perception network;

[0012] The image features are respectively input into the detection network and the segmentation network in the preset visual perception network to obtain the road section image carrying the pole sign road surface elements detected by the detection network from the image features, and to obtain the various non-pole sign road surface elements segmented from the image features by the segmentation network.

[0013] In an optional specific embodiment, the vehicle is provided with a collection device for collecting road section images, and the collection device includes: an inertial measurement sensor, a wheel speed sensor, and a global positioning navigation system;

[0014] The current position coordinates of the vehicle on the map include:

[0015] Acquiring the vehicle speed and acceleration of the vehicle end collected by the inertial measurement sensor or the wheel speed sensor, and acquiring the first position coordinates of the vehicle by integrating the vehicle speed or the acceleration;

[0016] The second position coordinates of the vehicle located by the global positioning navigation system are obtained, and the first position coordinates and the second position coordinates are used as input parameters of a preset Kalman filtering method to obtain the current position coordinates of the vehicle.

[0017] In an optional specific implementation, determining the pixel coordinates of the non-pole sign road element on the road segment image based on the current position coordinates of the vehicle, and performing coordinate transformation processing on the pixel coordinates to generate the corresponding first vectorized image includes:

[0018] Determining the position coordinates of the acquisition device based on the current position coordinates of the vehicle and the relative position between the acquisition device and the vehicle;

[0019] Determine the pixel coordinates of the non-pole sign road surface elements on the road section image according to the position coordinates of the acquisition device and a preset mapping relationship between the acquisition device and the non-pole sign road surface elements;

[0020] The pixel coordinates of each non-pole sign road surface element are converted into three-dimensional coordinates, and the three-dimensional coordinates are subjected to multi-frame fusion processing according to the acquisition time corresponding to the three-dimensional coordinates to obtain a first vectorized image corresponding to the non-pole sign road surface element.

[0021] In an optional specific embodiment, the processing method further includes:

[0022] Determining whether lane line elements in the non-pole sign road surface element are parallel;

[0023] When it is determined that the lane line elements are not parallel, the camera parameters in the acquisition device are updated according to a pre-stored camera calibration method to obtain updated camera parameters.

[0024] A second aspect of the present invention provides a method for processing map updates, which is applied to the cloud and includes:

[0025] receiving a road section image of a road section to be updated that carries road sign elements and a first vectorized image sent by a vehicle end, wherein the first vectorized image is obtained by performing coordinate transformation on pixel coordinates of non-road sign elements in the road section to be updated;

[0026] Performing triangulation processing on the road section image carrying the pole sign road surface element and its adjacent road section images to obtain a second vectorized image corresponding to the pole sign road surface element;

[0027] Aggregating the first vectorized image and the second vectorized image to obtain a vectorized map of the road section to be updated;

[0028] The vectorized map is matched with a pre-stored high-precision map library to update the map of the road section to be updated according to the matching result.

[0029] In an optional specific implementation, triangulating the road segment image carrying the pole sign road surface element and its adjacent road segment images to obtain a second vectorized image corresponding to the pole sign road surface element includes:

[0030] Performing sign classification processing on each road section image carrying the signboard road surface element to obtain a sign image set after the sign classification processing;

[0031] For each brand image in the brand image set, using a corner detection method, obtain the pixel coordinates of a first corner point of the brand image, and obtain the pixel coordinates of a second corner point of a brand image adjacent to the brand image at the acquisition time corresponding to the brand image;

[0032] The pixel coordinates of the first corner point and the pixel coordinates of the second corner point are triangulated to obtain the three-dimensional coordinates of the corner point of the sign image, and the three-dimensional coordinates of the corner point are subjected to multi-frame fusion processing according to the acquisition time corresponding to the three-dimensional coordinates of the corner point to obtain a second vectorized image corresponding to the road surface element of the sign.

[0033] In an optional specific embodiment, the method further includes:

[0034] Receiving the non-pole-sign road surface element sent by the vehicle end, and determining a target high-precision map corresponding to the road section to be updated in the pre-stored high-precision map library according to the road section identifier in the non-pole-sign road surface element;

[0035] Establishing a mapping rule between the non-pole sign road element and the target high-precision map, and calculating the current position coordinates of the vehicle at the vehicle terminal according to the mapping rule;

[0036] The mapping rules include: 2D-3D constraints.

[0037] A third aspect of the present invention provides a map update processing device, which is applied to a vehicle side and includes:

[0038] An acquisition module is used to acquire a road segment image of a road segment to be updated on a map, and to segment road surface elements with pole signs and road surface elements without pole signs from the road segment image using a preset visual perception network;

[0039] A first communication module is configured to upload the road section image carrying the pole sign road surface element to the cloud, so that the cloud generates a second vectorized image corresponding to the pole sign road surface element;

[0040] The first processing module is used to obtain the current position coordinates of the vehicle at the vehicle end on the map, determine the pixel coordinates of the non-pole sign road element on the road section image based on the current position coordinates of the vehicle, perform coordinate transformation processing on the pixel coordinates to generate a corresponding first vectorized image, and upload the first vectorized image to the cloud, so that the cloud completes the update processing of the map of the road section to be updated based on the second vectorized image and the first vectorized image.

[0041] A fourth aspect of the present invention provides a map update processing device, the processing device being applied to a cloud, and the device comprising:

[0042] The second communication module is configured to receive a road section image containing road sign elements in the road section to be updated and a first vectorized image sent by the vehicle end, wherein the first vectorized image is obtained by performing coordinate transformation on pixel coordinates of non-road sign elements in the road section to be updated;

[0043] a second processing module configured to triangulate the road segment image carrying the pole sign road surface element and its adjacent road segment images to obtain a second vectorized image corresponding to the pole sign road surface element;

[0044] The second processing module is configured to aggregate the first vectorized image and the second vectorized image to obtain a vectorized map of the road section to be updated;

[0045] The updating module is used to match the vectorized map with a pre-stored high-precision map library to update the map of the road section to be updated according to the matching result.

[0046] A fifth aspect of the present invention provides a map update processing system, comprising: a vehicle side and a cloud side;

[0047] The vehicle side executes the processing method provided in the first aspect of the present invention; the cloud side executes the processing method provided in the second aspect of the present invention.

[0048] The sixth aspect of the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, it implements the processing method provided in the first aspect and / or the second aspect of the present invention.

[0049] The present invention provides a method, device, system and medium for processing map updates. A road section image of a road section to be updated on a map is collected, and a preset visual perception network is used to segment road surface elements and non-road surface elements from the road section image; the road section image carrying the road surface elements is uploaded to the cloud, so that the cloud generates a second vectorized image corresponding to the road surface elements; the current position coordinates of the vehicle on the map are obtained, and the pixel coordinates of the non-road surface elements on the road section image are determined according to the current position coordinates of the vehicle, and the pixel coordinates are transformed to generate a corresponding first vectorized image, and the first vectorized image is uploaded to the cloud, so that the cloud completes the update based on the second vectorized image and the first vectorized image. Update processing of the map of the new road section; accordingly, the cloud receives the road section image carrying the pole-sign road surface elements and the first vectorized image in the road section to be updated sent by the vehicle side, and the first vectorized image is obtained by coordinate transformation of the pixel coordinates of the non-pole-sign road surface elements in the road section to be updated; the road section image carrying the pole-sign road surface elements and its adjacent road section images are triangulated to obtain the second vectorized image corresponding to the pole-sign road surface elements; the first vectorized image and the second vectorized image are aggregated to obtain the vectorized map of the road section to be updated; the vectorized map is matched with the pre-stored high-precision map library to update the map of the road section to be updated according to the matching processing result. Compared with the existing technology, the processing method provided by the present invention avoids placing all road surface elements of the road section to be updated on the vehicle side for processing, and uploads the road surface elements with smaller pixels and larger numbers to the cloud for vectorization processing, thereby reducing the traffic of uploading a large number of road surface elements from the vehicle side to the cloud and the calculation amount on the vehicle side, thereby improving the processing efficiency of the road section to be updated; in addition, using the processing method provided by the present invention, the initial high-precision map of the road section to be updated is stored in the cloud as a basis for judging whether the updated map is accurate, thereby improving the accuracy of the updated map of the road section to be updated. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a first embodiment of a map update processing method provided by the present invention;

[0051] Figure 2 A schematic diagram of the network structure of a preset visual perception network provided by the present invention;

[0052] Figure 3 A flowchart of a second embodiment of a map update processing method provided by the present invention;

[0053] Figure 4 A flowchart of a third embodiment of a map update processing method provided by the present invention;

[0054] Figure 5 A flowchart of a fourth embodiment of a map update processing method provided by the present invention;

[0055] Figure 6 A schematic structural diagram of a first embodiment of a map update processing device provided by the present invention;

[0056] Figure 7 This is a structural diagram of a second embodiment of a map update processing device provided by the present invention;

[0057] Figure 8 This is a structural diagram of a first embodiment of a map update processing system provided by the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0059] High-precision map updates are of great significance to the field of autonomous driving. They can improve the accuracy of autonomous driving and thus improve driver safety. In existing technologies, high-precision map updates mainly rely on crowdsourcing methods. Generally, existing technologies can implement map updates in the following two specific ways.

[0060] First: Establish an association between the road network location and its topological relationship of the crowdsourced map and the road network of the high-precision map, and then extract the lane association relationship from the obtained road network association relationship. By comparing lane features (lane shape, lane spacing, etc.), determine whether the lane features of the crowdsourced map and the lane features of the high-precision map have changed. If so, record the number of the section where the lane is located, and then use a map collection vehicle to collect maps of the section corresponding to the section number to complete the update of the high-precision map; Second: Utilize the road image information to be updated collected by all vehicles with autonomous driving functions, and then perform road surface element recognition processing on the road image information on the vehicle side to obtain all road surface elements of the road to be updated, and then vectorize these road elements to obtain a vectorized map of the road to be updated, and then upload the vectorized map to the cloud and use it as a high-precision map of the road section to be updated.

[0061] However, the first implementation method in the existing technology requires using a map collection vehicle to collect road section information and process the map only after determining that an update is needed. This results in the map used for autonomous driving not being updated in a timely manner. In addition, the map generated by the single-trip collection results of the map collection vehicle is used as the subsequent high-precision map, which cannot guarantee the accuracy of the updated map; and the second implementation method results in a large amount of data being processed on the vehicle side and the accuracy cannot be guaranteed after the map is formed. That is, this implementation method can neither guarantee the processing efficiency of the update nor the accuracy of the map update.

[0062] Based on the above technical problems, the inventive concept of the present invention is to design a map updating processing method with high processing efficiency and high accuracy.

[0063] Figure 1 This is a flow chart of the first embodiment of the map update processing method provided by the present invention. Figure 1 As shown, the map update processing method provided in this embodiment may include:

[0064] S101. Collect a road section image of a road section to be updated on a map, and use a preset visual perception network to segment road surface elements with pole signs and road surface elements without pole signs from the road section image.

[0065] It should be noted that the processing method provided in this embodiment is applied to the vehicle side, wherein the vehicle side may specifically be a vehicle-mounted device or a vehicle computer in an ordinary vehicle, and the vehicle-mounted device or the vehicle computer carries or houses a SOC chip.

[0066] The application scenario of the method provided in this embodiment may be: when the map corresponding to certain road sections on the map needs to be updated, the road section image information of the road section to be updated can be collected and processed by an ordinary vehicle driven by the user, that is, the user can complete the information collection of the road section while driving on the road section to be updated, and then complete the update processing of the map of the road section to be updated based on the road section image.

[0067] It's worth noting that image capture of the road section to be updated can be accomplished by the dashcam in a standard vehicle. It's conceivable that when a user drives a standard vehicle on a road section, the vehicle's dashcam can capture real-time image information of that section, and the dashcam can transmit the captured images to the vehicle-side for processing in real time or periodically.

[0068] It is conceivable that the road section image processed by the vehicle side when processing the road section to be updated is usually a road section image set. For the convenience of explanation, this embodiment and subsequent embodiments are explained with a single road section image.

[0069] Optionally, the above-mentioned driving recorder has a built-in low-cost, low-computing-power chip, such as a 2T int8 NPU and an 11.2K CPU; in addition, the driving recorder has a 2G memory and includes a 2-megapixel camera, an inertial measurement sensor IMU for civilian aircraft, and a global positioning navigation system GNSS with RTK technology.

[0070] Correspondingly, the vehicle side obtains the data collected by the driving recorder through the driving algorithm, such as the road section images captured by the camera in the driving recorder, and uses the chip's hardware clock to uniformly timestamp the camera, IMU and GNSS, and uses the timestamp to synchronize the data of these three sensors.

[0071] The vehicle then uses a pre-set visual perception network to process the time-synchronized road segment image to obtain road surface element information for the road segment to be updated. Optionally, the vehicle inputs the road segment image into a pre-set visual perception network for road surface element recognition and extraction. This allows the vehicle to capture both pole-signed and non-pole-signed road surface elements within the road segment image. Pole-signed road surface elements include information such as utility poles, traffic signs, and directional signs; non-pole-signed road surface elements include zebra crossings and lane markings.

[0072] The preset visual perception network mentioned above is a lightweight visual perception network that can present road section images with fewer image features, so that the recognition and processing of road surface elements in road section images can be completed with fewer image features, thereby improving processing efficiency.

[0073] Optionally, the vehicle side uses the backbone network in the preset visual perception network to extract image features of the road section image; then the image features are respectively input into the detection network and the segmentation network in the preset visual perception network to obtain the road section image carrying the pole sign pavement elements detected from the image features by the detection network, and to obtain the various non-pole sign pavement elements segmented out from the image features by the segmentation network; finally, the vehicle side uploads the road section image carrying the pole sign pavement elements to the cloud.

[0074] Figure 2 A schematic diagram of the network structure of the preset visual perception network provided by the present invention is shown as follows: Figure 2 As shown, the preset visual perception network includes: a backbone network 201, a detection network 202 and a segmentation network 203.

[0075] Among them, the backbone network can be a lightweight backbone network mobileyenet2.0, the detection network can be NanoDet, and after the segmentation, the new network can be a reference U-net and Si-Net network structure.

[0076] For example, when a road segment image is 512*512, the vehicle uses the backbone network to first upsample the lowest network layer to 32*32 resolution to predict categories. The number of categories is 7. The vehicle then uses the 32*32*7 upsampling layer to obtain a 64*64*7 upsampling layer, which is then numerically fused with the convolution results of the corresponding 32*32*96 upsampling layer of the backbone network. This process continues, ultimately resulting in a final semantic pixel-level prediction at a 128*128 resolution to obtain the image features of the road segment image.

[0077] The vehicle then inputs the obtained image features into the detection network and segmentation network. Specifically, the vehicle uses the neck and head network structures of the NanoDet detection network, with a detection category of 2, and processes the image features in 66ms using the computing power of a 2.0T int8 processor, acquiring images of road sections with road signs and road surface elements. Correspondingly, the vehicle uses the segmentation network to segment the image features using a semantic contour algorithm at a speed of 33ms, acquiring road surface elements without road signs and road surface elements.

[0078] S102: Upload the road section image carrying the pole sign and road surface elements to the cloud, so that the cloud generates a second vectorized image corresponding to the pole sign and road surface elements.

[0079] According to the processing steps provided in the above S101, the vehicle side performs road element recognition processing on the road section image set of the road section to be updated according to the above processing steps, and can obtain all the pole-sign road surface elements and non-pole-sign road surface elements of the road section to be updated; at the same time, in order to reduce the amount of data processing, the vehicle side uploads the image where the pole-sign road surface elements are located to the cloud for processing, thereby improving the processing efficiency of the road section to be updated.

[0080] Correspondingly, after the cloud receives the road section images carrying the pole sign road surface elements, it will process these road section images accordingly to obtain a second vectorized image corresponding to the pole sign road surface elements.

[0081] S103. Obtain the current position coordinates of the vehicle on the map, determine the pixel coordinates of non-pole sign road elements on the road section image based on the current position coordinates of the vehicle, perform coordinate transformation processing on the pixel coordinates to generate a corresponding first vectorized image, and upload the first vectorized image to the cloud, so that the cloud completes the update processing of the map of the road section to be updated based on the second vectorized image and the first vectorized image.

[0082] In this embodiment, the vehicle-side vectorizes the acquired non-pole sign road surface elements. First, the vehicle-side determines the pixel coordinates of the non-pole sign road surface elements on the road section image based on the current position information of the vehicle on the map. Optionally, the specific implementation method for the vehicle-side to obtain the vehicle's current position information can be: the vehicle-side uses multi-sensor fusion technology to process the data collected by multiple sensors in the driving recorder to obtain the vehicle's current position coordinates on the map, which can also be understood as obtaining the vehicle's current position coordinates on the road section to be updated.

[0083] It should be noted that the relative position relationship between the vehicle and the collection device is pre-stored in the vehicle side. The specific implementation method of this relative position relationship includes but is not limited to: it is set by technical personnel in this field, and based on this relative position relationship and the current position relationship of the vehicle, the current position coordinates of the collection device can be calculated.

[0084] In addition, the vehicle side pre-stores the mapping relationship between the collection equipment and non-pole sign road elements, and the specific implementation method of setting the mapping relationship includes but is not limited to: setting it by technical personnel in this field based on the performance of the collection equipment, and the internal physical parameters of different collection devices are inconsistent, and the corresponding mapping relationship will also be inconsistent.

[0085] In this embodiment, the vehicle side will solve the pixel coordinates of the non-pole sign road surface elements on the road section image based on the current position coordinates of the acquisition device and the aforementioned mapping relationship.

[0086] It's conceivable that the vehicle-side computer, using the aforementioned pixel coordinate calculation and processing method, can obtain the pixel coordinates of each non-pole sign road element in each road segment image. Considering the sequential acquisition times of each road segment image, the vehicle-side computer can obtain the pixel coordinates of each non-pole sign road element at different acquisition times. This means that the vehicle-side computer has updated the pixel coordinates of each non-pole sign road element.

[0087] The vehicle then performs a reverse projection transformation on the pixel coordinates of the non-pole sign road surface elements, converting them into three-dimensional coordinates in the world coordinate system. Then, based on the acquisition time corresponding to each 3D coordinate and the sequential positional relationship between the acquisition times, the multiple 3D coordinates are fused to obtain a first vectorized image corresponding to the non-pole sign road surface elements.

[0088] Considering that the first vectorized image only represents the non-pole-sign road surface elements in the road section to be updated, it is necessary to combine the first vectorized image corresponding to the non-pole-sign road surface elements with the second vectorized image corresponding to the pole-sign road surface elements to obtain the vector map corresponding to all road surface elements in the road section to be updated.

[0089] Optionally, the vehicle uploads the first vectorized image to the cloud for subsequent processing.

[0090] Correspondingly, after the cloud receives the first vectorized image, it combines it with the internally generated second vectorized image for aggregation processing, thereby generating a vectorized map corresponding to the road section to be updated.

[0091] In this embodiment, a method for processing map updates is provided, which is applied to the vehicle side. A road section image of the road section to be updated on the map is collected, and a preset visual perception network is used to segment the road section image into road elements with pole signs and non-pole signs. The road section image carrying the road elements with pole signs is uploaded to the cloud, so that the cloud generates a second vectorized image corresponding to the road elements with pole signs. The current position coordinates of the vehicle on the map are obtained, and the pixel coordinates of the non-pole signs on the road section image are determined based on the current position coordinates of the vehicle. The pixel coordinates are transformed to generate a corresponding first vectorized image, and the first vectorized image is uploaded to the cloud. Compared with the existing technology, using the processing method provided by the present invention, the road section image of the road section to be updated on the map is subjected to road element recognition processing by the vehicle side, thereby obtaining the road elements of the road section to be updated, and the non-pole signs on the road section elements are vectorized by the vehicle side, and the non-pole signs on the road section elements are uploaded to the cloud for vectorization processing by the cloud, which improves the processing efficiency of map updates.

[0092] In addition, the processing method provided in this embodiment can help complete the update of high-precision maps of national highways, urban trunk roads and some urban ordinary roads; at the same time, if there is no high-precision map library stored in the cloud, the processing method provided in this embodiment can be used to aggregate the vectorized maps generated by the bicycles to generate maps for use as high-precision maps, that is, the processing method provided in this embodiment can also be used to build high-precision maps on roads without high-precision maps.

[0093] The following combination Figure 3 How to obtain the first vectorized image of non-pole sign road elements in the map update processing method provided by the present invention is further explained. Figure 3 This is a flow chart of a second embodiment of the map update processing method provided by the present invention, as shown in FIG. Figure 3 As shown, the map update processing method provided in this embodiment may include:

[0094] S30 1. Obtain the vehicle speed and acceleration collected by the inertial measurement sensor or the wheel speed sensor, and obtain the first position coordinates of the vehicle by integrating the vehicle speed or acceleration.

[0095] It should be noted that, in order to obtain the first vectorized image of non-pole sign road elements, the vehicle side first needs to confirm the current position coordinates of the vehicle where the vehicle side is located.

[0096] It is conceivable that the equipment used to collect images of the road section to be updated includes: inertial measurement sensors, wheel speed sensors, and a global positioning navigation system. These sensors and the positioning navigation system are used to determine the vehicle's position. The combination of multiple sensors and systems can eliminate the drift caused by accumulated data from the inertial measurement sensors and the problem of global positioning navigation systems missing positioning information due to terrain.

[0097] Based on this, this embodiment requires comprehensive processing of the vehicle's position information collected by the acquisition device to obtain more accurate vehicle position information. Optionally, the vehicle needs to first initialize the data collected by the inertial measurement sensor or wheel speed sensor to obtain the vehicle's speed and acceleration. Then, by integrating the speed or acceleration, the vehicle's first position coordinates are obtained.

[0098] Then, the vehicle side executes step S302 to obtain the location information of the vehicle side based on the global positioning navigation system, and processes according to the following step S302 to obtain the current location coordinates of the vehicle.

[0099] S302: Obtain the second position coordinates of the vehicle located by the global positioning navigation system, and use the first position coordinates and the second position coordinates as input parameters of a preset Kalman filtering method to obtain the current position coordinates of the vehicle.

[0100] In this embodiment, the vehicle also needs to initialize the longitude, latitude, and altitude determined by the global positioning navigation system to obtain the vehicle's current location coordinates. This embodiment utilizes RTK technology within the global positioning navigation system to achieve global positioning. Compared to existing GPS positioning, which only guarantees local accuracy, the method provided by this embodiment can improve accuracy.

[0101] It is worth mentioning that the vehicle-side is pre-configured with a Kalman filter method for obtaining a relatively accurate current vehicle position coordinate. Therefore, the vehicle-side inputs the first and second position coordinates into the pre-configured Kalman filter method to obtain a relatively accurate current vehicle position coordinate.

[0102] S303: Determine the location coordinates of the acquisition device according to the current location coordinates of the vehicle and the relative position between the acquisition device and the vehicle.

[0103] It should be noted that the vehicle-side collection equipment is installed on the vehicle when the vehicle leaves the factory, or is pre-set according to the position of the collection equipment. Therefore, the relative position relationship between the vehicle and the collection equipment is certain, that is, the coordinates of the collection equipment relative to the vehicle are certain.

[0104] For example, the acquisition device can be a driving recorder, which is usually installed near the rearview mirror of the vehicle. It can be understood that the acquisition device has a fixed position relative to the vehicle. As long as the installation position of the driving recorder does not change, the relative position between the acquisition device and the vehicle will not change. Therefore, if the current position of the vehicle is known, the position coordinates of the acquisition device can be known.

[0105] Specifically, after obtaining the current position coordinates of the vehicle, the vehicle side uses the coordinate values ​​in each direction of the current position coordinates and the relative position based on the relative position of the vehicle and the acquisition device pre-stored in the vehicle side to perform difference calculations, obtain the difference calculation results, and use the difference calculation results as the position coordinates of the acquisition device.

[0106] After obtaining the position coordinates of the acquisition device, the vehicle side can obtain the pixel coordinates of each non-pole sign road surface element according to the following step S304.

[0107] S304: Determine the pixel coordinates of the non-pole sign road surface elements on the road section image according to the position coordinates of the acquisition device and a preset mapping relationship between the acquisition device and the non-pole sign road surface elements.

[0108] In this embodiment, the vehicle-side pre-stores a mapping relationship between the acquisition device and non-pole sign road elements. This mapping relationship can be set by a person skilled in the art based on the performance of the vehicle acquisition device. The vehicle-side can then use this mapping relationship and the current location information of the acquisition device to reverse-interpret the pixel coordinates of the non-pole sign road elements.

[0109] It is conceivable that the vehicle side needs to process the road section image set of the road section to be updated, and each road section image has a corresponding acquisition time. Therefore, after the vehicle side processes each road section image according to the aforementioned S304, the vehicle side can obtain the pixel coordinate sequence of each non-pole sign road surface element at different acquisition times.

[0110] S305: Convert the pixel coordinates of the non-pole sign road surface element into three-dimensional coordinates, and perform multi-frame fusion processing on the three-dimensional coordinates according to the acquisition time corresponding to the three-dimensional coordinates to obtain a first vectorized image corresponding to the non-pole sign road surface element.

[0111] In this embodiment, the vehicle side performs a reverse projection transformation on the pixel coordinates of each non-pole sign road surface element, that is, converts the pixel coordinates of the non-pole sign road surface element into three-dimensional coordinates in the world coordinate system.

[0112] Because each pixel coordinate has a corresponding acquisition time, each 3D coordinate also has a corresponding acquisition time. Then, according to the order of the acquisition times corresponding to the 3D coordinates, the 3D coordinates are fused together in a multi-frame process based on their positional relationship. This allows the vehicle to obtain the first vectorized image corresponding to the non-pole sign road surface elements. To facilitate the acquisition of a vectorized map corresponding to all road surface elements in the road section to be updated, the vehicle must upload the obtained first vectorized image to the cloud.

[0113] This embodiment specifically illustrates how a vehicle-side vectorization process is performed on non-pole sign road elements to obtain a first vectorized image corresponding to these non-pole sign road elements. Compared to existing technologies, this embodiment utilizes a Kalman filter method to obtain more accurate vehicle position coordinates based on road elements derived from an improved visual perception network. These coordinates are then used to determine the more accurate pixel coordinates of the non-pole sign road elements. As can be expected, the first vectorized image obtained from these more accurate pixel coordinates is also more accurate.

[0114] In an optional embodiment, in order to ensure the accuracy of vector processing of non-pole sign road elements, the vehicle side needs to determine whether the external parameters of the camera in the acquisition device need to be adjusted before converting the non-pole sign road elements into three-dimensional coordinates.

[0115] Specifically, it is determined whether the lane line elements in the non-pole sign road surface elements are parallel. When it is determined that the lane line elements are not parallel, a pre-stored camera calibration method is called to update the external parameters of the camera sensor in the acquisition device to obtain the updated external parameters of the camera sensor.

[0116] It's worth noting that converting 2D pixel coordinates in camera coordinates to 3D coordinates in the world coordinate system requires the use of the camera's intrinsic and extrinsic parameters. These parameters are typically calibrated before the camera leaves the factory, and they are affected by the camera's location. Therefore, different camera positions or geographic locations can affect the camera's extrinsic parameters. Therefore, calibrating the camera's extrinsic parameters is crucial for converting pixel coordinates to world coordinates.

[0117] It should be noted that a camera calibration method is pre-stored in the vehicle side, which is used to update the external parameters of the camera to obtain camera parameters suitable for the current scene, and the specific implementation method of the pre-stored camera calibration method includes but is not limited to: set by technical personnel in this field based on experience.

[0118] In this embodiment, the vehicle side can determine whether the camera's external parameters need to be adjusted by detecting whether the lane line elements in the non-pole sign road surface elements are parallel. Specifically, the vehicle side extracts the lane line elements in the non-pole sign road surface elements according to the road surface element identification, and at the same time, according to the aforementioned processing steps, reversely projects the lane line elements to the world coordinate system, and detects whether the lane line elements are in a parallel state.

[0119] When the vehicle determines that the current lane line elements are not in a parallel state, the pre-stored camera calibration method is called to update the camera's external parameters to obtain the updated camera parameters.

[0120] Then, the vehicle side uses the updated camera parameters to convert the pixel coordinates of non-pole road elements into three-dimensional coordinates.

[0121] In this optional embodiment, the accuracy of the first vectorized image corresponding to the non-pole sign road surface element obtained is further ensured by calibrating camera parameters. Compared to the method of calibrating the camera's focal length, center point, and distortion using Zhang Zhengyou's pinhole camera calibration principle, the camera calibration method provided in this embodiment reduces the dependence on the number of images and speeds up factory calibration, with the time required to calibrate a camera being less than 30 seconds. Moreover, the camera calibration method provided in this embodiment, in a longitude measurement experiment, uses the results of the camera calibration method to calculate the position between each frame of the two images, i.e., for two frames with bright and vanishing points. The position between the cameras of the two frames can then be measured using a measuring device, and the measured value is used as the true value. The error between the position obtained by the camera calibration method and the true value is calculated to be less than 0.7%. This proves that the camera calibration method provided in this embodiment is time-saving and highly accurate, thereby improving the accuracy of vectorized processing of non-pole sign road surface elements.

[0122] The following combination Figure 4 The following further describes how the cloud uploads images of road surface elements located on vehicle poles in the map update processing method provided by the present invention. Figure 4 This is a flow chart of a third embodiment of the map update processing method provided by the present invention, as shown in FIG. Figure 4 As shown, the map update processing method provided in this embodiment may include:

[0123] S401, receiving a road section image containing road sign elements in the road section to be updated and a first vectorized image sent by a vehicle end, wherein the first vectorized image is obtained by performing coordinate transformation on pixel coordinates of non-road sign elements in the road section to be updated.

[0124] It should be noted that the cloud and the vehicle can be connected via network communication, that is, the vehicle and the cloud can be provided with a network communication interface for sending and receiving information. Optionally, the cloud can be a server or server cluster with massive data processing capabilities.

[0125] In this embodiment, after the vehicle side processes the road section image of the road section to be updated using the processing method provided in the aforementioned embodiment, it uploads the first quantized image corresponding to the non-pole sign road surface elements to the cloud, and also uploads the road section image carrying the pole sign road surface elements separated from the road section image of the road section to be updated to the cloud.

[0126] Correspondingly, after receiving the aforementioned information, the cloud will process it according to S402 to S404 below.

[0127] S402: Triangulate the road segment image carrying the pole sign road surface element and its adjacent road segment images to obtain a second vectorized image corresponding to the pole sign road surface element.

[0128] In this embodiment, after receiving the road segment image with the pole sign pavement element uploaded by the vehicle, the cloud performs vectorization processing on the pole sign pavement element. For ease of explanation, in this embodiment and subsequent embodiments, the road segment image with the pole sign pavement element is referred to as the first road segment image.

[0129] Optionally, the cloud first needs to extract the pixel coordinates of the corner points of the sign in the first road section image. Based on the fact that the camera used in the driving recorder is a monocular camera, the cloud also needs to obtain the pixel coordinates of the corner points in the road section image adjacent to the first road section image, so as to obtain the three-dimensional coordinates of the pole sign road surface elements in the first road section image based on the pixel coordinates of the corner points in these two road section images.

[0130] After the three-dimensional coordinates of the pole sign pavement element are obtained, the three-dimensional coordinates of the pole sign pavement element can be fused according to the aforementioned fusion processing steps to obtain a second vectorized image of the pole sign pavement element.

[0131] S403: Aggregate the first vectorized image and the second vectorized image to obtain a vectorized map of the road section to be updated.

[0132] In this embodiment, the cloud aggregates the multiple first vectorized images acquired in the aforementioned embodiment and the multiple second vectorized images acquired in this embodiment. Optionally, the cloud may aggregate the multiple first vectorized images and the multiple second vectorized images in chronological order, based on the time sequence of the generation times corresponding to the respective first vector images or the generation times corresponding to the respective second vector images, to obtain a vectorized map of the road section to be updated.

[0133] In an optional embodiment, since the accuracy of a single vehicle collecting the road section to be updated is not enough, the cloud can also aggregate the results of multiple trips of a single vehicle. For example, the cloud can aggregate the vectorized maps of the road section to be updated for 30 trips.

[0134] S404: Match the vectorized map with a pre-stored high-precision map library to update the map of the road section to be updated according to the matching result.

[0135] It should be noted that a high-precision map library is pre-stored in the cloud, which is used as a basis for judging the accuracy of the map update processing of the road section to be updated. Optionally, excluding the extreme case where the road section to be updated is completely changed, when the cloud obtains the high-precision map corresponding to the road section to be updated from the high-precision map library, some road surface elements in the vectorized map of the road section to be updated are changed, and the remaining road surface elements can be completely matched with the high-precision map corresponding to the road section to be updated, it means that the current road section to be updated needs to be updated. The vectorized map of the road section to be updated can be used to update the map of the road section to be updated; on the contrary, when all road surface elements in the vectorized map of the road section to be updated do not completely match the high-precision map corresponding to the road section to be updated, or only a few road surface elements completely match the high-precision map corresponding to the road section to be updated, it indicates that the accuracy of the vectorized map of the road section to be updated is insufficient to be used to update the map of the road section to be updated.

[0136] It is conceivable that after the cloud generates a vectorized map of the road section to be updated, in order to ensure the accuracy of the map update for the updated section, the cloud needs to match the vectorized map with the high-precision map corresponding to the road section to be updated. If it is determined that some road surface elements can be matched with the high-precision map of the road section to be updated, the vectorized map of the road section to be updated is used to update the map of the road section to be updated. Optionally, the vectorized map of the road section to be updated can replace the high-precision map corresponding to the road section to be updated.

[0137] In this embodiment, the specific process of updating a map to be updated using a map update processing method in the cloud is specifically explained. The cloud receives an image of a road section carrying road sign elements in the road section to be updated, sent by the vehicle, and a first vectorized image. The first vectorized image is obtained by transforming the coordinates of the non-road sign elements in the road section to be updated at the location to be updated. The image of the road section carrying the road sign elements and its adjacent road section images are triangulated to obtain a second vectorized image corresponding to the road sign elements. The first vectorized image and the second vectorized image are aggregated to obtain a vectorized map of the road section to be updated. The vectorized map is matched with a pre-stored high-precision map library to update the map of the road section to be updated based on the matching results. Compared to existing technologies, the cloud vectorizes road sign elements with fewer pixels and a larger amount of data, thereby reducing the amount of data processing on the vehicle side. Furthermore, the cloud's data processing efficiency is higher than that of the vehicle side, thereby improving the processing efficiency of map updates. In this embodiment, a high-precision map library is pre-stored in the cloud, which can be used as a standard for judging the accuracy of the vectorized map of the road section to be updated, thereby improving the accuracy of the map update.

[0138] The following combination Figure 5 How the cloud obtains the second vectorized image corresponding to the non-pole sign road surface element in the map update processing method provided by the present invention is further explained. Figure 5 This is a flowchart of a fourth embodiment of the map update processing method provided by the present invention, as shown in FIG. Figure 5 As shown, the processing method includes:

[0139] S501: Perform sign classification processing on each road segment image carrying a pole sign road surface element to obtain a sign image set after the sign classification processing.

[0140] It should be noted that classification processing methods, such as the K-NN algorithm, are pre-stored in the cloud for image classification.

[0141] In this embodiment, after receiving road segment images containing road signs uploaded by the vehicle, the cloud performs sign classification on these images. Specifically, the cloud retrieves a pre-stored classification method and inputs the received road segment images into the classification method, thereby obtaining a set of sign images after the classification process.

[0142] It is worth mentioning that after the pole sign images are classified and processed in this embodiment, traffic signs, warning signs, etc. can be obtained, and the sign image set corresponding to each sign category contains at least 2 images.

[0143] S502. For each brand image in each brand image set, a corner detection method is used to obtain the pixel coordinates of the first corner point of the brand image, and the pixel coordinates of the second corner point of the brand image adjacent to the acquisition time corresponding to the brand image.

[0144] It should be noted that a corner point detection method is pre-stored in the cloud, which is used to detect the pixel coordinates of the corner points of the sign.

[0145] In this embodiment, the cloud performs corner detection processing on each brand image in each brand image set to obtain the first corner pixel coordinates of the current brand image. In order to convert the current brand corner information into unit coordinates, the cloud also needs to obtain the brand image adjacent to the current brand image at the time of acquisition, and use the pre-stored corner detection method to perform corner detection processing on the adjacent brand image to obtain the second corner pixel coordinates.

[0146] Then, the cloud processes the first corner point pixel coordinates and the second corner point pixel coordinates according to the following S503 to obtain a second vectorized image.

[0147] S503. Triangulate the pixel coordinates of the first corner point and the pixel coordinates of the second corner point to obtain the three-dimensional coordinates of the corner point of the sign image, and perform multi-frame fusion processing on the three-dimensional coordinates of the corner point according to the acquisition time corresponding to the three-dimensional coordinates of the corner point to obtain a second vectorized image corresponding to the road surface element of the sign.

[0148] In this embodiment, the cloud uses its pre-stored triangulation processing method to process the obtained first corner point pixel coordinates and the second corner point pixel coordinates to obtain the three-dimensional coordinates of the corner points of the current sign image.

[0149] Similar to non-pole sign road elements, the road section image carrying the pole sign road element has a corresponding collection time. Correspondingly, the sign image also has a corresponding collection time, and then the collection time of the three-dimensional coordinates of the corner points of each sign image.

[0150] Based on this, the cloud can obtain the three-dimensional coordinate sequence of the corner points of each brand image set according to the aforementioned processing method.

[0151] Furthermore, the cloud performs multi-frame fusion processing on the three-dimensional coordinates of the corner points of each sign image set according to the acquisition time sequence of the three-dimensional coordinate sequence of the corner points, thereby obtaining a second vectorized image corresponding to the sign road surface elements.

[0152] In this embodiment, the specific steps for vectorizing road sign elements on the cloud are specifically explained. That is, the cloud first classifies the signs on the received road section image, and then processes the classified sign image through corner point detection and triangulation processing methods, thereby obtaining a second vectorized image corresponding to the road sign element. Compared with the existing technology, the processing method provided by this embodiment places the road sign elements of the road section to be updated in the cloud for processing, which reduces the amount of data processed on the vehicle side and can improve the update efficiency of the road section map to be updated.

[0153] In an optional embodiment, another method for obtaining the current position coordinates of the vehicle is provided. Specifically, a non-pole-signed road surface element is received from the vehicle, and a target high-precision map corresponding to the road section to be updated is determined from a pre-stored high-precision map library based on the road section identifier in the non-pole-signed road surface element. A mapping rule is established between the non-pole-signed road surface element and the target high-precision map, and the current position coordinates of the vehicle located on the vehicle are calculated based on the mapping rule. The mapping rule includes: 2D-3D constraints.

[0154] In this optional embodiment, the vehicle side can utilize the mapping rules between the non-pole sign road surface elements in the road section image and the pre-stored high-precision map library to reversely solve the current position coordinates of the vehicle side.

[0155] Since the pre-existing high-precision map library is stored in the cloud, the vehicle can upload the non-pole sign road elements obtained in the previous step to the cloud for processing. Accordingly, after obtaining the non-pole sign road elements of the road segment image to be updated, the cloud will traverse the pre-existing high-precision map library to find the target high-precision map corresponding to the road segment to be updated.

[0156] Next, the cloud matches the non-pole-signed road elements with the target high-precision map to obtain non-pole-signed road elements that can be fully matched with the target high-precision map. Then, these professional non-pole-signed road elements are used to establish mapping rules with the target high-precision map, that is, to obtain the mapping relationship between the non-pole-signed road elements and the actual geographic location.

[0157] Among them, the mapping rule can be a 2D-3D constraint, which can be understood as the inverse projection rule from the plane image to the three-dimensional space, that is, the conversion relationship between the entire collected road section image and the target high-precision map in the same coordinate system.

[0158] Correspondingly, the cloud calculates the coordinates of the current vehicle at its actual geographical location based on the mapping rule, thereby obtaining the current location coordinates of the vehicle.

[0159] This optional embodiment specifically illustrates the steps for determining the vehicle's current location based on non-pole sign road elements uploaded by the vehicle. The processing method provided by this optional embodiment broadens the scope of the present invention's application scenarios, making it applicable to a wide range of situations. Even without a global positioning navigation system, the vehicle's current location can still be determined.

[0160] Figure 6 This is a structural diagram of a first embodiment of a map update processing device provided by the present invention, as shown in FIG. Figure 6 As shown, the processing device 600 includes: a collection module 601 , a first communication module 602 and a first processing module 603 .

[0161] Among them, the acquisition module 601 is used to collect the road section image of the road section to be updated on the map, and use the preset visual perception network to segment the road section elements and non-road section elements from the road section image; the first communication module 602 is used to upload the road section image carrying the road section elements to the cloud, so that the cloud generates a second vectorized image corresponding to the road section elements; the first processing module 603 is used to obtain the current position coordinates of the vehicle on the map, determine the pixel coordinates of the non-road section elements on the road section image according to the current position coordinates of the vehicle, and perform coordinate transformation processing on the pixel coordinates to generate the corresponding first vectorized image, and upload the first vectorized image to the cloud, so that the cloud completes the update processing of the map of the road section to be updated based on the second vectorized image and the first vectorized image.

[0162] Optionally, the acquisition module 601 is specifically configured to:

[0163] Utilize the backbone network in the preset visual perception network to extract image features of the road section image;

[0164] The image features are respectively input into the detection network and segmentation network in the preset visual perception network to obtain the road section image carrying the pole sign pavement elements detected by the detection network from the image features, and to obtain the various non-pole sign pavement elements segmented by the segmentation network from the image features.

[0165] Optionally, the first processing module 603 is specifically configured to:

[0166] Obtaining the vehicle speed and acceleration collected by an inertial measurement sensor or a wheel speed sensor, and obtaining the first position coordinates of the vehicle by integrating the vehicle speed or acceleration;

[0167] The second position coordinates of the vehicle located by the global positioning navigation system are obtained, and the first position coordinates and the second position coordinates are used as input parameters of a preset Kalman filtering method to obtain the current position coordinates of the vehicle.

[0168] Optionally, the first processing module 603 is further configured to:

[0169] Determine the location coordinates of the acquisition device based on the current location coordinates of the vehicle and the relative position between the acquisition device and the vehicle;

[0170] According to the position coordinates of the acquisition device, based on the preset mapping relationship between the acquisition device and the non-pole sign road surface elements, the pixel coordinates of the non-pole sign road surface elements on the road section image are determined;

[0171] The pixel coordinates of the non-pole sign road surface elements are converted into three-dimensional coordinates, and the three-dimensional coordinates are subjected to multi-frame fusion processing according to the acquisition time corresponding to the three-dimensional coordinates to obtain a first vectorized image corresponding to the non-pole sign road surface elements.

[0172] Optionally, the first processing module 603 is further configured to:

[0173] Determine whether lane line elements in non-pole road elements are parallel;

[0174] When it is determined that the lane line elements are not parallel, the camera parameters in the acquisition device are updated according to a pre-stored camera calibration method to obtain updated camera parameters.

[0175] The map update processing device provided in this embodiment has similar principles and technical effects to those of the above-mentioned map update processing method, and will not be described in detail here.

[0176] Figure 7 This is a structural diagram of a second embodiment of a map update processing device provided by the present invention, as shown in FIG. Figure 7 As shown, the processing device 700 includes: a second communication module 701 , a second processing module 702 and an updating module 703 .

[0177] Among them, the second communication module 701 is used to receive the road section image carrying the pole-sign pavement elements and the first vectorized image in the road section to be updated sent by the vehicle end, and the first vectorized image is obtained by coordinate transformation of the pixel coordinates of the non-pole-sign pavement elements in the road section to be updated; the second processing module 702 is used to triangulate the road section image carrying the pole-sign pavement elements and its adjacent road section images to obtain the second vectorized image corresponding to the pole-sign pavement elements; the second processing module 702 is used to aggregate the first vectorized image and the second vectorized image to obtain the vectorized map of the road section to be updated; the update module 703 is used to match the vectorized map with the pre-stored high-precision map library to update the map of the road section to be updated according to the matching processing results.

[0178] Optionally, the second processing module 702 is specifically configured to:

[0179] Performing sign classification processing on each road section image carrying a signboard pavement element to obtain a signboard image set after the signboard classification processing;

[0180] For each brand image in each brand image set, a corner detection method is used to obtain the pixel coordinates of the first corner point of the brand image, and the pixel coordinates of the second corner point of the brand image adjacent to the acquisition time corresponding to the brand image;

[0181] The pixel coordinates of the first corner point and the second corner point are triangulated to obtain the three-dimensional coordinates of the corner points of the sign image. The three-dimensional coordinates of the corner points are then subjected to multi-frame fusion processing according to the acquisition time corresponding to the three-dimensional coordinates of the corner points to obtain a second vectorized image corresponding to the road surface elements of the sign.

[0182] Optionally, the second processing module 702 is further configured to:

[0183] Receive the non-pole sign road surface elements sent by the vehicle end, and determine the target high-precision map corresponding to the road section to be updated in the pre-stored high-precision map library according to the road section identifier in the non-pole sign road surface elements;

[0184] Establish mapping rules between non-pole sign road elements and target high-precision maps, and calculate the current position coordinates of the vehicle at the vehicle end based on the mapping rules;

[0185] The mapping rules include: 2D-3D constraints.

[0186] The map update processing device provided in this embodiment has similar principles and technical effects to those of the above-mentioned map update processing method, and will not be described in detail here.

[0187] Figure 8 This is a structural diagram of a first embodiment of a map update processing system provided by the present invention, as shown in FIG. Figure 8 As shown, the processing system 800 includes: a vehicle side 801 and a cloud side 802 .

[0188] Among them, the vehicle side 801 executes the processing methods provided by the aforementioned embodiments 1 to 2; the cloud side 802 executes the processing methods provided by the aforementioned embodiments 3 to 4 to update the map of the road section to be updated on the map.

[0189] The map update processing system provided by this embodiment can provide the map update processing method of the present invention. Among them, the vehicle side can collect the road section image of the road section to be updated on the map, and use the preset visual perception network to segment the road section elements and non-road section elements from the road section image; upload the road section image carrying the road section elements to the cloud, so that the cloud generates a second vectorized image corresponding to the road section elements; obtain the current position coordinates of the vehicle on the map, determine the pixel coordinates of the non-road section elements on the road section image based on the current position coordinates of the vehicle, and perform coordinate transformation processing on the pixel coordinates to generate the corresponding first vectorized image, and upload the first vectorized image to the cloud, so that the cloud completes the update based on the second vectorized image and the first vectorized image. Update processing of the map of the road section; accordingly, the cloud can receive the road section image carrying the pole-sign road surface elements in the road section to be updated sent by the vehicle side and the first vectorized image, the first vectorized image is obtained by coordinate transformation of the non-pole-sign road surface elements in the road section to be updated at the position coordinates to be updated; the road section image carrying the pole-sign road surface elements and its adjacent road section images are triangulated to obtain the second vectorized image corresponding to the pole-sign road surface elements; the first vectorized image and the second vectorized image are aggregated to obtain the vectorized map of the road section to be updated; the vectorized map is matched with the pre-stored high-precision map library to update the map of the road section to be updated according to the matching processing results.

[0190] Compared with the existing technology, this processing system can realize the update processing of the map of the road sections to be updated on the map through interactive processing between the vehicle side and the cloud side, that is, the non-pole sign road surface elements in the road surface elements are placed on the vehicle side for vector processing, and the pole sign road surface elements in the road surface elements are placed on the cloud side for vector processing, thereby reducing the processing efficiency of the vehicle side and improving the processing efficiency of the map update; at the same time, the high-precision base map stored in the cloud side is used as the basis for accuracy calibration, which can also improve the processing accuracy of the map update.

[0191] The present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, it implements the processing methods provided in the aforementioned embodiments 1 to 2, and / or the aforementioned embodiments 3 to 4.

[0192] The computer-readable storage medium provided in this embodiment has similar principles and technical effects to those achieved by the above-mentioned map update processing method, and will not be described in detail here.

[0193] The present invention also provides a program product including execution instructions stored in a readable storage medium. At least one processor of a vehicle can read the execution instructions from the readable storage medium and execute the execution instructions to cause the vehicle to implement the map update processing methods provided in the various embodiments described above.

[0194] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0195] The modules described above as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0196] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or hardware plus software functional modules.

[0197] The above-mentioned integrated module implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above-mentioned software functional module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the method provided by each embodiment of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0198] In the embodiments of the above-mentioned network devices or terminal devices, it should be understood that the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing map updates, characterized in that: The processing method is applied to the vehicle side, and the method includes: Collecting a road section image of the road section to be updated, and using a preset visual perception network to segment the road section image into road elements with pole signs and road elements without pole signs; Uploading the road section image carrying the pole sign road surface element to the cloud, so that the cloud generates a second vectorized image corresponding to the pole sign road surface element; The current position coordinates of the vehicle at the vehicle end on the map are obtained, the pixel coordinates of the non-pole sign road element on the road section image are determined based on the current position coordinates of the vehicle, and the pixel coordinates are transformed to generate a corresponding first vectorized image, and the first vectorized image is uploaded to the cloud, so that the cloud completes the update processing of the map of the road section to be updated based on the second vectorized image and the first vectorized image.

2. The method according to claim 1, characterized in that The step of segmenting the road surface elements and non-road surface elements from the road segment image includes: Extracting image features of the road section image using a backbone network in a preset visual perception network; The image features are respectively input into the detection network and the segmentation network in the preset visual perception network to obtain the road section image carrying the pole sign road surface elements detected by the detection network from the image features, and to obtain the various non-pole sign road surface elements segmented from the image features by the segmentation network.

3. The method according to claim 1, characterized in that The vehicle is provided with a collection device for collecting road section images, the collection device including: an inertial measurement sensor, a wheel speed sensor and a global positioning navigation system; The current position coordinates of the vehicle on the map include: Acquiring the vehicle speed and acceleration of the vehicle end collected by the inertial measurement sensor or the wheel speed sensor, and acquiring the first position coordinates of the vehicle by integrating the vehicle speed or the acceleration; The second position coordinates of the vehicle located by the global positioning navigation system are obtained, and the first position coordinates and the second position coordinates are used as input parameters of a preset Kalman filtering method to obtain the current position coordinates of the vehicle.

4. The method according to claim 3, characterized in that The step of determining the pixel coordinates of the non-pole sign road surface element on the road segment image according to the current position coordinates of the vehicle, and performing coordinate transformation processing on the pixel coordinates to generate a corresponding first vectorized image includes: Determining the position coordinates of the acquisition device according to the current position coordinates of the vehicle and the relative position between the acquisition device and the vehicle; Determine the pixel coordinates of the non-pole sign road surface elements on the road section image according to the position coordinates of the acquisition device and a preset mapping relationship between the acquisition device and the non-pole sign road surface elements; The pixel coordinates of each non-pole sign road surface element are converted into three-dimensional coordinates, and the three-dimensional coordinates are subjected to multi-frame fusion processing according to the acquisition time corresponding to the three-dimensional coordinates to obtain a first vectorized image corresponding to the non-pole sign road surface element.

5. The method according to claim 4, characterized in that Also includes: Determining whether lane line elements in the non-pole sign road surface element are parallel; When it is determined that the lane line elements are not parallel, the camera parameters in the acquisition device are updated according to a pre-stored camera calibration method to obtain updated camera parameters.

6. A method for processing map updates, characterized in that: The processing method is applied to the cloud, and the method includes: receiving a road section image of a road section to be updated that carries road sign elements and a first vectorized image sent by a vehicle end, wherein the first vectorized image is obtained by performing coordinate transformation on pixel coordinates of non-road sign elements in the road section to be updated; Performing triangulation processing on the road section image carrying the pole sign road surface element and its adjacent road section images to obtain a second vectorized image corresponding to the pole sign road surface element; Aggregating the first vectorized image and the second vectorized image to obtain a vectorized map of the road section to be updated; The vectorized map is matched with a pre-stored high-precision map library to update the map of the road section to be updated according to the matching result.

7. The method according to claim 6, characterized in that The triangulating the road section image carrying the pole sign road surface element and its adjacent road section images to obtain a second vectorized image corresponding to the pole sign road surface element includes: Performing sign classification processing on each road section image carrying the signboard road surface element to obtain a sign image set after the sign classification processing; For each brand image in the brand image set, using a corner detection method, obtain the pixel coordinates of a first corner point of the brand image, and obtain the pixel coordinates of a second corner point of a brand image adjacent to the brand image at the acquisition time corresponding to the brand image; The pixel coordinates of the first corner point and the pixel coordinates of the second corner point are triangulated to obtain the three-dimensional coordinates of the corner point of the sign image, and the three-dimensional coordinates of the corner point are subjected to multi-frame fusion processing according to the acquisition time corresponding to the three-dimensional coordinates of the corner point to obtain a second vectorized image corresponding to the road surface element of the sign.

8. The method according to claim 6, characterized in that The method further comprises: Receiving the non-pole-sign road surface element sent by the vehicle end, and determining a target high-precision map corresponding to the road section to be updated in the pre-stored high-precision map library according to the road section identifier in the non-pole-sign road surface element; Establishing a mapping rule between the non-pole sign road element and the target high-precision map, and calculating the current position coordinates of the vehicle at the vehicle terminal according to the mapping rule; The mapping rules include: 2D-3D constraints.

9. A map update processing device, characterized in that: The processing device is applied to the vehicle side, and the device includes: An acquisition module is used to acquire a road segment image of a road segment to be updated on a map, and to segment road surface elements with pole signs and road surface elements without pole signs from the road segment image using a preset visual perception network; A first communication module is configured to upload the road section image carrying the pole sign road surface element to the cloud, so that the cloud generates a second vectorized image corresponding to the pole sign road surface element; The first processing module is used to obtain the current position coordinates of the vehicle at the vehicle end on the map, determine the pixel coordinates of the non-pole sign road element on the road section image based on the current position coordinates of the vehicle, perform coordinate transformation processing on the pixel coordinates to generate a corresponding first vectorized image, and upload the first vectorized image to the cloud, so that the cloud completes the update processing of the map of the road section to be updated based on the second vectorized image and the first vectorized image.

10. A map update processing device, characterized in that: The processing device is applied to the cloud, and the device includes: The second communication module is configured to receive a road section image containing road sign elements in the road section to be updated and a first vectorized image sent by the vehicle end, wherein the first vectorized image is obtained by performing coordinate transformation on pixel coordinates of non-road sign elements in the road section to be updated; a second processing module configured to triangulate the road segment image carrying the pole sign road surface element and its adjacent road segment images to obtain a second vectorized image corresponding to the pole sign road surface element; The second processing module is configured to aggregate the first vectorized image and the second vectorized image to obtain a vectorized map of the road section to be updated; The updating module is used to match the vectorized map with a pre-stored high-precision map library to update the map of the road section to be updated according to the matching result.

11. A map update processing system, characterized in that: include: Vehicle-side and cloud-side; The vehicle side performs the method according to any one of claims 1 to 5; The cloud performs the method according to any one of claims 6 to 8.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method according to any one of claims 1 to 5 and / or the method according to any one of claims 6 to 8 are implemented.

Images