Mapping method and device, server and storage medium

Abstract

The invention relates to a mapping method and device, a server and a storage medium. The mapping method comprises the steps that perception data uploaded by a vehicle end are acquired, and the perception data comprise perception information and semantic information; reconstructing the perception information and / or the semantic information, wherein the reconstruction at least comprises information fusion processing; and generating a semantic map according to the reconstructed perception information and / or semantic information for the vehicle end to use. According to the scheme provided by the invention, the semantic map with more comprehensive information can be generated, the accuracy of the semantic map is improved, the driving planning of automatic driving is more favorably assisted, and the automatic driving experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a mapping method, device, server and storage medium. Background Art

[0002] In autonomous driving technology, maps play an important role in vehicle navigation and driving planning.

[0003] In related technologies, vehicles can use sensor data (such as lidar, camera, radar, etc.) to build semantic maps. The semantic map built by the vehicle can contain road information such as obstacles, lane lines, road signs, pedestrians, other vehicles, etc. around the vehicle, which can be used for local path planning, obstacle avoidance, and environmental perception.

[0004] However, currently, due to the limitations of the vehicle's hardware capabilities such as sensors and computing speed, perception capabilities are limited, which may result in missing information in the semantic map built on the vehicle side, reducing the accuracy of the map. It also limits the effectiveness of the real-time use of perception data while the vehicle is driving, affecting the driving planning of autonomous driving and reducing the user's autonomous driving experience. Summary of the invention

[0005] In order to solve or partially solve the problems existing in the related technologies, the present application provides a mapping method, device, server and storage medium, which can generate a semantic map with more comprehensive information, improve the accuracy of the semantic map, and be more conducive to assisting driving planning of autonomous driving, thereby improving the user's autonomous driving experience.

[0006] The first aspect of the present application provides a mapping method, comprising: Acquire the perception data uploaded by the vehicle, wherein the perception data includes perception information and semantic information; Reconstructing the perception information and / or semantic information, wherein the reconstruction at least includes information fusion processing; A semantic map is generated based on the reconstructed perception information and / or semantic information for use by the vehicle end.

[0007] In one embodiment, before reconstructing the perception information and / or semantic information, the method further includes: The perception information and / or semantic information is pre-processed to obtain pre-processed perception information and / or semantic information.

[0008] In one embodiment, after reconstructing the perception information and / or semantic information, the method further includes: Post-processing the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information; Generating a semantic map according to the reconstructed perceptual information and / or semantic information includes: A semantic map is generated according to the post-processed perceptual information and / or semantic information.

[0009] In one embodiment, after post-processing the reconstructed perception information and / or semantic information to obtain post-processed perception information and / or semantic information, the method further includes: Performing final processing on the post-processed perceptual information and / or semantic information to obtain final processed perceptual information and / or semantic information; Generating a semantic map according to the post-processed perception information and / or semantic information includes: A semantic map is generated according to the ultimately processed perceptual information and / or semantic information.

[0010] In one embodiment, the pre-treatment includes at least one of the following treatment methods: Self-intersection processing, deduplication processing, area of ​​interest processing, trajectory collision processing, false detection scene processing, and branch construction processing.

[0011] In one embodiment, the deduplication process includes: when repeated lane lines and road boundaries are detected, segment cutting or segment deletion is performed according to the length of the repeated portion; The trajectory collision processing includes: when a lane line and a road boundary where a collision occurs are detected, partially or completely deleting the lane line and the road boundary where a collision occurs according to the length of the lane line and the road boundary where a collision occurs; The misdetected scene processing includes: deleting inaccurate portions of the road boundary after identification; The branch construction process includes: determining a single connected lane line and a road boundary as the same branch, wherein the same branch does not contain bifurcations or loops.

[0012] In one embodiment, the reconstructing the perceptual information and / or semantic information, wherein the reconstructing at least includes information fusion processing, includes: Geometrically fuse lane line and road boundary perception information; and / or, Semantically fuse the semantic information of lane lines and road boundaries.

[0013] In one embodiment, the geometric fusion of the perceived information of the lane line and the road boundary includes: Search for the association relationship of branches and determine the branches that can establish a matching relationship; Interrupt the branches that have established matching relationships; The branches after the interruption processing are geometrically fused.

[0014] In one embodiment, semantic fusion of the semantic information of the lane line and the road boundary includes: Determine the voting result value of each voting grid according to the voting record of each voting grid in the maintained grid area, and determine the semantics of the lane line and the road boundary in the voting grid according to the voting result value; Determine the semantic breakpoints based on the voting grid and record the corresponding semantic attribute information.

[0015] In one embodiment, the post-processing includes at least one of the following processing methods: Deduplication processing and connection processing.

[0016] In one embodiment, the final processing includes at least one of the following processing methods: Fusion confidence verification processing and trajectory collision processing.

[0017] In one embodiment, the acquired perception data uploaded by the vehicle end is uploaded to the cloud after being processed by the vehicle end, and the preset processing includes at least one of the following processing methods: The system collects perception data at a preset sampling frequency, limits the size of a single file storing perception data, and records ancillary information about lane lines and road boundaries.

[0018] A second aspect of the present application provides a mapping device, including: A data acquisition module is used to acquire the perception data uploaded by the vehicle, wherein the perception data includes perception information and semantic information; A fusion and reconstruction module, used to reconstruct the perception information and / or semantic information, wherein the reconstruction at least includes information fusion processing; A generation module is used to generate a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle side.

[0019] In one embodiment, the device further comprises: a pre-processing module, used for pre-processing the perceptual information and / or semantic information to obtain pre-processed perceptual information and / or semantic information; and / or, The post-processing module is used to post-process the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information.

[0020] A third aspect of the present application provides a server, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0021] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0022] The technical solution provided by this application may have the following beneficial effects: After acquiring the perception data uploaded by the vehicle, the present application reconstructs the perception information and / or semantic information in the perception data, wherein the reconstruction at least includes information fusion processing; and then generates a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle. Through the above processing, the cloud combines a large amount of perception data uploaded by the vehicle, complements each other, denoises each other, and fuses each other to achieve reconstruction of the perception data, thereby generating a more comprehensive semantic map based on the reconstructed perception data, improving the accuracy of the semantic map, and being more conducive to assisting driving planning for autonomous driving and improving the user's autonomous driving experience.

[0023] Furthermore, before reconstructing the perceptual information and / or semantic information, the present application may also include: pre-processing the perceptual information and / or semantic information to obtain pre-processed perceptual information and / or semantic information; after reconstructing the perceptual information and / or semantic information, the present application may also include: post-processing the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information; after post-processing the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information, the present application may also include: performing ultimate processing on the obtained post-processed perceptual information and / or semantic information to obtain ultimate processed perceptual information and / or semantic information. Through the above-mentioned various optimization processes such as pre-processing, post-processing and ultimate processing, the comprehensiveness and accuracy of the semantic map are further improved, and the mapping accuracy and efficiency are further improved.

[0024] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0026] Figure 1 It is a first flow chart of the mapping method shown in the present application; Figure 2 is a second flow chart of the mapping method shown in the present application; Figure 3is a third flow chart of the mapping method shown in the present application; Figure 4 is a schematic diagram of self-intersection processing in the cloud shown in the present application; Figure 5 is a schematic diagram of deduplication processing performed on the cloud side shown in the present application; Figure 6 It is a schematic diagram of branch building processing in the cloud shown in this application; Figure 7 This is a schematic diagram of branch interruption processing in the cloud shown in this application; Figure 8 It is a schematic diagram of branch geometry fusion processing performed on the cloud side shown in the present application; Fig. 9 It is a schematic diagram of trajectory collision processing in the final post-processing optimization in the cloud shown in the present application; Fig.10 This is a schematic diagram of the effect of cloud-based mapping results shown in this application; Fig.11 is a first structural schematic diagram of a mapping device shown in the present application; Fig.12 is a second structural schematic diagram of the mapping device shown in the present application; Fig.13 It is a schematic diagram of the structure of the server shown in this application. DETAILED DESCRIPTION

[0027] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0028] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the third information, and similarly, the third information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0030] The semantic map constructed by the relevant technology on the vehicle side may be missing information, which reduces the accuracy of the map and limits the effectiveness of the real-time use of perception data while the vehicle is driving, affecting the driving planning of autonomous driving and reducing the user's autonomous driving experience.

[0031] In response to the above problems, the present application provides a mapping method that can generate a semantic map with more comprehensive information, improve the accuracy of the semantic map, and be more conducive to assisting autonomous driving driving planning and improving the user's autonomous driving experience.

[0032] The technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0033] Figure 1 1 is a schematic diagram of the first process of the mapping method shown in the present application. The method can be applied to a cloud server (referred to as the cloud).

[0034] See also Figure 1 , the method comprising: S101, acquiring the perception data uploaded by the vehicle, where the perception data includes perception information and semantic information.

[0035] The vehicle (referred to as the vehicle end) can be equipped with various types of sensor equipment, which are used to collect corresponding perception data for various environmental objects of the vehicle during driving. The perception information obtained by this application may mainly include geometric information of lane lines and road boundaries, etc. Geometric information is generally used to characterize the geometric structure of the lane, and geometric information may include information such as lines and curves. The semantic information of the lane lines and road boundaries of this application may mainly include descriptive information such as color, solid lines, and dashed lines.

[0036] The perception data uploaded by the vehicle end and obtained by the cloud end is uploaded to the cloud end after being processed by the vehicle end. The preset processing includes at least one of the following processing methods: The system collects perception data at a preset sampling frequency, limits the size of a single file storing perception data, and records ancillary information about lane lines and road boundaries.

[0037] S102, reconstructing the perception information and / or semantic information, wherein the reconstruction at least includes information fusion processing.

[0038] This step may include: geometrically fusing the perception information of the lane line and the road boundary; and / or semantically fusing the semantic information of the lane line and the road boundary.

[0039] The perception information of lane lines and road boundaries is geometrically fused, including: Search for the association relationship of branches and determine the branches that can establish a matching relationship; Interrupt the branches that have established matching relationships; The branches after the interruption processing are geometrically fused.

[0040] The semantic fusion of the semantic information of the lane line and the road boundary includes: Determine the voting result value of each voting grid according to the voting record of each voting grid in the maintained grid area, and determine the semantics of the lane line and the road boundary in the voting grid according to the voting result value; Determine the semantic breakpoints based on the voting grid and record the corresponding semantic attribute information.

[0041] S103, generating a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle side.

[0042] From this embodiment, it can be found that after the present application obtains the perception data uploaded by the vehicle, the perception information and / or semantic information in the perception data is reconstructed, wherein the reconstruction at least includes information fusion processing; and then a semantic map is generated based on the reconstructed perception information and / or semantic information for use by the vehicle. Through the above processing, the cloud combines a large amount of perception data uploaded by the vehicle, complements each other, denoises each other, and fuses each other to achieve reconstruction of the perception data, so that a more comprehensive semantic map can be generated based on the reconstructed perception data, improving the accuracy of the semantic map, and being more conducive to assisting the driving planning of autonomous driving and improving the user's autonomous driving experience.

[0043] Figure 2 1 is a second flow chart of the mapping method shown in the present application. The method can be applied to a cloud server (referred to as the cloud).

[0044] See also Figure 2 , the method comprising: S201, acquiring the perception data uploaded by the vehicle, where the perception data includes perception information and semantic information.

[0045] The vehicle (referred to as the vehicle end) can be equipped with various types of sensor equipment, which are used to collect corresponding perception data for various environmental objects of the vehicle during driving. The perception information obtained by this application may mainly include geometric information of lane lines and road boundaries, etc. Geometric information is generally used to characterize the geometric structure of the lane, and geometric information may include information such as lines and curves. The semantic information of the lane lines and road boundaries of this application may mainly include descriptive information such as color, solid lines, and dashed lines.

[0046] The perception data uploaded by the vehicle end and obtained by the cloud end is uploaded to the cloud end after being processed by the vehicle end. The preset processing includes at least one of the following processing methods: The system collects perception data at a preset sampling frequency, limits the size of a single file storing perception data, and records ancillary information about lane lines and road boundaries.

[0047] S202: Pre-process the perception information and / or semantic information to obtain pre-processed perception information and / or semantic information.

[0048] Among them, the pre-processing includes at least one of the following processing methods: self-intersection processing, deduplication processing, region of interest processing, trajectory collision processing, false detection scene processing, and branch construction processing.

[0049] The deduplication process includes: when repeated lane lines and road boundaries are detected, segment cutting or segment deletion is performed according to the length of the repeated part; The trajectory collision processing includes: when the lane line and the road boundary where the collision occurs are detected, partial or complete deletion is performed according to the length of the lane line and the road boundary where the collision occurs; False detection scene processing includes: deleting inaccurate parts of the road boundary after identification; The branch construction process includes: determining a single connected lane line and a road boundary as the same branch, and the same branch does not contain bifurcations or loops.

[0050] S203, reconstructing the pre-processed perception information and / or semantic information, wherein the reconstruction at least includes information fusion processing.

[0051] This step may include: geometrically fusing the perception information of the lane line and the road boundary; and / or semantically fusing the semantic information of the lane line and the road boundary.

[0052] The perception information of lane lines and road boundaries is geometrically fused, including: Search for the association relationship of branches and determine the branches that can establish a matching relationship; Interrupt the branches that have established matching relationships; The branches after the interruption processing are geometrically fused.

[0053] The semantic fusion of the semantic information of the lane line and the road boundary includes: Determine the voting result value of each voting grid according to the voting record of each voting grid in the maintained grid area, and determine the semantics of the lane line and the road boundary in the voting grid according to the voting result value; Determine the semantic breakpoints based on the voting grid and record the corresponding semantic attribute information.

[0054] S204, generating a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle side.

[0055] It can be found from this embodiment that before reconstructing the perception information and / or semantic information, the present application may also include: pre-processing the perception information and / or semantic information to obtain the perception information and / or semantic information after pre-processing; the pre-processing may include self-intersection processing, deduplication processing, region of interest processing, trajectory collision processing, false detection scene processing, branch construction processing, etc. Through the above-mentioned various pre-processing methods, the comprehensiveness and accuracy of the semantic map are further improved, and the mapping accuracy and efficiency are further improved.

[0056] Figure 3 This is a schematic diagram of the third process of the mapping method shown in this application. This process can be applied to a cloud server (referred to as the cloud), and the cloud and the vehicle (referred to as the vehicle end) can interact.

[0057] This application automatically constructs a high-precision semantic map that can be used for autonomous driving by fusing and reconstructing the lane lines and road boundaries perceived by the vehicle in the cloud. This application greatly optimizes the input data by pre-processing the perceived road boundaries and lane lines such as de-duplication, de-distortion, and denoising; it then reconstructs multiple frames of road boundaries and lane lines by interruption, matching, fusion, reconnection, and precise de-duplication, and finally generates a map and outputs it. The map generated and output by this application in the cloud has lane line information with accuracy that meets the requirements. The semantic information of the road boundaries and lane lines can more accurately assist the planning of autonomous driving, as well as assist in the judgment of driving speed and lane changes, thereby greatly improving the intelligence level of autonomous driving.

[0058] See also Figure 3 , the method comprising: S301, the cloud obtains the perception data uploaded by the vehicle, and the perception data includes the perception information and semantic information of the lane line and the road boundary.

[0059] The cloud obtains the perception and semantic information of lane lines and road boundaries recorded and uploaded by the vehicle.

[0060] The perception information detected and recorded by the vehicle includes geometric information of lane lines and road boundaries, and semantic information includes auxiliary description information of lane lines and road boundaries.

[0061] The vehicle side may be equipped with various types of sensing devices, which are used to collect corresponding perception data for various environmental objects of the vehicle during driving. Among them, the sensing devices may include laser radar devices, ultrasonic radar devices, and vehicle-mounted camera devices. It should be noted that the devices used to collect data in the embodiments of the present application are not limited to the above examples, but may also be other devices that can collect environmental data during vehicle driving.

[0062] The perception information detected by the vehicle includes road vector information, etc., and the road vector information may include multiple road vector elements. Road vector elements are a kind of road mapping elements, which are used to describe information such as roads and routes around roads, signs, etc. Road vector elements include but are not limited to lane lines, dotted lines, lane boundary lines, road boundary lines, road surface arrows, stop lines, sidewalk lines or crosswalks, etc., and these road vector elements can be represented by coordinate points.

[0063] For example, the vehicle side can use lidar equipment to detect obstacles around the vehicle during driving, use camera equipment to take pictures of the vehicle's surroundings, and perform feature extraction and feature recognition on the pictures to determine the geometric information such as road signs, speed bumps, lane lines, obstacles, etc. in the vehicle's current environment.

[0064] The perception information acquired by the present application may mainly include geometric information of lane lines and road boundaries, etc. Geometric information is generally used to characterize the geometric structure of lanes, and geometric information may include information such as lines and curves. The geometric information of lane lines and road boundaries must comply with preset rules, such as the geometric connection of lane lines must comply with geometric connection rules, the geometric ranking of lanes must comply with arrangement rules, and the smoothness of lane center lines must comply with smoothness rules.

[0065] The semantic information of the present application can be divided into different types of semantic information according to the objects in the real environment, such as semantic information for lane lines, semantic information for road boundaries, semantic information for road signs, semantic information for obstacles, etc.

[0066] The semantic information of lane lines and road boundaries in this application may mainly include descriptive information such as color, solid line, and dashed line, among which the color of lane lines may be given a certain meaning. For example, a yellow line corresponds to a road dividing line, and a white line corresponds to no road dividing line. By converting semantic information into road color, the real road information can be restored.

[0067] Considering the limitation of vehicle-side computing capacity, this application does not perform the mapping fusion of lane lines and road boundaries on the vehicle side. Instead, it chooses to record the perception information and semantic information on the vehicle side and upload it to the cloud. The cloud side then performs mapping fusion calculation based on the perception information and semantic information uploaded by the vehicle side. In other words, the vehicle side can obtain preliminary road element results based on the perception results, and the cloud side will perform post-processing corrections.

[0068] Taking into account network traffic and hardware limitations, the vehicle side of this application can collect perception data at a preset sampling frequency. The vehicle side can sample the recorded perception information to avoid storing excessive information on the vehicle side, which causes excessive time consumption or upload failure due to excessive data upload to the cloud. Considering that it is not necessary for the vehicle to collect repeated data within a certain range, this application adopts the method of recording a certain amount of perception data every time it moves a certain distance, and reduces the sampling frequency through downsampling processing.

[0069] In order to avoid a single storage file being too large, which makes it difficult to upload to the cloud and cannot be processed in parallel, this application can limit the size of a single file storing perception data. For example, a frame limit is set for each file stored on the vehicle side. After recording a certain number of frames of data, the data is stored on the disk and no longer stored. In other words, this application limits a single file through the disk file, and each frame of data can correspond to a sampling of a unit distance.

[0070] The content currently recorded by the vehicle side includes lane lines and road boundaries with various semantic information, which are all the information required for real vehicle road driving. This application takes into account the accuracy of map construction and noise filtering, and can also record some ancillary information of lane lines and road boundaries, such as the position of lane lines and road boundaries relative to the vehicle when they are recorded and / or the number of observations, etc., to confirm the credibility of lane lines and road boundaries. The vehicle side can record the status of the vehicle when the vehicle passes through certain ground signs, such as speed bumps, speed information, acceleration information, etc. Ancillary information such as speed bumps can be stored in characters or similar information.

[0071] S302, the cloud pre-processes the perception information and semantic information of the lane lines and road boundaries uploaded by the vehicle.

[0072] The cloud pre-processes the acquired perception information and semantic information of lane lines and road boundaries so that the input information can be optimized.

[0073] Among them, the pre-processing in the cloud can include at least one of the following processing methods: self-intersection processing (denoising), deduplication processing (self-deduplication, mutual deduplication), ROI (region of interest) processing, trajectory collision processing, false detection scene processing, and branch construction processing.

[0074] 1) Self-intersection processing (denoising) See also Figure 4 Due to the limitations of vehicle-side perception, unreasonable and messy lane lines or road boundaries may appear, and such line segments may have problems such as bends and loops. The present application uses a preset algorithm, such as the cross product of the line segment point sequence, to determine whether the line segment is self-intersecting. If it is determined to be self-intersecting, the line segment is extracted for denoising, such as interrupting and reconnecting it to achieve smoothing optimization, that is, turning the curved and looped curves into straight lines, so that the line segment becomes without bends or loops, etc.

[0075] 2) Deduplication processing (self-deduplication, mutual deduplication) The deduplication process includes: when repeated lane lines and road boundaries are detected, line segments are cut or deleted according to the length of the repeated parts.

[0076] See also Figure 5 Due to the limitations of vehicle-side perception, unreasonable repeated lane lines and road boundaries may appear. When such lane lines and road boundaries appear, this application performs deduplication processing to perform optimization operations. The optimization operations include cutting or direct deletion based on the length of the repeated parts.

[0077] For example, lane lines that are more than a preset ratio, such as 80%, are considered duplicates, and if the duplicated line segment is less than a preset length, such as 2m, it can be deleted.

[0078] 3) ROI processing ROI is the area of ​​the image that is focused on when analyzing and processing the image. This area may be the location of the target on the image. In machine vision and image processing, the area to be processed is outlined in the image in the form of a box, circle, ellipse, irregular polygon, etc., which is called the region of interest. The typical shape of ROI can be a rectangle or other shapes.

[0079] This application chooses not to use lane lines and road boundaries that are too far away because of the limitations of perception. Data that is too far away is not accurate enough to be used. For these lane lines and road boundaries, this application uses ROI to filter them when using them, and chooses not to use them after filtering them out.

[0080] 4) Track collision processing The trajectory collision processing includes: when a lane line and a road boundary where a collision occurs are detected, partial or complete deletion is performed according to the length of the lane line and the road boundary where a collision occurs.

[0081] Because the cloud is the post-processor of all data, the cloud knows the historical trajectory of the vehicle. It is impossible for a road boundary to cross the trajectory, because crossing represents a collision. Therefore, this application can determine that any road boundary that collides with the historical trajectory is perception noise or unreliable data. For this part of the lane line and road boundary that collided, it can be partially or completely deleted according to its length.

[0082] 5) False detection scenario processing False detection scene processing includes deleting inaccurate parts of the road boundary after identification.

[0083] This application processes lane lines in scenes where perception is prone to misdetection. For example, for scenes such as large curvature ramp scenes, upstream perception is usually unable to accurately give accurate road boundaries in the distance that conform to the actual curvature. If the inaccurate parts of the road boundaries are simply fused, it is easy to cause road closures that do not conform to the actual situation. Therefore, for these scenes, this application identifies the inaccurate parts of the road boundaries in advance, and then deletes the inaccurate parts of the road boundaries in advance before fusion, leaving only the accurate parts of the road boundaries for fusion.

[0084] This application can perform multi-frame confirmation for scenes with large curvature. This application will establish a grid map in this scenario, and each grid will record the number of times it was observed, the distance at which it was observed, etc.; it will also record the number of times it was observed as non-existent and the distance at which it was observed as non-existent. The score is calculated using the above four types of data, and unreliable observation data is filtered out based on the score. For example, if the score is less than or equal to the preset threshold, it is considered to be unreliable observation data and is filtered out.

[0085] 6) Branch build processing Branch processing includes: determining a single connected lane line and road boundary as the same branch, and the same branch does not contain forks or loops.

[0086] Among them, branch is a single fusion unit used in this application scheme. The fusion is operated based on its single geometric connection attribute. There are no loops, bifurcations, or duplicate points in the branch. It can be guaranteed that each fusion is a simple one-to-one fusion to ensure the geometric accuracy of the fusion, and there will be no loss of useful information due to multiple fusion or wrong fusion.

[0087] The branch of the present application is a connection between one or more lane lines and road boundaries. This connection will go through a series of operations such as interruption, deduplication, delooping and reconnection to ensure that there will be no forks or loops in the connection relationship. This ensures that each branch is a small fused individual, and that each subsequent fusion is a one-to-one fusion between the simplest branches, thereby ensuring the accuracy and completeness of each fusion.

[0088] See also Figure 6 , is an example of branch construction. If there is a fork, they are not considered to be the same branch. Generally, a single connected lane line and road boundary are considered to be the same branch, such as Figure 6 The line segments of the same color in the figure, for example, the red line segment 61, the blue line segment 62 and the green line segment 63.

[0089] S303, the cloud performs geometric fusion on the perception information of the lane line and the road boundary after pre-processing.

[0090] The perception information of lane lines and road boundaries is geometrically fused, including: searching for the association relationship between branches, determining branches that can establish matching relationships; interrupting branches that have established matching relationships; and geometrically fusing the interrupted branches.

[0091] 1) Search for branch association relationships The association relationship between branches is determined by the distance between the points between the branches. When the distance between the points of two branches is close enough, for example, less than or equal to the preset distance threshold, and the proportion of the number of matching points to the number of points of the branch itself is large enough, for example, greater than or equal to the preset ratio threshold, a matching relationship can be established between the two branches.

[0092] 2) Perform branch interruption processing Perform various interruption processing between branches that have established matching relationships.

[0093] For example, if the distance between matching points between branches suddenly increases, for example, it is greater than a preset distance threshold, the branch forms a loop, or there are nodes near the branch, then the branch is interrupted, and the interrupted branch can be geometrically fused and updated later.

[0094] If a point can be connected to more than two geometric line segments (lane lines or road boundaries), the point is a breakpoint between branches.

[0095] Among them, when at least two geometric line segments (lane lines or road boundaries) are connected to the same point, the point is a node.

[0096] See also Figure 7 , Figure 7 The leftmost part of the diagram is two frames of data, the yellow line 701 is the first frame, and the green line 702 is the second frame. The reason for the interruption is that there is a section from left to right that can be matched based on the distance, but the distance increases (for example, greater than or equal to the preset distance threshold) and cannot be matched, so the interruption starts from the point where it cannot be matched. In other words, this situation is a branch point interruption, the yellow line 701 is interrupted into the yellow line 7011 and the red line 7012 at the branch point, and the green line 702 is interrupted into the green line 7021 and the blue line 7022 at the branch point. In addition, in this case, the green line 7021 is merged into the yellow line 7011, because the yellow line 7011 and the green line 7021 can be successfully matched based on the distance, so the yellow line 7011 and the green line 7021 are merged; because the blue line 7022 and the red line 7012 are too far away to match, they form branches separately.

[0097] Figure 7 In the case of the middle part of the diagram, after determining that the green line 711 forms a loop, the green line 711 is interrupted at the loop point and divided into the red line 7111 and the green line 7112. In other words, this case is interrupted at the bending point. In addition, in this case, the yellow line 712 is merged into the red line 7111, and the blue line 713 is merged into the green line 7112.

[0098] Figure 7 The right part of the case is a node interruption. For the case of nodes, see 6 where the red line 61, the blue line 62, and the green line 63 are connected to the same point, which is a node. Figure 7 In the figure, after the blue line 724 is interrupted at the node, a gray line 7241 and a blue line 7242 are obtained. The gray line 7241 is integrated into the yellow line 723, and the blue line 7242 is integrated into the green line 722.

[0099] 3) Perform branch geometry fusion and update See also Figure 8 , branches with similar geometric distances, for example, less than or equal to a preset distance threshold, are geometrically merged, for example, the red branch 81 and the green branch 82 are geometrically merged to obtain the blue branch 83.

[0100] The geometric fusion can be performed in a relatively smooth manner, ensuring that each fusion can fully consider the error of each perception. The two matched branches will be corrected in the matching direction at the same time.

[0101] Branches that have not been merged with any other branches are considered newly observed branches and are placed in the map as new elements.

[0102] S304, the cloud performs semantic fusion on the semantic information of the lane line and the road boundary after pre-processing.

[0103] Among them, the semantic information of lane lines and road boundaries is semantically fused, including: determining the voting result value of each voting grid according to the voting records of each voting grid in the maintained grid area, and determining the semantics of the lane lines and road boundaries in the voting grid according to the voting result value; determining the semantic break point according to the voting grid and recording the corresponding semantic attribute information.

[0104] Semantic fusion can include the following processes: 41) Grid Semantic Voting Since the entire post-processing process is carried out along the vehicle trajectory, the present application can maintain a grid area near the vehicle trajectory in the cloud with a total area of ​​a preset area, such as 100m*100m, and a grid size of a preset size, such as 0.2m*0.2m, to record lane line attributes.

[0105] Among them, each grid in the grid area, namely the voting grid, can record the voting records of each lane line type (such as dotted line, solid line) and each lane color (such as yellow, white, orange). The voting result of the grid area of ​​each voting grid takes a larger value, such as the maximum voting value, to obtain the semantics of a lane line.

[0106] In the fusion process, the lane lines within a certain range can be scored by a classifier, and the semantics of the area can be determined based on high scores, such as the highest score.

[0107] The record handling methods may include: All lane lines in each frame observation are updated with their attributes in the grids that the lane line passes through. For example, if there is a yellow single solid line in the observation frame, the number of "single solid line" type votes in all grids that this lane line passes through is increased by 1, and the number of "yellow" color votes is increased by 1. By operating all lane lines in a single frame observation in this way, the semantic voting grid can be maintained frame by frame.

[0108] This application adds the next frame to the previous historical frame. When the historical frame (the content in the grid) is no longer updated, the final voting classification is performed.

[0109] 42) Assign break points based on voting grid Not all grids passed through a branch have exactly the same semantics. For example, if a branch passes through 10 grids, and the semantics of the first five grids are "white dashed lines" and the semantics of the last five grids are "white solid lines", a semantic breakpoint is determined at the position of the fifth grid, and the corresponding attributes are recorded to facilitate subsequent segmentation and output.

[0110] S305, the cloud performs post-processing on the fused lane lines and road boundaries.

[0111] The post-processing of the present application may be referred to as fusion post-processing, and the fusion post-processing may include at least one of the following processing methods: deduplication processing and connection processing.

[0112] 51) Deduplication This application directly performs a deduplication operation on the merged branch. Because branches are matched one-to-one, there may be duplicate parts after merging, so deduplication is performed on this part to ensure geometric accuracy and prevent map clutter.

[0113] 52) Connection processing This application performs a round of connection on the deduplicated branches to prevent the map from being cluttered and the single connectivity of the branches.

[0114] S306, outputting mapping elements in the cloud.

[0115] In this step, the branch can be further cut geometrically and semantically.

[0116] For example, when part of a branch is far enough behind the vehicle, for example, greater than or equal to a preset distance threshold, and has not been updated for a long time, for example, greater than a preset time threshold, these branches can be segmented according to semantic and geometric distances and output as mapping elements. This application trusts branches far away from the vehicle, so these branches no longer need to be fused and can be directly output as mapping elements.

[0117] Different semantic information may also appear in the same branch. These branches can be cut into different lane lines and road boundaries and re-output as mapping elements. This part of the mapping elements will not be used for fusion after being output. This application can cut (split) the same branch with different semantics. For example, there are 100 points in a branch, 20 points can be yellow dashed lines, 60 points can be white solid lines, and 20 points can be double yellow lines. In the end, a section of yellow dashed lines, a section of white solid lines, and a section of double yellow lines will be retained in the map.

[0118] S307, the cloud performs final processing on the output mapping elements.

[0119] The final processing of this application can be called the final optimization processing.

[0120] The final optimization process may include at least one of the following processing methods: fusion confidence verification processing and trajectory collision processing.

[0121] 71) Fusion confidence verification processing In the final optimization process, the present application performs fusion confidence verification, such as screening and deleting mapping elements with extremely small fusion times. Because the number of fusions is extremely small, it means that the number of observations is extremely small, so the confidence of the mapping element is low, so it can be deleted after screening.

[0122] This application can calculate the confidence level based on the number of fusion times of each lane line. If the number of fusion times is low, the confidence level is low. When the confidence level is less than or equal to the preset confidence threshold, the mapping element can be deleted.

[0123] 72) Track collision processing In the final optimization process, the present application may perform another verification and optimization of trajectory collision.

[0124] The trajectory collision processing in the final optimization process is similar to the trajectory collision processing process in the pre-processing, which is also to avoid unreasonable road boundary mapping.

[0125] See also Fig. 9 , the green line 91 is the driving trajectory, the red lines 91 and 92 are the road boundary lines, but the red line 94 collides with the trajectory, so the red line 94 needs to be deleted.

[0126] S308, the cloud performs fusion processing on traffic information such as ground signs / traffic signs, and generates a semantic map based on mapping elements.

[0127] This application can use multi-frame fusion to update the geometry and attributes of traffic information such as ground arrows, sidewalks, stop lines, traffic lights, traffic signs, ETC (Electronic Toll Collection) signs, etc. recorded in the uploaded data, and put the updated results into the map.

[0128] These traffic information are slightly different from lane lines and road boundaries. Because the geometric information of traffic information is simple, it does not need to have complex geometric fusion processing like lane lines and road boundaries. Each semantic of these traffic information contains different content. For example, arrows may have signs such as straight, right turn, left turn, U-turn, etc., but sidewalks only have range information.

[0129] After this application outputs mapping elements and integrates traffic information such as ground signs / traffic signs, a map can be generated based on the mapping elements.

[0130] Among them, the mapping methods include but are not limited to: SLAM (Simultaneous Localization and Mapping) method and the like.

[0131] This application scheme, applied to the use scenario of autonomous driving vehicle, has many advantages.

[0132] For example, in terms of positioning, when the GNSS signal is not good, the vehicle sensing data of the present application can have multiple sources, so the moving vehicle can be accurately positioned by matching real-time sensing data with mapping data. The relevant technology is limited by hardware capabilities such as sensors and computing speed, and the perception ability is limited. Obstacles that are too far away cannot be accurately perceived, and obscured areas cannot be accurately perceived, which limits the effect of using perception data in real time while driving. The present application uses cloud post-processing to memorize the perceived data, so that a large amount of perception data can be combined to complement and denoise each other. When the vehicle passes the same place again, it can use more accurate map data to replace the obscured or unclear perception data, so as to make smarter and more accurate driving planning.

[0133] In terms of automated processing, efficiency and coverage, this application can replace complex manual work, improve efficiency and achieve full coverage. The workload of manual mapping in related technologies is huge, and it is impossible to manually map the entire route for long routes. However, this application can achieve full coverage by using perception self-mapping. This is because the cloud can effectively memorize the entire route that the vehicle has passed once through the map fusion algorithm, and fuse it into a usable semantic map.

[0134] This application improves the mapping accuracy through various perceptual optimization, cropping, smoothing, matching and fusion operations, making the mapping results usable and close to the results of manual mapping.

[0135] Fig.10 This is a schematic diagram of the effect of the semantic map generated by the mapping algorithm of this application. The effect of applying this application solution can be seen in Fig.10 shown.

[0136] Fig.10 The semantic information displayed in the image includes road boundary lines, lane lines, ground arrows, stop lines, safety islands, etc. The randomly selected mapping results also reflect the extremely high semantic accuracy and geometric accuracy of the current mapping algorithm of this application.

[0137] As can be seen from this example, the present application performs pre-processing such as de-duplication, de-distortion, and denoising on the perceived road boundary lines and lane lines, and then reconstructs multiple frames of road boundary lines and lane lines by interruption, matching, fusion, reconnection, and precise de-duplication, so that the lane line information of the semantic map output from the cloud is more accurate. The semantic information of the road boundary lines and lane lines can more accurately assist the planning of autonomous driving, as well as assist in the judgment of driving speed and lane change, thereby greatly improving the intelligence level of autonomous driving.

[0138] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a mapping device, a server and corresponding embodiments.

[0139] Fig.11 It is a first structural schematic diagram of the mapping device shown in this application.

[0140] See also Fig.11 The present application provides a mapping device 110 , including: a data acquisition module 111 , a fusion and reconstruction module 112 , and a generation module 113 .

[0141] The data acquisition module 111 is used to acquire the perception data uploaded by the vehicle, and the perception data includes perception information and semantic information. The perception information acquired in the present application may mainly include geometric information of lane lines and road boundaries, and the geometric information may include information such as lines and curves. The semantic information of lane lines and road boundaries in the present application may mainly include description information such as color, solid line, and dashed line.

[0142] The fusion reconstruction module 112 is used to reconstruct the perception information and / or semantic information, wherein the reconstruction includes at least information fusion processing. The fusion reconstruction module 112 can geometrically fuse the perception information of the lane line and the road boundary; and / or semantically fuse the semantic information of the lane line and the road boundary. The geometric fusion of the perception information of the lane line and the road boundary includes: searching for the association relationship of the branches, determining the branches that can establish a matching relationship; interrupting the branches that have established a matching relationship; and geometrically fusing the interrupted branches. The semantic fusion of the semantic information of the lane line and the road boundary includes: determining the voting result value of each voting grid according to the voting record of each voting grid in the maintained grid area, determining the semantics of the lane line and the road boundary in the voting grid according to the voting result value; determining the semantic interruption point according to the voting grid and recording the corresponding semantic attribute information.

[0143] The generation module 113 is used to generate a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle side.

[0144] From this embodiment, it can be found that the device provided by the present application reconstructs the perception information and / or semantic information in the perception data after acquiring the perception data uploaded by the vehicle, wherein the reconstruction at least includes information fusion processing; and then generates a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle. Through the above processing, the cloud combines a large amount of perception data uploaded by the vehicle, complements each other, denoises each other, and fuses each other to achieve reconstruction of the perception data, so that a more comprehensive semantic map can be generated based on the reconstructed perception data, improving the accuracy of the semantic map, and being more conducive to assisting the driving planning of autonomous driving and improving the user's autonomous driving experience.

[0145] Fig.12 It is a second structural schematic diagram of the mapping device shown in this application.

[0146] See also Fig.12 The present application provides a mapping device 120 , including: a data acquisition module 111 , a fusion and reconstruction module 112 , and a generation module 113 .

[0147] The functions of the data acquisition module 111, the fusion reconstruction module 112, and the generation module 113 can be seen in Fig.11 Description in .

[0148] The mapping device 110 may further include: a pre-processing module 114 and / or a post-processing module 115 .

[0149] A pre-processing module 114 is used to pre-process the perception information and / or semantic information to obtain the pre-processed perception information and / or semantic information; and / or, The post-processing module 115 is used to post-process the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information.

[0150] Among them, the pre-processing includes at least one of the following processing methods: self-intersection processing, deduplication processing, region of interest processing, trajectory collision processing, false detection scene processing, and branch construction processing.

[0151] The deduplication process includes: when repeated lane lines and road boundaries are detected, segment cutting or segment deletion is performed according to the length of the repeated part; The trajectory collision processing includes: when the lane line and the road boundary where the collision occurs are detected, partial or complete deletion is performed according to the length of the lane line and the road boundary where the collision occurs; False detection scene processing includes: deleting inaccurate parts of the road boundary after identification; The branch construction process includes: determining a single connected lane line and a road boundary as the same branch, and the same branch does not contain bifurcations or loops.

[0152] Post-processing includes at least one of the following processing methods: deduplication processing and connection processing.

[0153] This application directly performs a deduplication operation on the merged branch. Because branches are matched one-to-one, there may be repeated parts after merging, so this part is deduplicated to ensure geometric accuracy and prevent map clutter. This application performs a round of connection on the deduplicated branch to prevent map clutter and single connectivity of the branch.

[0154] The device provided in the present application performs pre-processing such as de-duplication, de-distortion, and denoising on the perceived road boundary lines and lane lines, and then reconstructs multiple frames of road boundary lines and lane lines by interruption, matching, fusion, reconnection, and precise de-duplication, so that the lane line information of the semantic map output from the cloud is more accurate. The semantic information of the road boundary lines and lane lines can more accurately assist the planning of autonomous driving, as well as assist in the judgment of driving speed and lane change, thereby greatly improving the intelligence level of autonomous driving.

[0155] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0156] Fig.13 It is a schematic diagram of the structure of the server shown in this application.

[0157] See also Fig.13 , the server 1000 includes a memory 1010 and a processor 1020 .

[0158] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0159] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0160] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.

[0161] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0162] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0163] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A mapping method, characterized in that: include: Acquire the perception data uploaded by the vehicle, wherein the perception data includes perception information and semantic information; Reconstructing the perception information and / or semantic information, wherein the reconstruction at least includes information fusion processing; A semantic map is generated based on the reconstructed perception information and / or semantic information for use by the vehicle end.

2. The method according to claim 1, characterized in that Before reconstructing the perception information and / or semantic information, the method further includes: The perception information and / or semantic information is pre-processed to obtain pre-processed perception information and / or semantic information.

3. The method according to claim 1, characterized in that After reconstructing the perception information and / or semantic information, the method further includes: Post-processing the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information; Generating a semantic map according to the reconstructed perceptual information and / or semantic information includes: A semantic map is generated according to the post-processed perceptual information and / or semantic information.

4. The method according to claim 3, characterized in that After the reconstructed perception information and / or semantic information is post-processed to obtain post-processed perception information and / or semantic information, the method further includes: Performing final processing on the post-processed perceptual information and / or semantic information to obtain final processed perceptual information and / or semantic information; Generating a semantic map according to the post-processed perception information and / or semantic information includes: A semantic map is generated according to the ultimately processed perceptual information and / or semantic information.

5. The method according to claim 2, characterized in that: The pre-processing includes at least one of the following processing methods: Self-intersection processing, deduplication processing, area of ​​interest processing, trajectory collision processing, false detection scene processing, and branch construction processing.

6. The method according to claim 5, characterized in that The deduplication process includes: when repeated lane lines and road boundaries are detected, segment cutting or segment deletion is performed according to the length of the repeated part; The trajectory collision processing includes: when a lane line and a road boundary where a collision occurs are detected, partially or completely deleting the lane line and the road boundary where a collision occurs according to the length of the lane line and the road boundary where a collision occurs; The misdetected scene processing includes: deleting inaccurate portions of the road boundary after identification; The branch construction process includes: determining a single connected lane line and a road boundary as the same branch, wherein the same branch does not contain bifurcations or loops.

7. The method according to claim 3, characterized in that The reconstructing the perception information and / or semantic information, wherein the reconstructing at least includes information fusion processing, comprises: Geometrically fuse lane line and road boundary perception information; and / or, Semantically fuse the semantic information of lane lines and road boundaries.

8. The method according to claim 7, characterized in that The geometric fusion of the perception information of the lane line and the road boundary includes: Search for the association relationship of branches and determine the branches that can establish a matching relationship; Interrupt the branches that have established matching relationships; The branches after the interruption processing are geometrically fused.

9. The method according to claim 7, characterized in that: The semantic fusion of the semantic information of the lane line and the road boundary includes: Determine the voting result value of each voting grid according to the voting record of each voting grid in the maintained grid area, and determine the semantics of the lane line and the road boundary in the voting grid according to the voting result value; Determine the semantic breakpoints based on the voting grid and record the corresponding semantic attribute information.

10. The method according to claim 3, characterized in that The post-processing includes at least one of the following processing methods: Deduplication processing and connection processing.

11. The method according to claim 4, characterized in that The final processing includes at least one of the following processing methods: Fusion confidence verification processing and trajectory collision processing.

12. The method according to any one of claims 1 to 11, characterized in that: The acquired perception data uploaded by the vehicle end is uploaded to the cloud after being processed by the vehicle end. The preset processing includes at least one of the following processing methods: The system collects perception data at a preset sampling frequency, limits the size of a single file storing perception data, and records ancillary information about lane lines and road boundaries.

13. A mapping device, characterized in that: include: A data acquisition module is used to acquire the perception data uploaded by the vehicle, wherein the perception data includes perception information and semantic information; A fusion and reconstruction module, used to reconstruct the perception information and / or semantic information, wherein the reconstruction at least includes information fusion processing; A generation module is used to generate a semantic map based on the reconstructed perception information and / or semantic information for use by the vehicle side.

14. The device according to claim 13, characterized in that The device also includes: a pre-processing module, used for pre-processing the perceptual information and / or semantic information to obtain pre-processed perceptual information and / or semantic information; and / or, The post-processing module is used to post-process the reconstructed perceptual information and / or semantic information to obtain post-processed perceptual information and / or semantic information.

15. A server, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 12.

16. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 12.

Images