A method, device, electronic device and storage medium for processing semantic elements
By obtaining point cloud maps and reference images, and updating vectorized graphics of semantic elements multiple times, the low-precision problem caused by insufficient point clouds is solved, the accuracy of navigation maps is improved, and the needs of autonomous driving technology are met.
Patent Information
- Application Number
- CN202110565715.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-24
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, the point clouds scanned by the lidar are not dense enough, resulting in low accuracy of the extracted semantic elements, which further leads to insufficient accuracy of the built navigation map and cannot meet the current needs of autonomous driving technology.
By obtaining the point cloud diagram and the reference image, the initial vectorized figure of the semantic element is determined and updated multiple times based on this, until the overlap between the vectorized figure and the reference image area reaches a preset threshold after the update, thereby improving the semantic element in the point cloud diagram. characterization range and accuracy in.
提高了语义元素的精度,进一步提升了构建的导航地图精度,满足了自动驾驶技术的需求。
Smart Images

Figure CN113191323B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer processing technology, and in particular to a method, device, electronic device and storage medium for processing semantic elements. Background Art
[0002] In recent years, autonomous driving technology has developed rapidly. As an essential element of autonomous driving vehicles, high-precision semantic navigation maps play a fundamental role in perception, positioning, planning and vehicle control during autonomous driving. Among them, map semantic information, as a key component of high-precision semantic navigation maps, is indispensable for map navigation planning.
[0003] Most of the related technologies extract semantic elements such as dotted line segments, road signs, and traffic lights through point clouds scanned by lidar. However, the scanned point clouds are often not dense enough, resulting in low accuracy of the extracted semantic elements, which further leads to low accuracy of the constructed navigation map and cannot meet the needs of current autonomous driving technology. Summary of the invention
[0004] The embodiments of the present disclosure at least provide a method, device, electronic device and storage medium for processing semantic elements.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing semantic elements, the method comprising:
[0006] Acquire a point cloud image containing semantic elements and a reference image corresponding to the point cloud image;
[0007] Based on the semantic elements in the point cloud image, determining an initial vectorized graphic representing the semantic elements;
[0008] An updated vectorized graphic of the semantic element is determined based on the initial vectorized graphic and the image area of the semantic element in the reference image; the overlap degree between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; the updated vectorized graphic is used to update the representation range of the semantic element in the point cloud image.
[0009] By adopting the above-mentioned method for processing semantic elements, firstly, based on the semantic elements in the point cloud map, the initial vectorized graphic representing the semantic element can be determined, and based on this initial vectorized graphic and the image area of the semantic element in the reference image, the updated vectorized graphic of the semantic element can be determined. The updated vectorized graphic here can be obtained by updating the initial vectorized graphic multiple times based on the overlap between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image. When the overlap is greater than the preset threshold, it can be explained to a certain extent that the representation range of the semantic element corresponding to the current updated vectorized graphic in the point cloud map reaches the standard range of the reference image, and the accuracy of the expressed semantic element is relatively high, which further enables the construction of a more accurate map.
[0010] In a possible implementation, the determining, based on the semantic element in the point cloud image, an initial vectorized graphic representing the semantic element includes:
[0011] Acquire vector information of semantic elements in the point cloud image; the vector information includes one or more of size information, rotation information and translation information;
[0012] Based on the vectorization information, the initial vectorization graphic is generated.
[0013] The initial vectorized graphics of the embodiment of the present disclosure can be generated based on vectorized information such as size information, rotation information, and translation information of the semantic element, so as to achieve multi-dimensional feature expression for the semantic element.
[0014] In a possible implementation manner, determining the updated vectorized graphic of the semantic element according to the initial vectorized graphic and the image region of the semantic element in the reference image includes:
[0015] determining an initial value of a target offset function based on the initial vectorized graph;
[0016] Based on the initial value of the target offset function and the constructed target offset function, the updated vectorized graphic of the semantic element is determined when the value of the target offset function is minimized; the target offset function is used to indicate the pixel error of the corresponding pixel point of the semantic element in the reference image.
[0017] The target offset function here is used to indicate the pixel error of the corresponding pixel point of the semantic element in the reference image. The smaller the pixel error, the closer the semantic element represented by the updated vectorized graphic is to the semantic element represented by the reference image, that is, the semantic element expression capability with higher precision can be obtained. Here, based on the initial value of the target offset function determined by the initial vectorized graphic, multiple iterative operations can be performed until an updated vectorized graphic that meets the requirements is obtained.
[0018] In a possible implementation, the target shift function is constructed according to the following steps:
[0019] For each target sampling point in the point cloud image, based on the vectorized representation information of the semantic element to which the target sampling point belongs and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the image coordinate system where the reference image is located, determine the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located; the vectorized representation information is used to represent the vectorized information of the semantic element;
[0020] The target offset function is constructed according to the projection point coordinate representation information and the image area of the semantic element in the reference image.
[0021] In a possible implementation, constructing the target offset function according to the projection point coordinate representation information and the image area of the semantic element in the reference image includes:
[0022] Searching for a pixel point corresponding to the projection point coordinate representation information from the semantic element in the image area of the reference image;
[0023] The target offset function is constructed based on the pixel value of the found pixel point and the preset pixel value corresponding to the projection point coordinate representation information.
[0024] In a possible implementation, constructing the target offset function based on the pixel value of the found pixel point and the preset pixel value corresponding to the projection point coordinate representation information includes:
[0025] Setting the grayscale value of the image area of the semantic element in the reference image to a target grayscale value, and setting the grayscale values of other image areas in the reference image to other grayscale values different from the target grayscale value;
[0026] The target offset function is constructed based on the first grayscale value of the pixel point found and the preset grayscale value corresponding to the projection point coordinate representation information; the first difference between the preset grayscale value and the first grayscale value is less than the second difference between the preset grayscale value and the other grayscale values.
[0027] Here, in order to facilitate the rapid construction of the target offset function, the grayscale values of the image area of the semantic element in the reference image and other image areas can be set first. In this way, when the preset grayscale value corresponding to the projection point coordinate representation information is determined, the target offset function can be quickly constructed based on the relationship between the preset grayscale value, the first grayscale value and other grayscale values.
[0028] In a possible implementation, for each target sampling point included in the semantic element in the point cloud image, determining the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located based on the vectorized representation information of the semantic element to which the target sampling point belongs and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the image coordinate system where the reference image is located, includes:
[0029] For each of the target sampling points, scale transformation is performed on the target sampling point based on size representation information in the vectorized representation information of the semantic element to which the target sampling point belongs, so as to obtain a transformed sampling point;
[0030] Determine the projection coordinate representation information of the rotation representation information and the translation representation information in the vectorized representation information of the semantic element to which the transformed sampling point belongs in the world coordinate system;
[0031] The projection coordinate representation information is converted into projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located.
[0032] Here, the projection transformation for the target sampling point can be realized in two dimensions (ie, the size dimension and the rotation and translation dimension). In order to better realize the result comparison with the reference image, the target sampling point can be projected and transformed into the image coordinate system where the reference image is located.
[0033] In a possible implementation, the determining, based on the initial value of the target offset function and the constructed target offset function, the updated vectorized graph of the semantic element when the value of the target offset function is minimized includes:
[0034] Determining an initial adjustment amount for the vectorized graph of the semantic element based on an initial value of the target offset function, and determining an updated vectorized graph of the semantic element based on the initial adjustment amount;
[0035] The following steps are executed in a loop:
[0036] Determine the derivative function value of this iteration based on the vectorization information corresponding to the updated vectorized graphic determined in the previous iteration; the derivative function value is obtained by performing a derivative operation on the vectorization representation information of the target offset function;
[0037] Determining an adjustment amount for the vectorized graphic of the semantic element based on the derivative function value of this iteration;
[0038] When the determined adjustment amount is less than the preset adjustment amount, the iteration is stopped, and an updated vectorized graphic for the semantic element is determined based on the adjustment amount.
[0039] Here, multiple adjustments can be made to the vectorized graphics of the semantic elements based on the derivative operation until an updated vectorized graphics having a high-precision semantic element expression capability is obtained.
[0040] In a possible implementation, the method further includes:
[0041] Determining a local point cloud image for the semantic element based on the updated vectorized image;
[0042] A high-precision map is generated based on the local point cloud image of the semantic element.
[0043] Here, the updated vectorized image can be used to express semantic elements with higher precision, so that the accuracy of the generated high-precision map is also improved.
[0044] In a second aspect, an embodiment of the present disclosure further provides a device for processing semantic elements, the device comprising:
[0045] An acquisition module, used to acquire a point cloud image containing semantic elements and a reference image corresponding to the point cloud image;
[0046] A determination module, configured to determine an initial vectorized graphic representing the semantic element based on the semantic element in the point cloud image;
[0047] An updating module is used to determine an updated vectorized graphic of the semantic element based on the initial vectorized graphic and the image area of the semantic element in the reference image; the overlap degree between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; and the updated vectorized graphic is used to update the representation range of the semantic element in the point cloud image.
[0048] In the third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for processing semantic elements as described in the first aspect and any of its various embodiments are performed.
[0049] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the method for processing semantic elements as described in the first aspect and any of its various embodiments.
[0050] For a description of the effects of the apparatus, electronic device, and computer-readable storage medium for processing semantic elements, please refer to the description of the method for processing semantic elements, which will not be repeated here.
[0051] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0053] Figure 1 A flowchart of a method for processing semantic elements provided by an embodiment of the present disclosure is shown;
[0054] Figure 2 A schematic diagram showing a semantic element processing device provided by an embodiment of the present disclosure;
[0055] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0057] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0058] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.
[0059] Research has found that most of the related technologies extract semantic elements such as dotted line segments, road signs, and traffic lights through point clouds scanned by lidar. However, the scanned point clouds are often not dense enough, resulting in low accuracy of the extracted semantic elements, which further leads to low accuracy of the constructed navigation map and cannot meet the needs of current autonomous driving technology.
[0060] Based on the above research, the present disclosure provides a method, device, electronic device and storage medium for processing semantic elements, and the semantic elements expressed are of high accuracy, which further enables the construction of a map with higher accuracy.
[0061] To facilitate understanding of this embodiment, a method for processing semantic elements disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the method for processing semantic elements provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method for processing semantic elements can be implemented by a processor calling a computer-readable instruction stored in a memory.
[0062] See also Figure 1 FIG. 1 is a flowchart of a method for processing semantic elements provided by an embodiment of the present disclosure, and the method includes steps S101 to S103, wherein:
[0063] S101: Acquire a point cloud image containing semantic elements and a reference image corresponding to the point cloud image;
[0064] S102: Determine an initial vectorized graphic representing the semantic element based on the semantic element in the point cloud image;
[0065] S103: Determine an updated vectorized graphic of the semantic element based on the initial vectorized graphic and the image area of the semantic element in the reference image; the overlap degree between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; the updated vectorized graphic is used to update the representation range of the semantic element in the point cloud image.
[0066] In order to facilitate understanding of the method for processing semantic elements provided by the embodiment of the present disclosure, the application scenario of the method can be briefly described first. The above-mentioned method for processing semantic elements can be mainly applied to the construction of high-precision maps. The process of building a high-precision map is the process of restoring the semantic elements with higher precision. The semantic elements here may include traffic elements such as lane lines (including virtuality, color, position, etc.), road signs, traffic lights, and may also include elements such as buildings (such as schools), and may also include other elements, and the embodiment of the present disclosure does not make specific restrictions on this.
[0067] In addition, the navigation information of the high-precision map is used to describe which lane to drive from and which lane to drive to. It is the key information that determines the vehicle's driving trajectory. It can specifically include information such as roads, lanes, lane connection relationships, lane types, lane turning attributes, etc. Navigation information is further refined from the information of semantic elements to a certain extent, so the accuracy of semantic elements also directly determines the performance of the entire high-precision map navigation.
[0068] However, the solution of extracting semantic elements from the point cloud scanned by the laser radar has low precision of the extracted semantic elements because the scanned point cloud is not dense enough. It is precisely to solve the above problem that the embodiment of the present disclosure provides a solution for approximating the semantic elements in the point cloud image and the semantic elements in the reference image based on vectorized representation, so that the precision of the represented semantic elements is higher.
[0069] Among them, the point cloud map in the embodiment of the present disclosure can be collected by using radar equipment, and the point cloud maps collected in different target scenes are also different. The point cloud map here can be dense or sparse, and there is no specific limitation on this.
[0070] The radar equipment here can adopt a rotary scanning laser radar, and can also adopt other radar equipment, and there is no specific restriction on this. Taking the rotary scanning laser radar as an example, the laser radar can obtain the relevant three-dimensional point cloud map in the surrounding environment when rotating and scanning in the horizontal direction. Here, in the process of rotating scanning, the laser radar can adopt a multi-line scanning method, that is, the emission uses multiple laser tubes to emit in sequence, and the structure is that multiple laser tubes are arranged longitudinally, that is, in the process of rotating scanning in the horizontal direction, multi-layer scanning in the vertical direction is performed. There is a certain angle between each laser tube, and the vertical emission field of view can be 30°~40°. In this way, when the laser radar device rotates a scanning angle, a data packet returned by the laser emitted by multiple laser tubes can be obtained. The data packets obtained at each scanning angle are spliced to obtain a frame of point cloud map (corresponding to a rotation of 360° scanning). After the laser radar scans one circle, the collection of a frame of point cloud map can be completed.
[0071] The point cloud image in the embodiment of the present disclosure may include semantic elements. For the sake of illustration, the following description will be given by taking lane lines as semantic elements.
[0072] Here, the initial vectorized graph representing the semantic element can be determined based on the semantic element in the point cloud. In the disclosed embodiment, the initial vectorized graph can be represented by vectorized information such as the size, rotation, and translation of the semantic element. In addition, other information that can represent the semantic element can also be used to represent it, and no specific limitation is made here.
[0073] In a specific application, a parallelogram may be used as an initial vectorized graphic. The length and width of the parallelogram correspond to the size of the semantic element, and the rotation and translation may indicate the rotation angle and translation distance relative to the coordinate system.
[0074] In order to achieve higher-precision expression of semantic elements, the disclosed embodiment can perform multiple updates of the vectorized graphics based on the initial vectorized graphics and the image area of the semantic elements in the reference image. The purpose of the update here is to gradually expand the representation range of the semantic elements in the point cloud until this representation range can match the standard range corresponding to the reference image, that is, the overlap between the projection area of the corresponding updated vectorized graphics after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold.
[0075] The reference image in the embodiment of the present disclosure may be an image captured by a camera device during the process of the radar device acquiring the point cloud image, and the image may contain semantic elements. Here, the lane line is still taken as an example of a semantic element. When a reference image containing a lane line is captured, the image area of the semantic element in the reference image may be determined by image detection.
[0076] It should be noted that the projection area is smaller than the image area of the semantic element in the reference image to a certain extent. This is mainly because in practical applications, the image area here can be the maximum image range for representing the semantic element, and the representation range determined by the three-dimensional point cloud image as a relatively sparse representation will not exceed this maximum image range.
[0077] Considering that determining the updated vectorized graphics is a key step of the method for processing semantic elements provided by the embodiment of the present disclosure, the process of determining the updated vectorized graphics may be specifically described below, which specifically includes the following steps:
[0078] Step 1: determining an initial value of a target offset function based on an initial vectorized graph;
[0079] Step 2: Based on the initial value of the target offset function and the constructed target offset function, determine the updated vectorized graphic of the semantic element when the value of the target offset function is minimized; the target offset function is used to indicate the pixel error of the corresponding pixel point of the semantic element in the reference image.
[0080] Here, a target deviation function may be constructed in advance, and then the constructed target offset function may be iteratively calculated based on an initial value of the target offset function until the updated vectorized graph of the semantic element is determined to minimize the value of the target offset function.
[0081] Since the target offset function is used to indicate the pixel error of the corresponding pixel point of the semantic element in the reference image, the larger the value of the target offset function is, the larger the pixel error is, and vice versa. In order to achieve high overlap mapping, the embodiment of the present disclosure can select the updated vectorized graphic with the smallest error.
[0082] The target offset function can be constructed according to the following steps:
[0083] Step 1: for each target sampling point in the point cloud image, based on the vectorized representation information of the semantic element to which the target sampling point belongs and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the image coordinate system where the reference image is located, determine the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located; the vectorized representation information is used to represent the vectorized information of the semantic element;
[0084] Step 2: Construct a target offset function based on the projection point coordinate representation information and the image area of the semantic element in the reference image.
[0085] The target sampling point here may be a sampling point obtained by sampling the vectorized graphics corresponding to the semantic element. For example, 1 cm sampling may be performed on a parallelogram to construct a target offset function from all sampling points to the reference image.
[0086] In order to construct the target offset function, we can first determine the projection point coordinate representation information of each target sampling point in the point cloud image in the image coordinate system of the reference image, and then compare it with the reference image based on the projection point coordinate representation information. The above process of determining the projection point coordinate representation information can be achieved through the following steps:
[0087] Step 1: for each target sampling point, scale transformation is performed on the target sampling point based on the size representation information in the vectorized representation information of the semantic element to which the target sampling point belongs, so as to obtain a transformed sampling point;
[0088] Step 2: determining the projection coordinate representation information of the rotation representation information and the translation representation information in the vectorized representation information of the semantic element to which the transformed sampling point belongs in the world coordinate system;
[0089] Step 3: Convert the projection coordinate representation information into projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located.
[0090] In the process of realizing the projection point coordinate representation information, the method for processing semantic elements provided by the embodiments of the present disclosure can first perform a scale transformation on the target sampling point based on the size representation information, and based on this size transformation, the size update of the vectorized graphics representing the semantic element can be realized. Then, the rotation representation information and the translation representation information in the vectorized representation information of the semantic element to which the transformed sampling point belongs, and the projection coordinate representation information in the world coordinate system can be determined. This is mainly because the above-mentioned rotation representation information and the translation representation information can represent the conversion relationship between the local point cloud coordinate system where the semantic element is located and the world coordinate system. In this way, based on the conversion relationship between the world coordinate system and the image coordinate system where the reference image is located, the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located can be determined.
[0091] The target offset function constructed by the embodiment of the present disclosure may be pixel-oriented. Here, first, the pixel corresponding to the projection point coordinate representation information may be found from the image area of the semantic element in the reference image, and then the target offset function may be constructed based on the pixel value of the found pixel and the preset pixel value corresponding to the projection point coordinate representation information.
[0092] In order to facilitate understanding of the construction process of the above target offset function, it can be further explained in combination with the following formula.
[0093]
[0094] Among them, I1(P) is used to represent the pixel value of the found pixel point. The preset pixel value used for representation is the coordinate representation information of the projection point of the target sampling point in the image coordinate system where the reference image is located, corresponding to the corresponding pixel value.
[0095] Among them, P corresponds to the target sampling point, and λ corresponds to the size transformation factor. Here we can let It is used to represent the projection point coordinate information of the target sampling point in the image coordinate system of the reference image. Among them, K is the internal parameter matrix, c P represents the projection point coordinate information of the target sampling point under the camera coordinate system, T cw and T wl It is used to represent the transformation relationship between the world coordinate system and the camera coordinate system, and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the world coordinate system.
[0096] In order to quickly realize the construction of the target offset function, it can be implemented based on the grayscale information of the image, including the following steps:
[0097] Step 1: setting the grayscale value of the image area of the semantic element in the reference image to the target grayscale value, and setting the grayscale values of other image areas in the reference image to other grayscale values different from the target grayscale value;
[0098] Step 2: construct a target offset function based on the first grayscale value of the pixel point found and the preset grayscale value corresponding to the projection point coordinate representation information; the first difference between the preset grayscale value and the first grayscale value is less than the second difference between the preset grayscale value and other grayscale values.
[0099] Here, different grayscale values can be set for the image area of the semantic element in the reference image and other image areas in the reference image, respectively. In this way, based on the first grayscale value of the pixel point found and the preset grayscale value corresponding to the projection point coordinate representation information, a target offset function can be quickly constructed.
[0100] In order to achieve a high degree of overlap between the projection area corresponding to the projection point coordinate representation information and the above-mentioned image area, here, the preset grayscale value and the first grayscale value can be set closer, for example, they can both be set to 255.
[0101] The method for processing semantic elements provided by the embodiments of the present disclosure can determine updated vectorized graphics of semantic elements based on multiple iterations.
[0102] Here, firstly, the initial adjustment amount of the vectorized graphics of the semantic element can be determined based on the initial value of the target offset function, and the updated vectorized graphics of the semantic element can be determined based on the initial adjustment amount. Then, the derivative function value of the second iteration is determined based on the vectorization information corresponding to the updated vectorized graphics determined in the first iteration; the derivative function value is obtained by performing a derivative operation on the vectorized representation information of the target offset function; and the adjustment amount of the vectorized graphics of the semantic element is determined based on the derivative function value of the second iteration.
[0103] Here, if it is determined that the adjustment amount is less than the preset adjustment amount, a third iteration is performed, and the adjustment amount is determined according to the derivation calculation process of the second iteration, and so on. If it is determined that the adjustment amount is less than the preset adjustment amount, the iteration is stopped, and an updated vectorized graphic for the semantic element is determined based on the adjustment amount.
[0104] It can be known that the method for processing semantic elements provided by the embodiments of the present disclosure can gradually update the vectorized graphics of the semantic elements. When the adjustment amount reaches a small value, it means that the determined updated vectorized graphics have approached the image area where the semantic elements presented in the reference image are located, thereby stopping the iteration.
[0105] The embodiment of the present disclosure provides an updated vectorized graphic based on high-precision expression capability to update a high-precision map, which specifically includes the following steps:
[0106] Step 1: Determine a local point cloud map for the semantic element based on the updated vectorized graphics;
[0107] Step 2: Generate a high-precision map based on the local point cloud of semantic elements.
[0108] Here, firstly, based on the correspondence between the vectorized information and the point cloud image, the updated vectorized graphics can be restored to a local point cloud image for the semantic elements, and then a high-precision map can be generated based on the local point cloud image.
[0109] It should be noted that in the embodiment of the present disclosure, an initial high-precision map can be generated based on the local point cloud map, and the local point cloud map can also be fused with the initial high-precision map to generate an updated high-precision map. Since the accuracy of the semantic elements represented by the updated vectorized graphics is higher, the accuracy of the updated high-precision map is also improved accordingly.
[0110] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0111] Based on the same inventive concept, an apparatus for processing semantic elements corresponding to the method for processing semantic elements is also provided in the embodiments of the present disclosure. Since the principle of solving the problem by the apparatus in the embodiments of the present disclosure is similar to the method for processing semantic elements mentioned above in the embodiments of the present disclosure, the implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be repeated.
[0112] Reference Figure 2 , which is a schematic diagram of a semantic element processing device provided by an embodiment of the present disclosure, the device includes: an acquisition module 201, a determination module 202, and an update module 203; wherein,
[0113] An acquisition module 201 is used to acquire a point cloud image containing semantic elements and a reference image corresponding to the point cloud image;
[0114] A determination module 202, configured to determine an initial vectorized graphic representing the semantic element based on the semantic element in the point cloud image;
[0115] The updating module 203 is used to determine the updated vectorized graphics of the semantic element based on the initial vectorized graphics and the image area of the semantic element in the reference image; the overlap degree between the projection area of the updated vectorized graphics after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; the updated vectorized graphics is used to update the representation range of the semantic element in the point cloud image.
[0116] In the disclosed embodiments, first, based on the semantic elements in the point cloud map, an initial vectorized graphic representing the semantic element can be determined, and based on the initial vectorized graphic and the image area of the semantic element in the reference image, an updated vectorized graphic of the semantic element can be determined. The updated vectorized graphic here can be obtained by updating the initial vectorized graphic multiple times based on the degree of overlap between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image. When the degree of overlap is greater than a preset threshold, it can be explained to a certain extent that the representation range of the semantic element corresponding to the current updated vectorized graphic in the point cloud map reaches the standard range of the reference image, and the accuracy of the expressed semantic element is relatively high, which further enables the construction of a more accurate map.
[0117] In a possible implementation, the determination module 202 is configured to determine an initial vectorized graphic representing the semantic element based on the semantic element in the point cloud image according to the following steps:
[0118] Obtaining vector information of semantic elements in the point cloud image; the vector information includes one or more of size information, rotation information, and translation information;
[0119] Based on the vectorization information, an initial vectorization graphic is generated.
[0120] In a possible implementation, the updating module 203 is configured to determine an updated vectorized graphic of the semantic element according to the initial vectorized graphic and the image region of the semantic element in the reference image according to the following steps:
[0121] determining an initial value of a target offset function based on the initial vectorized graph;
[0122] Based on the initial value of the target offset function and the constructed target offset function, the updated vectorized graphic of the semantic element is determined when the value of the target offset function is minimized; the target offset function is used to indicate the pixel error of the corresponding pixel point of the semantic element in the reference image.
[0123] In a possible implementation, the updating module 203 is configured to construct a target offset function according to the following steps:
[0124] For each target sampling point in the point cloud image, based on the vectorized representation information of the semantic element to which the target sampling point belongs and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the image coordinate system where the reference image is located, determine the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located; the vectorized representation information is used to represent the vectorized information of the semantic element;
[0125] According to the projection point coordinate representation information and the image area of the semantic element in the reference image, a target offset function is constructed.
[0126] In a possible implementation, the updating module 203 is configured to construct a target offset function according to the projection point coordinate representation information and the image area of the semantic element in the reference image in the following steps:
[0127] Finding pixel points corresponding to the projection point coordinate representation information in the image area of the semantic element in the reference image;
[0128] A target offset function is constructed based on the pixel value of the found pixel point and the preset pixel value corresponding to the projection point coordinate representation information.
[0129] In a possible implementation, the updating module 203 is used to construct a target offset function based on the pixel value of the found pixel point and the preset pixel value corresponding to the projection point coordinate representation information according to the following steps:
[0130] The grayscale value of the image area of the semantic element in the reference image is set to the target grayscale value, and the grayscale values of other image areas in the reference image are set to other grayscale values different from the target grayscale value;
[0131] A target offset function is constructed based on the first grayscale value of the pixel point found and the preset grayscale value corresponding to the projection point coordinate representation information; a first difference between the preset grayscale value and the first grayscale value is less than a second difference between the preset grayscale value and other grayscale values.
[0132] In a possible implementation, the updating module 203 is configured to determine, for each target sampling point included in the semantic element in the point cloud image, projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located based on the vectorized representation information of the semantic element to which the target sampling point belongs and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the image coordinate system where the reference image is located, according to the following steps:
[0133] For each target sampling point, scale transformation is performed on the target sampling point based on size representation information in the vectorized representation information of the semantic element to which the target sampling point belongs, so as to obtain a transformed sampling point;
[0134] Determine the projection coordinate representation information of the rotation representation information and the translation representation information in the vectorized representation information of the semantic element to which the transformed sampling point belongs in the world coordinate system;
[0135] The projection coordinate representation information is converted into the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located.
[0136] In a possible implementation, the updating module 203 is configured to determine the updated vectorized graph of the semantic element when the value of the target offset function is minimized based on the initial value of the target offset function and the constructed target offset function according to the following steps:
[0137] Determining an initial adjustment amount for the vectorized graphic of the semantic element based on an initial value of the target offset function, and determining an updated vectorized graphic of the semantic element based on the initial adjustment amount;
[0138] The following steps are executed in a loop:
[0139] Determine the derivative function value of this iteration based on the vectorized information corresponding to the updated vectorized graphics determined in the previous iteration; the derivative function value is obtained by performing a derivative operation on the vectorized representation information of the target offset function;
[0140] Based on the derivative function value of this iteration, determining the adjustment amount of the vectorized graphics for the semantic element;
[0141] When the determined adjustment amount is less than the preset adjustment amount, the iteration is stopped, and an updated vectorized graphic for the semantic element is determined based on the adjustment amount.
[0142] In a possible implementation manner, the above device further includes:
[0143] The fusion module 204 is used to determine a local point cloud map for the semantic element based on the updated vectorized graphics; and generate a high-precision map based on the local point cloud map of the semantic element.
[0144] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0145] The present disclosure also provides an electronic device, such as Figure 3 , which is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, includes: a processor 301, a memory 302, and a bus 303. The memory 302 stores machine-readable instructions executable by the processor 301 (for example, Figure 2The device acquires the execution instructions corresponding to the module 201, the determination module 202, and the update module 203, etc.), when the electronic device is running, the processor 301 communicates with the memory 302 through the bus 303, and the machine-readable instructions are executed by the processor 301 to perform the following processing:
[0146] Obtaining a point cloud image containing semantic elements and a reference image corresponding to the point cloud image;
[0147] Based on the semantic elements in the point cloud image, determining an initial vectorized graphic representing the semantic elements;
[0148] Based on the initial vectorized graphic and the image area of the semantic element in the reference image, an updated vectorized graphic of the semantic element is determined; the overlap degree between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; the updated vectorized graphic is used to update the representation range of the semantic element in the point cloud image.
[0149] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for processing semantic elements described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.
[0150] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the method for processing semantic elements described in the above method embodiment. Please refer to the above method embodiment for details, which will not be repeated here.
[0151] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0152] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0154] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0155] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0156] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A method for processing semantic elements, characterized in that: The method comprises: Acquire a point cloud image containing semantic elements and a reference image corresponding to the point cloud image; Based on the semantic elements in the point cloud image, determining an initial vectorized graphic representing the semantic elements; According to the initial vectorized graph and the image area of the semantic element in the reference image, an updated vectorized graph of the semantic element is determined; the overlap degree between the projection area of the updated vectorized graph after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; the updated vectorized graph is used to update the representation range of the semantic element in the point cloud image; The step of determining the updated vectorized graph of the semantic element according to the initial vectorized graph and the image area of the semantic element in the reference image comprises: determining an initial value of a target offset function based on the initial vectorized graph; determining the updated vectorized graph of the semantic element when the value of the target offset function is minimized based on the initial value of the target offset function and the constructed target offset function; the target offset function is used to indicate a pixel error of a corresponding pixel point of the semantic element in the reference image; The step of determining the updated vectorized graph of the semantic element based on the initial value of the target offset function and the constructed target offset function so that the value of the target offset function is minimized comprises: determining an initial adjustment amount for the vectorized graph of the semantic element based on the initial value of the target offset function, and determining the updated vectorized graph of the semantic element based on the initial adjustment amount; The following steps are executed in a loop: determining a derivative function value of this iteration based on the vectorization information corresponding to the updated vectorized graphic determined in the previous iteration; the derivative function value is obtained by performing a derivative operation on the vectorized representation information of the target offset function; determining an adjustment amount for the vectorized graphic of the semantic element based on the derivative function value of this iteration; when the determined adjustment amount is less than a preset adjustment amount, stopping iteration, and determining an updated vectorized graphic for the semantic element based on the adjustment amount.
2. The method according to claim 1, characterized in that The determining, based on the semantic element in the point cloud image, an initial vectorized graphic representing the semantic element includes: Acquire vector information of semantic elements in the point cloud image; the vector information includes one or more of size information, rotation information and translation information; Based on the vectorization information, the initial vectorization graphic is generated.
3. The method according to claim 1, characterized in that The target shift function is constructed according to the following steps: For each target sampling point in the point cloud image, based on the vectorized representation information of the semantic element to which the target sampling point belongs and the transformation relationship between the local point cloud coordinate system where the semantic element to which the target sampling point belongs and the image coordinate system where the reference image is located, determine the projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located; the vectorized representation information is used to represent the vectorized information of the semantic element; The target offset function is constructed according to the projection point coordinate representation information and the image area of the semantic element in the reference image.
4. The method according to claim 3, characterized in that The constructing the target offset function according to the projection point coordinate representation information and the image area of the semantic element in the reference image includes: Searching for a pixel point corresponding to the projection point coordinate representation information from the semantic element in the image area of the reference image; The target offset function is constructed based on the pixel value of the found pixel point and the preset pixel value corresponding to the projection point coordinate representation information.
5. The method according to claim 4, characterized in that The target offset function is constructed based on the pixel value of the found pixel point and the preset pixel value corresponding to the projection point coordinate representation information, including: Setting the grayscale value of the image area of the semantic element in the reference image to a target grayscale value, and setting the grayscale values of other image areas in the reference image to other grayscale values different from the target grayscale value; The target offset function is constructed based on the first grayscale value of the pixel point found and the preset grayscale value corresponding to the projection point coordinate representation information; the first difference between the preset grayscale value and the first grayscale value is less than the second difference between the preset grayscale value and the other grayscale values.
6. The method according to any one of claims 3 to 5, characterized in that: The step of determining, for each target sampling point included in the semantic element in the point cloud image, projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located based on the vectorized representation information of the semantic element to which the target sampling point belongs and a transformation relationship between a local point cloud coordinate system where the semantic element to which the target sampling point belongs and an image coordinate system where the reference image is located, includes: For each of the target sampling points, scale transformation is performed on the target sampling point based on size representation information in the vectorized representation information of the semantic element to which the target sampling point belongs, so as to obtain a transformed sampling point; Determine the projection coordinate representation information of the rotation representation information and the translation representation information in the vectorized representation information of the semantic element to which the transformed sampling point belongs in the world coordinate system; The projection coordinate representation information is converted into projection point coordinate representation information of the target sampling point in the image coordinate system where the reference image is located.
7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Determining a local point cloud image for the semantic element based on the updated vectorized image; A high-precision map is generated based on the local point cloud image of the semantic element.
8. A device for processing semantic elements, characterized in that: The device comprises: An acquisition module, used to acquire a point cloud image containing semantic elements and a reference image corresponding to the point cloud image; A determination module, configured to determine an initial vectorized graphic representing the semantic element based on the semantic element in the point cloud image; An updating module, configured to determine an updated vectorized graphic of the semantic element according to the initial vectorized graphic and the image area of the semantic element in the reference image; the overlap between the projection area of the updated vectorized graphic after being projected onto the reference image and the image area of the semantic element in the reference image is greater than a preset threshold; and the updated vectorized graphic is used to update the representation range of the semantic element in the point cloud image; The updating module, when determining the updated vectorized graph of the semantic element according to the initial vectorized graph and the image area of the semantic element in the reference image, is used to: determine an initial value of a target offset function based on the initial vectorized graph; determine the updated vectorized graph of the semantic element when the value of the target offset function is minimized based on the initial value of the target offset function and the constructed target offset function; the target offset function is used to indicate a pixel error of a corresponding pixel point of the semantic element in the reference image; The updating module, when determining the updated vectorized graph of the semantic element in a case where the value of the target offset function is minimized based on the initial value of the target offset function and the constructed target offset function, is used to: determine an initial adjustment amount for the vectorized graph of the semantic element based on the initial value of the target offset function, and determine the updated vectorized graph of the semantic element based on the initial adjustment amount; The following steps are executed in a loop: determining a derivative function value of this iteration based on the vectorization information corresponding to the updated vectorized graphic determined in the previous iteration; the derivative function value is obtained by performing a derivative operation on the vectorized representation information of the target offset function; determining an adjustment amount for the vectorized graphic of the semantic element based on the derivative function value of this iteration; when the determined adjustment amount is less than a preset adjustment amount, stopping iteration, and determining an updated vectorized graphic for the semantic element based on the adjustment amount.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for processing semantic elements as described in any one of claims 1 to 7 are performed.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing semantic elements as described in any one of claims 1 to 7 are executed.
