Shoulder extraction methods and apparatus, electronic devices and computer-readable media

Abstract

The present disclosure provides a shoulder extraction method, device, electronic device and computer-readable medium, which relate to the field of image processing. The method comprises: obtaining point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; dividing the point cloud data of the road into point cloud blocks; inputting the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block; and obtaining the shoulder line of the road according to the shoulder coordinates of each point cloud block. The technical solution of the embodiment of the present invention can determine the shoulder coordinates in the point cloud block of the road through a deep neural network trained based on deep reinforcement learning, and determine the shoulder line of the road according to the shoulder coordinates of each point cloud block. The shoulder extraction method has high accuracy and strong robustness, and can save labor costs.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to a road shoulder extraction method and device, an electronic device, and a computer-readable medium. Background Art

[0002] Road shoulder extraction is an indispensable part of intelligent road perception technology, which can assist in street scene simulation, autonomous driving and high-precision map production. At the same time, it is also a difficult problem in computer vision research. On the one hand, road shoulders exist in various forms, with different heights, textures, colors and widths. On the other hand, the curb area is often blocked by other objects, such as vehicles, pedestrians, leaves and shadows.

[0003] Existing shoulder extraction technologies are mainly divided into rule-based and supervised learning-based methods. In rule-based methods, various rules are often formulated based on people's perception of the appearance of the shoulder. However, such empirical rules are too simple and difficult to cover various scenarios. Therefore, such methods have the problem of poor accuracy and robustness. In supervised learning-based methods, a large number of samples are often collected and annotated in advance to train the model for street scene perception. In order to meet the robustness requirements, this method needs to provide a large number of manually annotated samples in advance. The sample content must be rich, diverse and evenly distributed, which greatly increases the cost of manual annotation. Therefore, this method has the defect of low automation in the training stage.

[0004] Therefore, it is extremely meaningful to find a shoulder extraction method with high accuracy, good robustness and labor saving.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In view of this, the present disclosure provides a road shoulder extraction method and device, an electronic device and a computer-readable medium, which can reduce labor costs while improving the accuracy and robustness of road shoulder extraction.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to the first aspect of the embodiment of the present disclosure, a shoulder extraction method is proposed, which includes: obtaining point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; dividing the point cloud data of the road into point cloud blocks; inputting the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block; and obtaining the shoulder line of the road according to the shoulder coordinates of each point cloud block.

[0009] In some embodiments, dividing the point cloud data of the road into point cloud blocks includes: using a cuboid of a preset size to extract each point cloud block from the point cloud data of the road along the direction of the road according to a preset step length; and projecting the point cloud data in each point cloud block to the coordinate system of each point cloud block.

[0010] In some embodiments, the deep neural network includes a three-dimensional space transformation matrix prediction network, a first two-layer perceptron model, a first feature space transformation matrix prediction network, a first three-layer perceptron model and a classifier based on a recurrent neural network; wherein, the point cloud data in each point cloud block are respectively input into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block, including: dividing the point cloud data in each point cloud block into three-dimensional point cloud data including three-dimensional coordinate information; passing the three-dimensional point cloud data in each point cloud block through the three-dimensional space transformation matrix prediction network to estimate a first transformation matrix, and applying the first transformation matrix to the three-dimensional point cloud data to achieve the three-dimensional point The invention relates to an embodiment of the present invention, wherein the three-dimensional point cloud data is aligned; the aligned three-dimensional point cloud data is subjected to feature extraction through the first two-layer perceptron model; the features of the three-dimensional point cloud data are subjected to the first feature space transformation matrix prediction network to estimate the second transformation matrix, and the second transformation matrix is ​​applied to the features of the three-dimensional point cloud data to achieve feature alignment of the three-dimensional point cloud data; the features of the three-dimensional point cloud data that have undergone feature alignment are subjected to feature extraction through the first three-layer perceptron model to obtain the global feature vector of the three-dimensional point cloud data; the global feature vector of the three-dimensional point cloud data is input into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

[0011] In some embodiments, the point cloud data of the road also includes three-dimensional color information and intensity information, and the deep neural network also includes a four-dimensional space transformation matrix prediction network, a second two-layer perceptron model, a second feature space transformation matrix prediction network and a second three-layer perceptron model; inputting the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block also includes: dividing the point cloud data in each point cloud block into four-dimensional point cloud data including three-dimensional color information and intensity information; passing the four-dimensional point cloud data in each point cloud block through the four-dimensional space transformation matrix prediction network to estimate a third transformation matrix, and applying the third transformation matrix to the four-dimensional point cloud data to achieve the four The invention relates to an embodiment of the present invention, wherein the three-dimensional point cloud data is aligned; the aligned four-dimensional point cloud data is subjected to feature extraction through the second two-layer perceptron model; the features of the four-dimensional point cloud data are subjected to the second feature space transformation matrix prediction network to estimate the fourth transformation matrix, and the fourth transformation matrix is ​​applied to the features of the four-dimensional point cloud data to achieve feature alignment of the four-dimensional point cloud data; the features of the four-dimensional point cloud data that have undergone feature alignment are subjected to feature extraction through the second three-layer perceptron model to obtain the global feature vector of the four-dimensional point cloud data; the three-dimensional point cloud data and the global feature vector of the four-dimensional point cloud data are cascaded and input into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

[0012] In some embodiments, the shoulder extraction method also includes: extracting training point cloud blocks from each point cloud block; inputting each training point cloud block into the deep neural network respectively to determine the shoulder coordinates of each training point cloud block; obtaining a reward value for each training point cloud block based on the shoulder coordinates of each training point cloud block; and autonomously training the deep neural network using the reward value of each training point cloud block to optimize the network parameters of the deep neural network.

[0013] In some embodiments, the reward value of each training point cloud block includes the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block; wherein, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block includes: obtaining the gradient reward value of each training point cloud block according to the point cloud elevation difference on both sides of the shoulder of each training point cloud block; obtaining the height reward value of each training point cloud block according to the point cloud elevation and the driving trajectory elevation within a preset range of each training point cloud shoulder; obtaining the elevation consistency reward value of each training point cloud block according to the point cloud elevation similarity of the inner and outer areas of the shoulder of each training point cloud block; obtaining the parallel driving trajectory reward value of each training point cloud block according to the shoulder slope and the driving trajectory slope of each training point cloud block; obtaining the reward value of each training point cloud block according to the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block.

[0014] In some embodiments, the point cloud data of the road also includes three-dimensional color information and intensity information of the road, and the reward value of each training point cloud block also includes an intensity consistency reward value and a color consistency reward value of each training point cloud block; wherein, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block also includes: obtaining the intensity consistency reward value of each training point cloud block according to the similarity of the point cloud intensity values ​​of the inner and outer areas of the shoulder of each training point cloud block; obtaining the color consistency reward value of each training point cloud block according to the similarity of the point cloud color values ​​of the inner and outer areas of the shoulder of each training point cloud block; obtaining the reward value of each training point cloud block according to the gradient reward value, height reward value, elevation consistency reward value, parallel driving trajectory reward value, intensity consistency reward value and color consistency reward value of each training point cloud block.

[0015] According to the second aspect of the embodiment of the present disclosure, the present disclosure proposes a shoulder extraction device, which includes: a point cloud acquisition module, configured to acquire point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; a point cloud block acquisition module, configured to divide the point cloud data of the road into point cloud blocks; a shoulder coordinate acquisition module, configured to input the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning, so as to determine the shoulder coordinates of each point cloud block; and a shoulder line generation module, configured to obtain the shoulder line of the road according to the shoulder coordinates of each point cloud block.

[0016] According to the third aspect of an embodiment of the present disclosure, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the shoulder extraction methods described above.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, a road shoulder extraction method as described in any one of the above items is implemented.

[0018] According to the shoulder extraction method, device, electronic device and computer-readable medium provided by certain embodiments of the present disclosure, the road point cloud is divided into blocks, and the shoulder coordinates in the point cloud block are determined by means of a deep neural network trained based on deep reinforcement learning, and then the shoulder line of the road is determined based on the shoulder coordinates of the point cloud block. In the technical solution provided by the embodiments of the present disclosure, on the one hand, the road point cloud is segmented to obtain point cloud blocks, thereby ensuring the quantity and quality of samples; on the other hand, the deep neural network is trained using a deep reinforcement learning method, and no manually labeled samples are required during the training process, thus saving manpower and material resources, and the method can automatically generate samples with rich and diverse content and balanced distribution, thereby ensuring the robustness of the trained deep neural network.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. The drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram showing an exemplary system architecture of a road shoulder extraction method or a road shoulder extraction device applied to an embodiment of the present disclosure is shown.

[0022] Figure 2 The figure is a flowchart of a road shoulder extraction method according to an embodiment of the present disclosure.

[0023] Figure 3 The figure is a flow chart of a method for segmenting a point cloud block according to an exemplary embodiment.

[0024] Figure 4 It is a schematic diagram of the deep reinforcement learning process shown according to the relevant technology.

[0025] Figure 5 The present invention is a flowchart of training a deep neural network based on deep reinforcement learning according to an exemplary embodiment.

[0026] Figure 6 It is a structural diagram of a deep neural network trained based on deep reinforcement learning according to an exemplary embodiment.

[0027] Figure 7 The figure is a flowchart of another road shoulder extraction method according to an embodiment of the present disclosure.

[0028] Figure 8 is a structural diagram of another deep neural network trained based on deep reinforcement learning according to an exemplary embodiment.

[0029] Fig. 9 The flowchart of another road shoulder extraction method according to an embodiment of the present disclosure is shown.

[0030] Fig.10 The figure is a block diagram of a road shoulder extraction device according to an exemplary embodiment.

[0031] Fig.11 It is a block diagram of another road shoulder extraction device according to an exemplary embodiment.

[0032] Fig.12 It is a block diagram of yet another road shoulder extraction device according to an exemplary embodiment.

[0033] Fig.13 It is a block diagram of yet another road shoulder extraction device according to an exemplary embodiment.

[0034] Fig.14 It is a block diagram of yet another road shoulder extraction device according to an exemplary embodiment.

[0035] Fig.15 The figure is a schematic diagram showing the structure of a computer system applied to a road shoulder extraction device according to an exemplary embodiment. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0037] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0038] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of one or more elements / components / etc.; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0041] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0042] Figure 1 A schematic diagram showing an exemplary system architecture that can be applied to a road shoulder extraction method or a road shoulder extraction device according to an embodiment of the present disclosure is shown.

[0043] like Figure 1 As shown, the system architecture 100 may include an end point cloud acquisition device 101, terminal devices 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0044] The user may use the point cloud acquisition device 101 to acquire the point cloud of the road. The point cloud acquisition device 101 may be, for example, a laser radar.

[0045] Users can use terminal devices 102 and 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 102 and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.

[0046] The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 102 and 103. The background management server may analyze and process the received request data, and feed back the processing results to the terminal device.

[0047] The server 105 may, for example, obtain point cloud data of the road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; the server 105 may, for example, divide the point cloud data of the road into point cloud blocks; the server 105 may, for example, input the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block; the server may, for example, obtain the shoulder line of the road based on the shoulder coordinates of each point cloud block.

[0048] It should be understood that Figure 1 The number of point cloud acquisition devices, terminal devices, networks and servers are merely illustrative. The server 105 may be a physical server or may be composed of multiple servers. Depending on the implementation requirements, it may have any number of terminal devices, networks and servers.

[0049] Figure 2 The figure is a flow chart of a road shoulder extraction method according to an exemplary embodiment.

[0050] Reference Figure 2 The shoulder extraction method provided by the embodiment of the present disclosure may include the following steps.

[0051] Step S201, obtaining point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road.

[0052] In some embodiments, the shoulder refers to the edge of the road, in a sense the shoulder is the curb.

[0053] In some embodiments, point cloud data refers to a massive collection of points on a target surface feature.

[0054] In some embodiments, a road point cloud may be collected by using a laser radar mounted on a vehicle, wherein the road point cloud data includes three-dimensional coordinate information of the road.

[0055] In some other embodiments, the vehicle may also be equipped with a GPS (Global Positioning System) and an IMU (Inertial Measurement Unit) to collect and obtain the vehicle's driving trajectory, wherein the driving trajectory refers to the location information of the route through which the vehicle travels.

[0056] In some embodiments, when collecting point clouds, it is necessary to try to ensure that the direction of the local driving trajectory is parallel to the road.

[0057] In some embodiments, because the laser radar for point cloud collection is always moving with the vehicle, after the laser radar collects the point cloud data, the point cloud data will be projected into a global coordinate system.

[0058] Step S202, dividing the point cloud data of the road into point cloud blocks.

[0059] In some embodiments, point cloud data of a road may be acquired by a point cloud acquisition device, and the point cloud data includes three-dimensional coordinates of the road in the same global coordinate system.

[0060] In some embodiments, the point cloud blocks formed by point cloud segmentation are consistent in size and shape.

[0061] Step S203: input the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block.

[0062] In some embodiments, after each point cloud block is input into a deep neural network trained based on deep reinforcement learning, the coordinates of the shoulder at the surface position of the point cloud block can be output.

[0063] Step S204, obtaining the shoulder line of the road according to the shoulder coordinates of each point cloud block.

[0064] The shoulder extraction method provided in the above embodiment divides the road point cloud into blocks, determines the shoulder coordinates in the point cloud block by means of a deep neural network trained based on deep reinforcement learning, and then determines the shoulder line of the road based on the shoulder coordinates of the point cloud block. In the technical solution provided in the embodiment of the present disclosure, on the one hand, the road point cloud is segmented to obtain point cloud blocks, thereby ensuring the quantity and quality of samples; on the other hand, the deep neural network is trained using a deep reinforcement learning method, and no manually labeled samples are required during the training process, thus saving manpower and material resources. In addition, the method can automatically generate samples with rich and diverse content and balanced distribution, thereby ensuring the robustness of the trained deep neural network.

[0065] In some embodiments, segmenting the point cloud data of the road into point cloud blocks may include: Figure 3 Steps shown.

[0066] Figure 3 The figure is a flowchart of a method for generating a point cloud block according to an exemplary embodiment.

[0067] Step S2021 , using a cuboid of a preset size, extracting each point cloud block from the point cloud data of the road along the road direction according to a preset step length.

[0068] In some embodiments, the road point cloud may be cut into blocks to facilitate segmented detection of shoulders on the road. For example, a cuboid of a preset size may be used to extract point cloud blocks from the point cloud data of the road along the road direction at a preset step length.

[0069] In some embodiments, the preset dimensions may be, for example, 40 meters in width, 1 meter in length, and 2 meters in height, wherein the length direction may refer to the direction of the local driving track, i.e., the direction of vehicle travel, the height direction is the direction perpendicular to the ground and upward, and the width direction is the direction perpendicular to the length and height.

[0070] In some embodiments, the width should not be too small, and the coordinates of the shoulders on both sides of the road are included in the final segmented point cloud block. In the specific implementation process, those skilled in the art can set it according to their needs, and this disclosure does not limit this.

[0071] In some embodiments, a cuboid of a preset size may be used to extract point cloud blocks in the road along the road direction at a preset step length, wherein the preset step length may be set to 1 meter to ensure that every point on the road is collected.

[0072] Step S2022, projecting the point cloud data in each point cloud block to the coordinate system of each point cloud block.

[0073] In some embodiments, each point cloud block coordinate system refers to a coordinate system generated in each point cloud block. For example, the point cloud block coordinate system can be established with the lower left vertex of the rectangular point cloud block as the origin, the driving track direction as the y-axis, the direction perpendicular to the ground and upward as the z-axis, and the direction perpendicular to the y-axis and the z-axis as the x-axis.

[0074] In some embodiments, the size and shape of each point cloud block are consistent, and the construction rules of each point cloud block coordinate system are also the same. That is, if some point cloud block coordinate systems are constructed with the lower left corner of the cuboid as the origin, then all point cloud block coordinate systems are constructed with the lower left corner of the cuboid as the origin, or if the driving track direction is the y-axis, then all point cloud block coordinate systems must be constructed with the driving track direction as the y-axis, and so on.

[0075] In some embodiments, the point cloud information of the road includes the three-dimensional coordinates of the road in a global coordinate system, and the point cloud information in each point cloud block includes the three-dimensional coordinates of the point cloud in each point cloud coordinate system.

[0076] The point cloud segmentation method provided in the above embodiment cuts the road point cloud into blocks by using a rectangular block of a preset size to form each point cloud block, and projects the point cloud in each point cloud block into the coordinate system of each point cloud block. The point cloud block samples are obtained by the point cloud block generation method, which ensures the consistency of the shape and size of the point cloud block.

[0077] Figure 4 It is a schematic diagram of the deep reinforcement learning process shown according to the relevant technology.

[0078] like Figure 4 As shown, the deep reinforcement learning process mainly involves several key elements such as agent 401, environment 402, action 403, reward 404, and state 405.

[0079] In the process of reinforcement learning, the agent 401 (e.g., a robot) is in the environment 402, and each state 405 is the perception of the environment 402 by the agent 401. After sensing the state 405 of the environment 402, the agent 401 affects the environment 402 by performing an action 403 to change the state of the environment 402, and then the environment 402 will give the agent 401 a reward 404 based on the potential reward function, and finally the agent will adjust its action strategy based on the reward 404. In this way, the agent 401 and the environment 402 continue to interact and generate more data. Reinforcement learning is that the agent performs actions according to a series of action strategies, interacts with the environment, and generates new rewards based on the new state of the environment using the reward function, and then the agent adjusts its action strategy based on the new reward. After multiple iterations, the agent can learn the action strategy required to complete the task.

[0080] Figure 5 The present invention is a flowchart of training a deep neural network based on deep reinforcement learning according to an exemplary embodiment.

[0081] Step S2031, extracting training point cloud blocks from each point cloud block.

[0082] In some embodiments, a cuboid of a preset size can be used to extract each point cloud block from the point cloud data of the road along the road direction according to a preset step length, and then a large number of spatially continuous point cloud blocks can be intercepted as training point cloud blocks. Assuming that the vehicle acquires the point cloud of the road in a continuous time, and assuming that the point clouds in each point cloud block are acquired at the same time, then the spatially continuous point cloud blocks are equivalent to the temporally continuous point cloud blocks.

[0083] Step S2032: Input each training point cloud block into the deep neural network to determine the shoulder coordinates of each training point cloud block.

[0084] In some embodiments, each training point cloud block is input into the deep neural network in sequence according to the order of acquisition time to determine the shoulder coordinates of each training point cloud block.

[0085] In some embodiments, a road generally has two shoulders, and the left shoulder or the right shoulder has at least two intersections with the surface of the cuboid point cloud block (one intersection in front and one intersection in the back). In this embodiment, the coordinates of the shoulders of each training point cloud block can be determined by only determining the coordinates of the two intersections of the left shoulder and the right shoulder of the shoulder with the front of the cuboid point cloud block (for example, the side away from the vehicle in the vehicle's forward direction).

[0086] In some embodiments, during the training process, the determination of the shoulder coordinates of each training point cloud block is random and may not be the actual shoulder coordinates.

[0087] Step S2033, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block.

[0088] In some embodiments, the reward value of each training point cloud block can be determined according to the shoulder coordinates of each training point cloud block according to a set reward function.

[0089] Step S2034, autonomously training the deep neural network using the reward value of each training point cloud block to optimize the network parameters of the deep neural network.

[0090] In some embodiments, the deep neural network optimizes its own network parameters according to the reward value of the training point cloud block, in the hope that the output shoulder coordinates after the next input of the training point cloud block can obtain a higher reward value.

[0091] The above embodiment exemplarily shows the process of training a deep neural network based on deep reinforcement learning. In this method, the training point cloud block is first determined in each point cloud block, and then the training point cloud blocks are sequentially input into the deep neural network. The deep neural network will then output the shoulder coordinates of the training point cloud block, and then the reward of the shoulder coordinates of the training point cloud block can be obtained according to the reward function. The reward of the shoulder coordinates will in turn act on the deep neural network to optimize the parameters of the deep neural network. After multiple iterations, the deep neural network can accurately output the shoulder coordinates corresponding to the point cloud block for the input point cloud block. The method of extracting shoulders by training a deep neural network flowchart based on deep reinforcement learning does not require manual marking of the shoulder position in advance, and can accurately and stably determine the shoulder coordinates.

[0092] In some embodiments, the reward value of each training point cloud block includes the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block; wherein, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block includes: obtaining the gradient reward value of each training point cloud block according to the point cloud elevation difference on both sides of the shoulder of each training point cloud block; obtaining the height reward value of each training point cloud block according to the point cloud elevation and the driving trajectory elevation within a preset range of each training point cloud shoulder; obtaining the elevation consistency reward value of each training point cloud block according to the point cloud elevation similarity of the inner and outer areas of the shoulder of each training point cloud block; obtaining the parallel driving trajectory reward value of each training point cloud block according to the shoulder slope and the driving trajectory slope of each training point cloud block; obtaining the reward value of each training point cloud block according to the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block.

[0093] In some embodiments, when collecting point clouds, it is necessary to try to ensure that the direction of the local driving trajectory is parallel to the road.

[0094] In some embodiments, the vehicle's traveling direction (or road direction) may be considered as the front-rear direction, and the direction perpendicular to the vehicle's traveling direction on a horizontal plane may be considered as the left-right direction.

[0095] In some embodiments, considering that the elevation difference of the point clouds on both sides of the shoulder of the road is generally large, the gradient reward can be defined as:

[0096] f1(a t )=μ1exp(-||H(LP r |a t )-H(RP r |a t )||2 / β1 (1)

[0097] Among them, LP r and RP r The point clouds within the width r of the left side of the shoulder (including the left shoulder and the right shoulder) and the width r of the right side of the shoulder (for example, r = 20cm), H(·) represents the point cloud average function, and μ1 and β1 are empirical parameters. The gradient reward indicates that the greater the elevation difference of the point clouds on both sides of the shoulder, the higher the reward value given.

[0098] In some embodiments, considering that the vehicle's driving track is generally on the road surface in actual situations, and the point cloud on the inner side of the road shoulder close to the driving track also falls on the ground, that is, the elevation difference between the point cloud of the driving track and the point cloud on the inner side of the road shoulder close to the driving track is similar. Therefore, the height reward can be defined as:

[0099] f2(a t )=μ2(1-exp(-||H(NP r |a t )-H orbit ||2 / β1) (2)

[0100] Among them, NP r represents the point cloud within the width of the inner side of the shoulder close to the vehicle trajectory r (for example, r = 10 cm), H orbit represents the average elevation of the point cloud on the driving track in the cut block, and μ2 and β2 are empirical parameters. The height reward indicates that the smaller the elevation difference between the point cloud close to the driving track in the shoulder and the driving track, the higher the reward given.

[0101] In some embodiments, the shoulder coordinates of the trained point cloud blocks are known, and the shoulders on the same shoulder line are connected in spatial order to determine the left shoulder line and the right shoulder line of the shoulder. Thus, the closed loop area formed by the left and right shoulder lines can be determined. Considering that the elevation values ​​of the point clouds in the closed loop area formed by the shoulder lines are similar, while the elevation values ​​of the point clouds inside and outside the closed loop area are not similar, the elevation consistency reward can be defined as:

[0102]

[0103] Among them, AP is a set of sampling point clouds generated by random sampling in the closed-loop area, BP is a set of sampling point clouds generated by random sampling in the training point cloud block (including both the closed-loop area and outside the closed-loop area), and VP = {AP + BP}.

[0104] in,

[0105] asso H (A, B) = cut H (A, B) = ∑ u＝A，v＝B exp(-||H(u)-H(v)||2 / β3) (4)

[0106] The elevation consistency reward indicates that if the elevation similarity of the point cloud within the closed loop area formed by the shoulder line is high and the elevation similarity of the point cloud inside and outside the closed loop area is low, the higher the reward given.

[0107] In some embodiments, considering that the driving track and the shoulder line in the area are parallel, the parallel driving track reward can be:

[0108] f4(a t )=μ4(exp(-||T(Curb|a t )-T orbit ||2 / β2) (5)

[0109] Among them, T orbit Represents the slope of the driving trajectory, and T(Curb) is the slope of the shoulder line.

[0110] In some embodiments, the training point cloud modules are input into the deep neural network in spatial order. The coordinates of the corresponding output shoulder are output for each training point cloud module. The shoulder of the current training point cloud module is connected with the shoulder coordinates of the previous training point cloud module, and the local shoulder line can be calculated, and then the slope T (Curb) of the shoulder line can be calculated.

[0111] In some embodiments, the parallel driving trajectory reward indicates that the higher the similarity between the slope of the shoulder line and the slope of the driving trajectory, the higher the reward given.

[0112] In some embodiments, the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block are added together to obtain the reward of the deep neural network. i (i=1, 2, 3, 4) to adjust the contribution ratio of each reward. i How to set (i=1, 2, 3, 4) is subject to the actual needs of those skilled in the art, and the present disclosure does not impose any limitation on this.

[0113] In some other embodiments, the point cloud data of the road also includes three-dimensional color information and intensity information of the road, and the reward value of each training point cloud block also includes an intensity consistency reward value and a color consistency reward value of each training point cloud block; wherein, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block also includes: obtaining the intensity consistency reward value of each training point cloud block according to the similarity of the point cloud intensity values ​​of the inner and outer areas of the shoulder of each training point cloud block; obtaining the color consistency reward value of each training point cloud block according to the similarity of the point cloud color values ​​of the inner and outer areas of the shoulder of each training point cloud block; obtaining the reward value of each training point cloud block according to the gradient reward value, height reward value, elevation consistency reward value, parallel driving trajectory reward value, intensity consistency reward value and color consistency reward value of each training point cloud block.

[0114] In some embodiments, the intensity consistency reward is similar to the elevation consistency reward and can be expressed as:

[0115]

[0116] Where I represents the strength of the point cloud. The strength consistency reward indicates that the strength similarity of the point cloud within the closed loop area formed by the shoulder line is high, and the strength similarity of the point cloud inside and outside the closed loop area is low, and the higher the reward given.

[0117] In some embodiments, the color consistency reward is also similar to the elevation consistency reward and can be expressed as:

[0118]

[0119] Where c represents the color of the point cloud. The color consistency reward indicates that the color similarity of the point cloud within the closed loop area formed by the shoulder line is high, and the color similarity of the point cloud inside and outside the closed loop area is low, and the higher the reward given.

[0120] Figure 6 It is a structural diagram of a deep neural network trained based on deep reinforcement learning according to an exemplary embodiment.

[0121] In some embodiments, the deep neural network includes a three-dimensional space transformation matrix prediction network 601, a first two-layer perceptron model 602, a first feature space transformation matrix prediction network 603, a first three-layer perceptron model 604 and a recurrent neural network-based classifier 605.

[0122] In some embodiments, the point cloud data in the training point cloud block is segmented into three-dimensional point cloud data including three-dimensional coordinate information. Inputting the training point cloud block into the deep neural network means inputting the three-dimensional point cloud data n×3 in the point cloud block into the deep neural network, where n represents the number of point clouds in the training point cloud block.

[0123] In some embodiments, the three-dimensional point cloud data in the training point cloud block is passed through the three-dimensional space transformation matrix prediction network 601 to estimate a first transformation matrix, and the first transformation matrix is ​​applied to the three-dimensional point cloud data to achieve alignment of the three-dimensional point cloud data.

[0124] In some embodiments, the aligned three-dimensional point cloud data is subjected to feature extraction through the first double-layer perceptron model 602. The first double-layer perceptron model 602 can extract features of the three-dimensional point cloud data, for example, can generate an n×64 feature vector.

[0125] In some embodiments, the feature vector of the three-dimensional point cloud data is passed through the first feature space transformation matrix prediction network 603 to estimate the second transformation matrix, and the second transformation matrix is ​​applied to the features of the three-dimensional point cloud data to achieve alignment of the features of the three-dimensional point cloud data.

[0126] In some embodiments, the features of the three-dimensional point cloud data that have undergone feature alignment are extracted using the first three-layer perceptron model 604 to obtain a global feature vector of the three-dimensional point cloud data. For example, the feature vector of the three-dimensional point cloud of the training point cloud block passes through the second transformation matrix and then passes through its first three-layer perceptron model to generate a 1×256 global feature vector.

[0127] In some embodiments, the first three-layer perceptron model 604 includes a downsampling pooling layer. The downsampling pooling layer takes the maximum value in each channel point cloud, for example, taking the maximum value of n×256 data to obtain 1×256 data.

[0128] In some embodiments, the global feature vector of the three-dimensional point cloud data is input into the recurrent neural network-based classifier 605 for classification to determine the shoulder coordinates of each point cloud block.

[0129] In some embodiments, the recurrent neural network based classifier 605 includes a cascade (contact) layer, a long short-term memory (LSTM) in the recurrent neural network, and a normalized exponential function (softmax).

[0130] In some embodiments, the cascade layer can connect the same number of data from different channels in series. Figure 6 As shown in FIG. 1 , when the global feature vector of the three-dimensional point cloud data is input into the recurrent neural network-based classifier 605, the first cascade layer 6051 combines the global feature vector and the previous moment reward r t-16053, and then the concatenated vector is introduced into the first long short time neural network 6052 through a point cloud extraction matrix. Further, the second cascade layer 6054 combines the features output by the first long short time neural network 6052 and the behavior a at the previous moment. t-1 6055 is cascaded and imported into the second long short-term neural network 6056, and finally a feature including the previous moment reward and the previous moment behavior is obtained. For example, the feature can be a 1×256 feature.

[0131] In some embodiments, the previous moment behavior and the previous moment reward refer to the shoulder coordinates of the training point cloud block output when the classifier was last trained and the reward corresponding to the shoulder coordinates.

[0132] In some embodiments, a training point cloud block will output the coordinates of two shoulder point clouds, and the point cloud coordinates of each shoulder include horizontal coordinates and vertical coordinates (assuming that the two shoulder points are on the same plane, so the two shoulder points do not have height coordinates). In some embodiments, in order to independently predict the above 4 coordinates and behavior values, a fully connected network is used to independently predict the above 5 parts (the horizontal coordinates LX t , left shoulder vertical coordinate LY t , horizontal coordinate of right shoulder RX t , right shoulder vertical coordinate RY t ) to make independent predictions.

[0133] In some embodiments, the point cloud data in each point cloud block are respectively input into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block, which may include: Figure 7 Steps shown.

[0134] Step S701 , dividing the point cloud data in each point cloud block into three-dimensional point cloud data including three-dimensional coordinate information.

[0135] Step S702, the three-dimensional point cloud data in each point cloud block is passed through the three-dimensional space transformation matrix prediction network to estimate a first transformation matrix, and the first transformation matrix is ​​applied to the three-dimensional point cloud data to achieve alignment of the three-dimensional point cloud data.

[0136] Step S703: extract features from the aligned three-dimensional point cloud data using the first two-layer perceptron model.

[0137] Step S704: The features of the three-dimensional point cloud data are passed through the first feature space transformation matrix prediction network to estimate a second transformation matrix, and the second transformation matrix is ​​applied to the features of the three-dimensional point cloud data to achieve alignment of the features of the three-dimensional point cloud data.

[0138] Step S705: extract features of the three-dimensional point cloud data that have undergone feature alignment using the first three-layer perceptron model to obtain a global feature vector of the three-dimensional point cloud data.

[0139] Step S706: input the global feature vector of the three-dimensional point cloud data into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

[0140] The road shoulder extraction method provided in the above embodiment can realize the extraction of the road shoulder coordinates in the point cloud block by automatically optimizing the network parameters through rewards. The method is simple, convenient and highly accurate.

[0141] In some other embodiments, the point cloud data of the road also includes three-dimensional color information and intensity information, and the deep neural network can be used as follows: Figure 8 The diagram includes: a three-dimensional space transformation matrix prediction network 801, a first two-layer perceptron model 802, a first feature space transformation matrix prediction network 803, a first three-layer perceptron model 804 and a recurrent neural network-based classifier 809, a four-dimensional space transformation matrix prediction network 805, a second two-layer perceptron model 806, a second feature space transformation matrix prediction network 807 and a second three-layer perceptron model 808.

[0142] In some embodiments, the point cloud data in each point cloud block are respectively input into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block. Figure 7 The embodiment shown also includes the following Fig. 9 The illustrated embodiment provides steps.

[0143] Step S901 , dividing the point cloud data in each point cloud block into four-dimensional point cloud data including three-dimensional color information and intensity information.

[0144] Step S902, the four-dimensional point cloud data in each point cloud block is passed through the four-dimensional space transformation matrix prediction network to estimate a third transformation matrix, and the third transformation matrix is ​​applied to the four-dimensional point cloud data to achieve alignment of the four-dimensional point cloud data.

[0145] Step S903: extracting features from the aligned four-dimensional point cloud data through the second two-layer perceptron model.

[0146] Step S904, the features of the four-dimensional point cloud data are passed through the second feature space transformation matrix prediction network to estimate a fourth transformation matrix, and the fourth transformation matrix is ​​applied to the features of the four-dimensional point cloud data to achieve alignment of the features of the four-dimensional point cloud data.

[0147] Step S905: extract features of the aligned four-dimensional point cloud data using the second three-layer perceptron model to obtain a global feature vector of the four-dimensional point cloud data.

[0148] Step S906, cascading the global feature vectors of the three-dimensional point cloud data and the four-dimensional point cloud data, and inputting them into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

[0149] The shoulder extraction method provided in the above embodiment uses the color information and intensity information in the point cloud block on the basis of the three-dimensional coordinates of the point cloud block, which greatly improves the accuracy and robustness of the shoulder extraction of the point cloud block.

[0150] Fig.10 is a block diagram of a road shoulder extraction device according to an exemplary embodiment. Fig.10 The device 1000 includes: a point cloud acquisition module 1001, a point cloud block acquisition module 1002, a shoulder coordinate acquisition module 1003, and a shoulder line generation module 1004.

[0151] The road point cloud acquisition module 1001 may be configured to acquire point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road.

[0152] The point cloud block acquisition module 1002 may be configured to divide the point cloud data of the road into point cloud blocks.

[0153] The shoulder coordinate acquisition module 1003 can be configured to input the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block.

[0154] The shoulder line generation module 1004 may be configured to obtain the shoulder line of the road according to the shoulder coordinates of each point cloud block.

[0155] Fig.11 is a block diagram of another road shoulder extraction device according to an exemplary embodiment. Fig.11 The point cloud block acquisition module 1002 may include: a point cloud module generation unit 10021 and a coordinate conversion unit 10022.

[0156] The point cloud module generation unit 10021 may be configured to use a cuboid of a preset size to extract each point cloud block from the point cloud data of the road along the road direction at a preset step length.

[0157] The coordinate conversion unit 10022 may be configured to project the point cloud data in each point cloud block into the coordinate system of each point cloud block.

[0158] Fig.12 A block diagram of yet another road shoulder extraction device is shown according to an exemplary embodiment.

[0159] In some embodiments, the deep neural network includes a three-dimensional space transformation matrix prediction network, a first two-layer perceptron model, a first feature space transformation matrix prediction network, a first three-layer perceptron model, and a classifier based on a recurrent neural network.

[0160] Reference Fig.12 The shoulder coordinate acquisition module 1003 includes: a three-dimensional point cloud acquisition unit 10031, a three-dimensional point cloud alignment unit 10032, a feature extraction unit 10033, a feature alignment unit 10034, a global feature vector acquisition unit 10035, and a shoulder coordinate determination unit 10036.

[0161] Among them, the three-dimensional point cloud acquisition unit 10031 can be configured to divide the point cloud data in each point cloud block into three-dimensional point cloud data including three-dimensional coordinate information.

[0162] The three-dimensional point cloud alignment unit 10032 can be configured to pass the three-dimensional point cloud data in each point cloud block through the three-dimensional space transformation matrix prediction network to estimate the first transformation matrix, and apply the first transformation matrix to the three-dimensional point cloud data to achieve alignment of the three-dimensional point cloud data.

[0163] The feature extraction unit 10033 can be configured to perform feature extraction on the aligned three-dimensional point cloud data through the first two-layer perceptron model.

[0164] The feature alignment unit 10034 can be configured to pass the features of the three-dimensional point cloud data through the first feature space transformation matrix prediction network to estimate the second transformation matrix, and apply the second transformation matrix to the features of the three-dimensional point cloud data to achieve alignment of the features of the three-dimensional point cloud data.

[0165] The global feature vector acquisition unit 10035 can be configured to extract features of the three-dimensional point cloud data that have undergone feature alignment using the first three-layer perceptron model to obtain a global feature vector of the three-dimensional point cloud data.

[0166] The shoulder coordinate determination unit 10036 inputs the global feature vector of the three-dimensional point cloud data into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

[0167] Fig.13 A block diagram of another road shoulder extraction device is shown according to an exemplary embodiment.

[0168] In some embodiments, the point cloud data of the road also includes three-dimensional color information and intensity information, and the deep neural network also includes a four-dimensional space transformation matrix prediction network, a second two-layer perceptron model, a second feature space transformation matrix prediction network and a second three-layer perceptron model.

[0169] Reference Fig.13 The shoulder coordinate acquisition module 1003 also includes: a four-dimensional point cloud acquisition unit 10037, a four-dimensional point cloud alignment unit 10038, a four-dimensional data feature extraction unit 10039, a four-dimensional data feature alignment unit 100310, a four-dimensional data global feature vector acquisition unit 100311, and a multi-dimensional data shoulder coordinate determination unit 100312.

[0170] Among them, the four-dimensional point cloud acquisition unit 10037 can be configured to divide the point cloud data in each point cloud block into four-dimensional point cloud data including three-dimensional color information and intensity information.

[0171] The four-dimensional point cloud alignment unit 10038 can be configured to pass the four-dimensional point cloud data in each point cloud block through the four-dimensional space transformation matrix prediction network to estimate a third transformation matrix, and apply the third transformation matrix to the four-dimensional point cloud data to achieve alignment of the four-dimensional point cloud data.

[0172] The four-dimensional data feature extraction unit 10039 can be configured to perform feature extraction on the aligned four-dimensional point cloud data through the second two-layer perceptron model.

[0173] The four-dimensional data feature alignment unit 100310 can be configured to pass the features of the four-dimensional point cloud data through the second feature space transformation matrix prediction network to estimate a fourth transformation matrix, and apply the fourth transformation matrix to the features of the four-dimensional point cloud data to achieve alignment of the features of the four-dimensional point cloud data.

[0174] The four-dimensional data global feature vector acquisition unit 100311 can be configured to extract features of the four-dimensional point cloud data that have undergone feature alignment using the second three-layer perceptron model to obtain a global feature vector of the four-dimensional point cloud data.

[0175] The multi-dimensional data shoulder coordinate determination unit 100312 can be configured to cascade the global feature vectors of the three-dimensional point cloud data and the four-dimensional point cloud data, and input them into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

[0176] Fig.14 A block diagram of another road shoulder extraction device according to an exemplary embodiment. Fig.14The shoulder extraction device 1000 also includes: a training point cloud block acquisition module 1005, a shoulder coordinate determination module 1006, a reward value acquisition module 1007, and a network parameter optimization module 1008.

[0177] The training point cloud block acquisition module 1005 may be configured to extract training point cloud blocks from each point cloud block.

[0178] The shoulder coordinate determination module 1006 may be configured to input each training point cloud block into the deep neural network to determine the shoulder coordinates of each training point cloud block.

[0179] The reward value acquisition module 1007 may be configured to acquire the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block.

[0180] The network parameter optimization module 1008 can be configured to autonomously train the deep neural network using the reward value of each training point cloud block to optimize the network parameters of the deep neural network.

[0181] In some embodiments, the reward value of each training point cloud block includes the gradient reward value, height reward value, elevation consistency reward value and parallel driving track reward value of each training point cloud block. Among them, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block includes: obtaining the gradient reward value of each training point cloud block according to the elevation difference of the point cloud on both sides of the shoulder of each training point cloud block; obtaining the height reward value of each training point cloud block according to the point cloud elevation within the preset range of each training point cloud shoulder and the driving track elevation; obtaining the elevation consistency reward value of each training point cloud block according to the point cloud elevation similarity of the inner and outer areas of the shoulder of each training point cloud block; obtaining the parallel driving track reward value of each training point cloud block according to the shoulder slope and driving track slope of each training point cloud block; obtaining the reward value of each training point cloud block according to the gradient reward value, height reward value, elevation consistency reward value and parallel driving track reward value of each training point cloud block.

[0182] In some embodiments, the point cloud data of the road also includes three-dimensional color information and intensity information of the road, and the reward value of each training point cloud block also includes an intensity consistency reward value and a color consistency reward value of each training point cloud block; wherein, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block also includes: obtaining the intensity consistency reward value of each training point cloud block according to the similarity of the point cloud intensity values ​​of the inner and outer areas of the shoulder of each training point cloud block; obtaining the color consistency reward value of each training point cloud block according to the similarity of the point cloud color values ​​of the inner and outer areas of the shoulder of each training point cloud block; obtaining the reward value of each training point cloud block according to the gradient reward value, height reward value, elevation consistency reward value, parallel driving trajectory reward value, intensity consistency reward value and color consistency reward value of each training point cloud block.

[0183] Since the various functional modules of the road shoulder extraction device 1000 of the exemplary embodiment of the present disclosure correspond to the steps of the exemplary embodiment of the road shoulder extraction method described above, they will not be described in detail here.

[0184] Reference below Fig.15 , which shows a schematic diagram of the structure of a computer system 1500 of a terminal device suitable for implementing an embodiment of the present application. Fig.15 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0185] like Fig.15 As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1502 or a program loaded from a storage part 1508 into a random access memory (RAM) 1503. In the RAM 1503, various programs and data required for the operation of the system 1500 are also stored. The CPU 1501, the ROM 1502, and the RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0186] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed, so that a computer program read therefrom is installed into the storage section 1508 as needed.

[0187] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 1509, and / or installed from a removable medium 1511. When the computer program is executed by a central processing unit (CPU) 1501, the above-mentioned functions defined in the system of the present application are executed.

[0188] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0189] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0190] The units involved in the embodiments described in the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a sending unit, an acquiring unit, a determining unit, and a first processing unit. The names of these units do not constitute limitations on the units themselves in some cases.

[0191] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device can implement functions including: obtaining point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; dividing the point cloud data of the road into point cloud blocks; inputting the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block; and obtaining the shoulder line of the road according to the shoulder coordinates of each point cloud block.

[0192] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for a computing device (which can be a personal computer, a server, a mobile terminal, or a smart device, etc.) to execute the method according to the embodiment of the present disclosure, for example Figure 2 One or more of the steps shown.

[0193] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0194] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not applied for by the present disclosure. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0195] It should be understood that the present disclosure is not limited to the detailed structures, drawings or implementations shown herein, but rather the present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A road shoulder extraction method, characterized in that: include: Acquire point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; Segmenting the point cloud data of the road into point cloud blocks; The point cloud data in each point cloud block are respectively input into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block; Obtaining the shoulder line of the road according to the shoulder coordinates of each point cloud block; Extract training point cloud blocks from each point cloud block; Inputting each training point cloud block into the deep neural network to determine the shoulder coordinates of each training point cloud block; Obtain the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block; Autonomously training the deep neural network using the reward value of each training point cloud block to optimize the network parameters of the deep neural network; The reward value of each training point cloud block includes the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block; wherein, the reward value of each training point cloud block is obtained according to the shoulder coordinates of each training point cloud block, including: Obtain the gradient reward value of each training point cloud block according to the elevation difference of the point cloud on both sides of the shoulder of each training point cloud block; Obtain the height reward value of each training point cloud block according to the point cloud elevation and the driving trajectory elevation within the preset range of each training point cloud shoulder; The elevation consistency reward value of each training point cloud block is obtained according to the point cloud elevation similarity of the inner and outer areas of the shoulder of each training point cloud block; Obtain the parallel driving trajectory reward value of each training point cloud block according to the shoulder slope and driving trajectory slope of each training point cloud block; The reward value of each training point cloud block is obtained according to the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block.

2. The method according to claim 1, characterized in that: The step of dividing the point cloud data of the road into point cloud blocks comprises: Using a cuboid of a preset size, extracting each point cloud block from the point cloud data of the road along the road direction according to a preset step length; Project the point cloud data in each point cloud block to the coordinate system of each point cloud block.

3. The method according to claim 1, characterized in that: The deep neural network includes a three-dimensional space transformation matrix prediction network, a first two-layer perceptron model, a first feature space transformation matrix prediction network, a first three-layer perceptron model and a classifier based on a recurrent neural network; wherein, the point cloud data in each point cloud block are respectively input into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block, including: Segmenting the point cloud data in each point cloud block into three-dimensional point cloud data including three-dimensional coordinate information; The three-dimensional point cloud data in each point cloud block is passed through the three-dimensional space transformation matrix prediction network to estimate a first transformation matrix, and the first transformation matrix is ​​applied to the three-dimensional point cloud data to achieve alignment of the three-dimensional point cloud data; Extract features from the aligned three-dimensional point cloud data through the first two-layer perceptron model; Passing the features of the three-dimensional point cloud data through the first feature space transformation matrix prediction network to estimate a second transformation matrix, and applying the second transformation matrix to the features of the three-dimensional point cloud data to achieve alignment of the features of the three-dimensional point cloud data; Extracting features of the three-dimensional point cloud data after feature alignment using the first three-layer perceptron model to obtain a global feature vector of the three-dimensional point cloud data; The global feature vector of the three-dimensional point cloud data is input into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

4. The method according to claim 3, characterized in that: The point cloud data of the road also includes three-dimensional color information and intensity information, the deep neural network also includes a four-dimensional space transformation matrix prediction network, a second two-layer perceptron model, a second feature space transformation matrix prediction network and a second three-layer perceptron model; inputting the point cloud data in each point cloud block into the deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block also includes: Segmenting the point cloud data in each point cloud block into four-dimensional point cloud data including three-dimensional color information and intensity information; The four-dimensional point cloud data in each point cloud block is passed through the four-dimensional space transformation matrix prediction network to estimate a third transformation matrix, and the third transformation matrix is ​​applied to the four-dimensional point cloud data to achieve alignment of the four-dimensional point cloud data; Extracting features from the aligned four-dimensional point cloud data through the second two-layer perceptron model; Passing the features of the four-dimensional point cloud data through the second feature space transformation matrix prediction network to estimate a fourth transformation matrix, and applying the fourth transformation matrix to the features of the four-dimensional point cloud data to achieve alignment of the features of the four-dimensional point cloud data; Extracting features of the four-dimensional point cloud data after feature alignment using the second three-layer perceptron model to obtain a global feature vector of the four-dimensional point cloud data; The global feature vectors of the three-dimensional point cloud data and the four-dimensional point cloud data are cascaded and input into the recurrent neural network-based classifier for classification to determine the shoulder coordinates of each point cloud block.

5. The method according to claim 1, characterized in that: The point cloud data of the road also includes three-dimensional color information and intensity information of the road, and the reward value of each training point cloud block also includes an intensity consistency reward value and a color consistency reward value of each training point cloud block; wherein, obtaining the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block also includes: Obtain the intensity consistency reward value of each training point cloud block according to the similarity of the point cloud intensity values ​​of the inner and outer areas of the shoulder of each training point cloud block; Obtain the color consistency reward value of each training point cloud block according to the similarity of the point cloud color values ​​of the inner and outer areas of the shoulder of each training point cloud block; The reward value of each training point cloud block is obtained according to the gradient reward value, height reward value, elevation consistency reward value, parallel driving trajectory reward value, intensity consistency reward value and color consistency reward value of each training point cloud block.

6. A road shoulder extraction device, characterized in that: include: A point cloud acquisition module, configured to acquire point cloud data of a road, wherein the point cloud data of the road includes three-dimensional coordinate information of the road; A point cloud block acquisition module, configured to divide the point cloud data of the road into point cloud blocks; A shoulder coordinate acquisition module is configured to input the point cloud data in each point cloud block into a deep neural network trained based on deep reinforcement learning to determine the shoulder coordinates of each point cloud block; A shoulder line generation module, configured to obtain the shoulder line of the road according to the shoulder coordinates of each point cloud block; A training point cloud block acquisition module is configured to extract training point cloud blocks from each point cloud block; A shoulder coordinate determination module is configured to input each training point cloud block into the deep neural network to determine the shoulder coordinates of each training point cloud block; The reward value acquisition module is configured to acquire the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block; the reward value of each training point cloud block includes the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block; wherein, acquiring the reward value of each training point cloud block according to the shoulder coordinates of each training point cloud block includes: acquiring the gradient reward value of each training point cloud block according to the elevation difference of the point cloud on both sides of the shoulder of each training point cloud block; and The height reward value of each training point cloud block is obtained based on the point cloud elevation and the driving trajectory elevation within the range; the elevation consistency reward value of each training point cloud block is obtained based on the point cloud elevation similarity of the inner and outer areas of the shoulder of each training point cloud block; the parallel driving trajectory reward value of each training point cloud block is obtained based on the shoulder slope and driving trajectory slope of each training point cloud block; the reward value of each training point cloud block is obtained based on the gradient reward value, height reward value, elevation consistency reward value and parallel driving trajectory reward value of each training point cloud block; The network parameter optimization module is configured to autonomously train the deep neural network using the reward value of each training point cloud block to optimize the network parameters of the deep neural network.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images