Point cloud simulation method and device for passable area, storage medium and electronic device

Abstract

The invention provides a point cloud simulation method and device for a passable area, a storage medium and an electronic device.The method comprises the steps that a virtual environment of a vehicle model is constructed, an obstacle set is added to the virtual environment, the obstacle set comprises a plurality of static obstacles and a plurality of dynamic obstacles, and the static obstacles and the dynamic obstacles correspond to the static obstacles and the dynamic obstacles. The vehicle model is provided with a simulation sensor which is the same as a real vehicle in position; emitting detection rays to the periphery according to the resolution of the analog sensor; calculating an intersection point between the detection ray and each obstacle in the obstacle set; judging whether the intersection point meets a preset condition or not; and if the intersection point meets the preset condition, outputting the intersection point as a simulation point cloud of the passable area boundary of the vehicle model, and obtaining an original simulation point set of the passable area. According to the embodiment, the defect that simulation software cannot directly simulate the sensor output point cloud is overcome, the simulation software can directly simulate the sensor point cloud, and the data consistency of the simulation point cloud and the real vehicle sensor point cloud is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a point cloud simulation method and device for a passable area, a storage medium and an electronic device. BACKGROUND

[0002] In related technologies, with the rapid development of automatic driving technology, automatic driving simulation testing has gradually become an important part of the development process of automatic driving systems. A simulator or virtual environment is used to simulate vehicles and environments, and automatic driving system software is embedded in the simulator to complete closed-loop testing. Simulation testing is located in the early stage of the entire testing process, aiming to achieve faster, more efficient, more accurate and higher scene coverage.

[0003] In simulation testing, virtual scenarios and virtual sensor simulations are provided by the simulation environment, sensor data is constructed as intelligent driving algorithm input, so as to test the performance of intelligent driving algorithms in different scenarios. Among them, environment perception as the first ring of intelligent driving directly affects vehicle decision control and is the key to realizing safe and reliable automatic driving, so high-precision simulation of sensors in simulation testing is particularly important. Point cloud data, as one of the sensor output types, directly reflects important information carriers of spatial structure and is widely used in obstacle detection, path planning and navigation positioning of automatic driving vehicles.

[0004] In related technologies, different solutions and data formats are used for sensors and their output data by different intelligent driving vehicles, and there are significant differences in the sensor simulation schemes and outputs provided by existing simulation software. It cannot be directly adapted to the corresponding intelligent driving vehicle scheme in the simulation process, and the obstacles in the environment are not considered, resulting in inaccurate simulated point cloud data and inaccurate passable area, and thus poor simulation effect.

[0005] In view of the above problems in related technologies, there is no efficient and accurate solution. SUMMARY

[0006] The present application provides a point cloud simulation method and device for a passable area, a storage medium and an electronic device to solve the technical problems in related technologies.

[0007] According to one embodiment of the present application, a point cloud simulation method for a passable area is provided, comprising: constructing a virtual environment of a vehicle model, and adding a set of obstacles in the virtual environment, wherein the set of obstacles comprises a plurality of static obstacles and a plurality of dynamic obstacles, and a simulation sensor identical to a real vehicle position is configured on the vehicle model; emitting a detection ray in all directions according to a resolution of the simulation sensor; calculating an intersection point between the detection ray and each obstacle in the set of obstacles; judging whether the intersection point meets a preset condition; if the intersection point meets the preset condition, outputting the intersection point as a simulation point cloud of a passable area boundary of the vehicle model, to obtain an original simulation point set of the passable area.

[0008] Optionally, judging whether the intersection point meets the preset condition comprises: determining a passable area detection range of the vehicle model; judging whether the intersection point is within the passable area detection range and whether the intersection point is blocked by an obstacle in the set of obstacles; and if the intersection point is within the passable area detection range and the intersection point is not blocked by an obstacle in the set of obstacles, determining that the intersection point meets the preset condition.

[0009] Optionally, after judging whether the intersection point meets the preset condition, the method further comprises: if the intersection point does not meet the preset condition, obtaining a farthest detection point of the detection ray; and outputting the farthest detection point as a simulation point cloud of the passable area of the vehicle model.

[0010] Optionally, adding the set of obstacles in the virtual environment comprises: editing the dynamic obstacles by using a software scene editor of the virtual environment, and storing in a road network file of the virtual environment; adding static obstacles in the virtual environment by using a file tag in a preset format, wherein the preset format is identical to a format of the road network file; configuring attribute information of the static obstacles, and recording corner point coordinates of the static obstacles by using a sub-tag, wherein the corner point coordinates are used to represent a shape of the static obstacles.

[0011] Optionally, before calculating the intersection point between the detection ray and the dynamic obstacles, the method further comprises: obtaining self-vehicle pose information of the vehicle model; constructing a vehicle coordinate system based on the self-vehicle pose information; and calculating relative coordinates of each obstacle in the set of obstacles in the virtual environment based on the vehicle coordinate system.

[0012] Optionally, before the intersection between the probe ray and the dynamic obstacle is calculated, the method further comprises: selecting a vehicle obstacle in the obstacle set, wherein the initial boundary shape of each obstacle in the obstacle set in the overhead angle is a quadrilateral; converting the boundary of the vehicle obstacle from a quadrilateral to an octagon; and configuring corresponding attribute information for each boundary line of the octagon, respectively.

[0013] Optionally, after the original simulation point set of the passable area is obtained, the method further comprises: collecting real vehicle data of the vehicle model; extracting real vehicle point cloud data and cluster sensor data based on millimeter wave radar of the passable area of the real vehicle data; and performing random disturbance on the original simulation point set based on the real vehicle point cloud data and the cluster sensor data to obtain a target simulation point set of the passable area.

[0014] Optionally, the random disturbance on the original simulation point set based on the real vehicle point cloud data and the cluster sensor data comprises: constructing a random distribution function with the intersection point between the cluster sensor data and the probe ray as a reference point and the distance between the real vehicle point cloud data and the reference point as an error on each probe ray of the simulation sensor; and performing random disturbance on the original simulation point set by using the random distribution function.

[0015] Optionally, after the original simulation point set of the passable area is obtained, the method further comprises: converting the original simulation point set into effective point cloud data in proto format; and inputting the effective point cloud data into an automatic driving module of a real vehicle controller and performing automatic parking test.

[0016] According to another embodiment of the present application, a point cloud simulation device for a passable area is provided, comprising: a first construction module configured to construct a virtual environment of a vehicle model and add an obstacle set in the virtual environment, wherein the obstacle set comprises a plurality of static obstacles and a plurality of dynamic obstacles, and the vehicle model is configured with simulation sensors identical to the position of a real vehicle; a transmitting module configured to transmit probe rays in all directions according to the resolution of the simulation sensors; a first calculation module configured to calculate the intersection between the probe rays and each obstacle in the obstacle set; a judgment module configured to judge whether the intersection meets a preset condition; and a first output module configured to output the intersection as a simulation point cloud of the boundary of the passable area of the vehicle model to obtain an original simulation point set of the passable area if the intersection meets the preset condition.

[0017] Optionally, the judging module comprises: a first determining unit, configured to determine a passable area detection range of the vehicle model; a judging unit, configured to judge whether the intersection point is within the passable area detection range and whether the intersection point is blocked by an obstacle in the obstacle set; and a second determining unit, configured to determine that the intersection point meets a preset condition if the intersection point is within the passable area detection range and the intersection point is not blocked by an obstacle in the obstacle set.

[0018] Optionally, the device further comprises: a first obtaining module, configured to obtain a farthest detection point of the detection ray if the intersection point does not meet the preset condition after the judging module judges whether the intersection point meets the preset condition; and a second output module, configured to output the farthest detection point as a simulation point cloud of a passable area of the vehicle model.

[0019] Optionally, the first constructing module comprises: an editing unit, configured to edit the dynamic obstacle by using a software scene editor of the virtual environment and store in a road network file of the virtual environment; an adding unit, configured to add a static obstacle in the virtual environment by using a file tag in a preset format, wherein the preset format is the same as a format of the road network file; and a configuring unit, configured to configure attribute information of the static obstacle and record corner point coordinates of the static obstacle by a sub-tag, wherein the corner point coordinates are used to represent a shape of the static obstacle.

[0020] Optionally, the device further comprises: a second obtaining module, configured to obtain self-vehicle pose information of the vehicle model before the first calculating module calculates the intersection point between the detection ray and the dynamic obstacle; a second constructing module, configured to construct a vehicle coordinate system based on the self-vehicle pose information; and a second calculating module, configured to calculate relative coordinates of each obstacle in the obstacle set in the virtual environment based on the vehicle coordinate system.

[0021] Optionally, the device further comprises: a selecting module, configured to select a vehicle obstacle in the obstacle set before the first calculating module calculates the intersection point between the detection ray and the dynamic obstacle, wherein an initial boundary shape of each obstacle in the obstacle set in a top view angle is a quadrilateral; a converting module, configured to convert the boundary of the vehicle obstacle from a quadrilateral to an octagon; and a configuring module, configured to respectively configure corresponding attribute information for each boundary line of the octagon.

[0022] Optionally, the device further comprises: a collection module, configured to collect real vehicle data of the vehicle model after the first output module obtains the original simulation point set of the passable area; an extraction module, configured to extract real vehicle point cloud data and cluster sensor data based on millimeter wave radar of the passable area of the real vehicle data; and a disturbance module, configured to disturb the original simulation point set randomly based on the real vehicle point cloud data and the cluster sensor data to obtain a target simulation point set of the passable area.

[0023] Optionally, the disturbance module comprises: a construction unit, configured to construct a random distribution function based on an intersection point of the cluster sensor data and a detection ray of the simulation sensor as a reference point and a distance between the real vehicle point cloud data and the reference point as an error on each detection ray of the simulation sensor; and a disturbance unit, configured to disturb the original simulation point set randomly by using the random distribution function.

[0024] Optionally, the device further comprises: a conversion module, configured to convert the original simulation point set into effective point cloud data in a proto format after the first output module obtains the original simulation point set of the passable area; and a test module, configured to input the effective point cloud data into an automatic driving module of a real vehicle controller and perform automatic parking test.

[0025] According to another aspect of the embodiments of the present application, a storage medium is also provided, which comprises a stored program, and the program performs the steps described above when running.

[0026] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; the memory is used to store a computer program; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0027] The embodiments of the present application also provide a computer program product comprising instructions which, when executed on a computer, cause the computer to perform the steps of the above method.

[0028] The beneficial effects of the present application are as follows:

[0029] 1. The simulation software cannot directly simulate the point cloud of the sensor output, and the simulation software can directly simulate the point cloud of the sensor, thereby realizing the data consistency of the simulation point cloud and the real vehicle sensor point cloud.

[0030] 2. For the regulation test of automatic parking, the sensor original front-end data simulation is simplified, and the FreeSpace simulation point cloud is directly simulated to the planning input, so that the test is more efficient and accurate, the effectiveness and reliability of the simulation test are significantly improved, the simulation ability and accuracy of the intelligent driving simulation system for complex scenes are significantly enhanced, the necessary test conditions of the simulation test are ensured, and the efficiency of the simulation module development is realized. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0032] Figure 1 is a hardware structure block diagram of a car according to an embodiment of the application;

[0033] Figure 2 is a flow chart of a point cloud simulation method of a passable area according to an embodiment of the application;

[0034] Figure 3 is a principle diagram of the point cloud simulation method of the passable area according to an embodiment of the application;

[0035] Figure 4 is a schematic diagram of selecting a FreeSpace point according to an embodiment of the application Figure 1 ;

[0036] Figure 5 is a schematic diagram of selecting a FreeSpace point according to an embodiment of the application Figure 2 ;

[0037] Figure 6 is a flow chart of a point cloud simulation method of a passable area according to an embodiment of the application;

[0038] Figure 7 is a structure block diagram of a point cloud simulation device of a passable area according to an embodiment of the application. DETAILED DESCRIPTION

[0039] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the protection scope of the present application. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0040] It should be noted that the terms "first", "second" and the like in the description and in the claims of the present application as well as above-mentioned appended drawings are intended to distinguish between similar objects and not necessarily in an ordinal sense. It will be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of efficient use, orderly implementation and / or other operational advantages, in other than the illustrated or described orientations. Furthermore, the terms "comprise", "include", "contain" and "have" and any variations thereof used in the specification and in the claims of the application are intended to cover both a complete and an incomplete set of steps or elements, processes, methods, products or devices, as the case can be, without necessarily excluding other steps or elements, processes, methods, products or devices, either implicitly or explicitly, that are not specifically recited.

[0041] Embodiment 1

[0042] The method embodiments provided by the embodiments of the present application can be executed in a car, a server, a processor, an automatic driving / assisted driving / smart driving controller or similar processing devices. Taking the case of running on a car, Figure 1 is a hardware structure block diagram of a car according to an embodiment of the present application. As shown in Figure 1 , the car can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above-mentioned car can also include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned car. For example, the car can also include more or fewer components than those shown in Figure 1 , or have a different configuration from Figure 1 .

[0043] The memory 104 can be used to store car programs, such as software programs of application software and modules, such as the car program corresponding to the point cloud simulation method of the passable area of a car according to an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the car program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the car through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0044] The transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the vehicle's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0045] In this embodiment, a point cloud simulation method for a traversable area is provided. Figure 2 FIG. 1 is a flow chart of a point cloud simulation method for a traversable area according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0046] Step S201: constructing a virtual environment for a vehicle model and adding an obstacle set to the virtual environment, wherein the obstacle set includes a plurality of static obstacles and a plurality of dynamic obstacles, and wherein the vehicle model is equipped with simulated sensors having the same positions as the actual vehicle;

[0047] The simulation sensor of this embodiment is a simulation sensor developed based on C++. It is aimed at regulatory control tests for scenarios such as automatic parking. Since the raw sensor data in regulatory control tests has no direct correlation with the intelligent driving regulatory control algorithm, the simulation of the raw sensor front-end data is simplified, and the planned input (the fused FreeSpace point cloud) is directly simulated. This not only ensures the necessary test conditions for the simulation test, but also achieves the efficiency of simulation module development.

[0048] The simulated sensors in this embodiment are based on the actual vehicle sensor description documents and layout relationships of the vehicle model, and combined with the actual FreeSpace point cloud output shape and range of the actual vehicle, the number of simulated sensors, mounting positions (vehicle coordinate system position x, y, z), detection range (unit: meter), and FreeSpace point cloud resolution (unit: degree) are configured as adjustable configuration parameters in the developed simulation module.

[0049] Step S202, emitting detection rays in all directions according to the resolution of the analog sensor;

[0050] Step S203, calculating the intersection point between the detection ray and each obstacle in the obstacle set;

[0051] The dynamic feedback torque of this embodiment is a torque outputted by a force simulation motor controlling the steering wheel in a direction opposite to the driver's hand force. The dynamic feedback torque is the hand force feedback acting on the driver's hand.

[0052] Step S204, determining whether the intersection meets a preset condition;

[0053] Step S205 : If the intersection meets a preset condition, the intersection is output as a simulated point cloud of the boundary of the passable area of ​​the vehicle model, and an original simulated point set of the passable area is obtained.

[0054] The FreeSpace in this embodiment is a drivable area that is dynamically adjusted based on environmental changes by collecting environmental data from various vehicle sensors (such as millimeter-wave radar and ultrasonic radar), extracting and processing features from the data, and outputting the boundaries of the drivable area. This embodiment simulates the boundary point cloud of the drivable area to determine the extent of the drivable area.

[0055] The output simulation point cloud of this embodiment inherits the attribute information of the obstacle where the intersection is located, as the FreeSpace point information of the point cloud, including position, ID, type, motion state, and source (such as from simulation software).

[0056] When multiple simulated sensors are configured on the vehicle model, the detection results of the multiple sensors (original simulated point sets) are fused and output.

[0057] Through the above steps, a virtual environment of the vehicle model is constructed, and an obstacle set is added to the virtual environment, wherein the obstacle set includes several static obstacles and several dynamic obstacles, wherein the vehicle model is equipped with a simulation sensor with the same position as the real vehicle; detection rays are emitted in all directions according to the resolution of the simulation sensor; the intersection between the detection rays and each obstacle in the obstacle set is calculated; it is determined whether the intersection meets the preset conditions; if the intersection meets the preset conditions, the intersection is output as a simulation point cloud of the boundary of the passable area of ​​the vehicle model, and the original simulation point set of the passable area is obtained, which overcomes the shortcoming that the simulation software cannot directly simulate the sensor output point cloud. The simulation software can directly simulate the sensor point cloud, achieve data consistency between the simulation point cloud and the real vehicle sensor point cloud, solve the technical problem in the related art that the simulation point cloud cannot be generated through simulation, and improve the efficiency and flexibility of point cloud data generation.

[0058] In the embodiment, adding the set of obstacles in the virtual environment comprises: editing the dynamic obstacle by using a software scene editor of the virtual environment, and storing in a road network file of the virtual environment; adding a static obstacle in the virtual environment by using a file tag in a preset format, wherein the preset format is the same as a format of the road network file; configuring attribute information of the static obstacle, and recording corner point coordinates of the static obstacle by a sub-tag, wherein the corner point coordinates are used to represent a shape of the static obstacle.

[0059] A virtual environment is built based on simulation software, an OpenDrive parking lot static road network is built in a software road network editor, and position and attribute information of static immovable static obstacles (walls, columns, road edges, etc.) are defined, and movable dynamic obstacles (dynamic traffic participants, conical barrels, etc.) are built in a software scene editor. During simulation running, the simulation software can output the above movable obstacle information (position, pose, motion state, size, type, etc.) based on Transmission Control Protocol (TCP) communication in a software self-provided data structure.

[0060] Taking VIRES Virtual Test Drive (VTD) software as an example, the embodiment uses the VTD software to build a road network and perform simulation, wherein a road network file is in an OpenDrive (.xodr) format, which belongs to an XML self-defined extended markup language format, supports user-defined addition of a tag, and adds static immovable obstacles (such as walls, columns, road edges, etc.) that cannot be covered in a software scene editor through an OpenDrive file <object>Tags are added and different obstacles are distinguished using different type, id and position attributes, while <cornerroad>The sub-label records the corner point coordinates of the static obstacle, so as to express the simplified shape of the irregular obstacle. The advantage of storing the static obstacle in the road network file is that the information can be read and stored in the simulation initialization stage, without being read in real time in the simulation process, thereby reducing the operation amount of the development module; for the movable obstacle (different dynamic traffic participants, conical barrels, etc.), the obstacle changes with the scene, and the obstacle is edited by a software scene editor and stored in an xml format scene file.

[0061] In one implementation scenario of the embodiment, determining whether the intersection point meets the preset condition includes: determining a passable area detection range of the vehicle model; determining whether the intersection point is in the passable area detection range and whether the intersection point is blocked by an obstacle in the obstacle set; if the intersection point is in the passable area detection range and the intersection point is not blocked by an obstacle in the obstacle set, it is determined that the intersection point meets the preset condition.

[0062] Optionally, whether the two obstacles are blocked can be determined according to the heights of the two obstacles and whether the two obstacles exist intersection points at the same detection ray.

[0063] The embodiment extracts the brief information of the dynamic obstacle (target size, position, posture, type, etc.) by simulation software, the position and posture information of the ego vehicle, and the static obstacle information directly read from the software road network file, and develops the sensor by combining the ray casting method; the intersection points between the detection rays of each sensor and the target are calculated, if the ray projection in the detection range of the sensor exists an intersection point, according to the height of each target, the unblocked point is selected and the target attribute and motion state and other parameters are assigned to the intersection point.

[0064] In another implementation scenario of the embodiment, after determining whether the intersection point meets the preset condition, further includes: if the intersection point does not meet the preset condition, obtaining the farthest detection point of the detection ray; outputting the farthest detection point as the simulation point cloud of the passable area of the vehicle model.

[0065] If there is no intersection point, the preset condition is not met, then the FreeSpace point in the direction is the farthest detection point of the detection ray, and finally the FreeSpace point cloud is output.

[0066] Figure 3 is the principle diagram of the passable area point cloud simulation method in the embodiment of the application, the basic use principle of the light projection method in the application is described, the main FreeSpace sensor coordinates after fusion are located at the vehicle coordinate origin, the intersection points (O1, O2…O10) of each obstacle (Obj1, Obj2, Obj3) are calculated by sending rays (l 1, l 2…ln) from the origin according to the sensor resolution, and each type of attribute contained in the above obstacle boundary line is inherited to the corresponding intersection point, the intersection point that is not blocked and closest to the origin is selected as the FreeSpace point of the obstacle detected by the sensor in the intersection point set, assuming that the height relationship of the three obstacles Obj1, Obj2, Obj3 is h Obj1 >h Obj3 >h Obj2 , the selected intersection points are O1, O5, O7, and O9 respectively; if there is no intersection point, the FreeSpace point in this direction is the farthest detection point of the real vehicle point cloud shape in this direction, and the height of the point is 0.

[0067] Figure 4 is a schematic diagram of selecting a FreeSpace point in the embodiment of the application Figure 1 , Figure 5 is a schematic diagram of selecting a FreeSpace point in the embodiment of the application Figure 2 , in Figure 4 , the obstacle Obj1 cannot block the obstacle Obj2, that is, the host vehicle can simultaneously detect the obstacles Obj1 and Obj2, so the ray in this direction will select two three-dimensional coordinate intersection points O1 and O2 as FreeSpace points, wherein the x and y coordinates of O1 and O2 are determined by the method shown in the top view shown in Figure 3 , and the height information of O1 and O2 is the height of the obstacles Obj1 and Obj2; in Figure 5 , the obstacle Obj2 is blocked by the obstacle Obj1, in this case, the host vehicle can only detect the obstacle Obj1, at this time, the ray in this direction only selects the three-dimensional coordinate intersection point O1 as the FreeSpace point, and the three-dimensional coordinate calculation method of O1 is the same as above.

[0068] In one embodiment of the embodiment, before calculating the intersection point between the detection ray and the dynamic obstacle, the method further comprises: acquiring self-vehicle pose information of the vehicle model; constructing a vehicle coordinate system based on the self-vehicle pose information; and calculating relative coordinates of each obstacle in the obstacle set in the virtual environment based on the vehicle coordinate system.

[0069] In this embodiment, the vehicle information that can be extracted by the simulation software includes the size, position, pose, type, etc. of the ego vehicle and the movable dynamic obstacles. For the information of static obstacles (walls, columns, road edges, etc.), the OpenDrive road network file is parsed based on the C++ third-party library tinyxml, and the tags defined in the format file are extracted. <object>The type of the self-defined static obstacle and the corner point position information are converted into the vehicle coordinate system through coordinate conversion based on the vehicle pose information, and are stored in a fixed container. For the information of the movable obstacle (different dynamic traffic participants, conical barrels, etc.), the obstacle data is read in real time during the simulation process through the RDB interface of the VTD software. The extracted information includes but is not limited to the size, position, attitude, type and the like of the target object. The relative coordinates of each target object based on the vehicle coordinate system of the ego vehicle (for example, taking the center of the ego vehicle as the origin, the driving direction as the positive direction of the x-axis, the left side as the positive direction of the y-axis, and the vertical direction as the z-axis) are calculated based on the vehicle pose information.

[0070] In one embodiment of the present embodiment, before calculating the intersection between the probe ray and the dynamic obstacle, further comprising: selecting a vehicle obstacle in the obstacle set, wherein the initial boundary shape of each obstacle in the obstacle set in the top view angle is a quadrilateral; converting the boundary of the vehicle obstacle from a quadrilateral to an octagon; and configuring corresponding attribute information for each boundary line of the octagon.

[0071] To increase the simulation realism, for the more complex obstacles such as vehicle obstacles, the real vehicle model is simulated, the vehicle obstacle is cut at the corner to change the vehicle obstacle from a quadrilateral to an octagon, the various attributes (id, size, type, motion state, etc.) of the obstacle are stored in each boundary line corresponding to the obstacle, and all obstacle boundary lines are stored in a container. During the running of the development module, the dynamic obstacles read in real time in the software and the static obstacles read in the initialization stage are combined into an obstacle set.

[0072] In the present embodiment, after obtaining the original simulation point set of the passable area, further comprising: collecting real vehicle data of the vehicle model; extracting real vehicle point cloud data and cluster sensor data based on millimeter wave radar of the passable area of the real vehicle data; and performing random disturbance on the original simulation point set based on the real vehicle point cloud data and the cluster sensor data to obtain a target simulation point set of the passable area.

[0073] In one example, the random disturbance on the original simulation point set based on the real vehicle point cloud data and the cluster sensor data includes: on each probe ray of the simulation sensor, taking the intersection point of the cluster sensor data and the probe ray as a reference point, and the distance between the real vehicle point cloud data and the reference point as an error, a random distribution function is constructed; and the random distribution function is used to randomly disturb the original simulation point set.

[0074] To avoid the FreeSpace point cloud output being too idealized, the point cloud boundary of the generated original simulation point set is too smooth, which is different from the point cloud noise in the real vehicle data. To meet the authenticity, a method of simulating sensor based on real vehicle FreeSpace point cloud analysis feedback is proposed. Real vehicle data is collected, and FreeSpace point cloud data and cluster sensor data (generated based on millimeter wave radar collected data) are extracted. On the same sensor detection angle (a ray), the intersection point of the real vehicle cluster and the scanning ray is taken as the reference point, the distance between the FreeSpace point and the reference point is taken as the error, the random distribution function of the FreeSpace point cloud is calculated, and the function is injected into the developed simulation sensor for random disturbance processing to obtain a more realistic target simulation point set of the passable area.

[0075] In one implementation scenario of the embodiment, after obtaining the original simulation point set of the passable area, the original simulation point set is converted into proto format effective point cloud data, and the effective point cloud data is input into the automatic driving module of the real vehicle controller and automatic parking test is performed. Based on the point cloud simulation method, the complex environment faced by the autonomous vehicle in the virtual environment is accurately simulated, and the point cloud data format required by the real vehicle controller algorithm is converted for the automatic driving system to verify the algorithm and optimize the decision logic.

[0076] Based on the point cloud output proto format in the real vehicle module communication, the above FreeSpace point cloud is matched and valued, the developed simulation module outputs point cloud in the simulation running process, and the simulation platform publishes messages in the simulation process and transmits them to the downstream modules on the vehicle for processing, such as inputting into the automatic driving module of the real vehicle controller and performing automatic parking and other vehicle tests.

[0077] The simulation verification is verified by building a real vehicle scene and collecting real vehicle data, reproducing the real vehicle scene in the simulation environment, and comparing the point cloud outputs of the simulation and the real vehicle. The simulation effectiveness is verified by building a real vehicle scene and collecting real vehicle data, reproducing the real vehicle scene in the simulation environment, and comparing the point cloud outputs. In the actual verification process, the simulation point cloud data and the real vehicle point cloud are compared, the same scene is built in the simulation according to the number and position of the obstacles around the real vehicle, the FreeSpace point cloud range and shape, the point cloud resolution, and the noise in the simulation data are basically consistent with the real vehicle, and the accuracy of the simulation point cloud is high.

[0078] The scheme of the embodiment is directed to the regulation test of automatic parking, simplifies the simulation of sensor original front-end data, and directly simulates the input of planning, that is, the fused FreeSpace point cloud, so as to ensure the necessary test conditions of simulation test and realize the efficiency of simulation module development. Figure 6 is a flowchart of the point cloud simulation method of the passable area in the embodiment of the application, including the related contents of the FreeSpace simulation module, and is divided into the corresponding processes of obstacle data storage, sensor detection and FreeSpace point cloud generation, the road network file and the real-time parsed ego vehicle and obstacle information in the simulation running process are provided by a third-party software, the obstacle information is processed through the basic data and files provided by the software, the FreeSpace sensor scanning system based on the light projection method is developed based on the ego vehicle information, the intersection points of the scanning system rays and the obstacle boundary lines are calculated, the FreeSpace point set is finally obtained after processing, and message publishing is performed.

[0079] In the generation process of the simulation point cloud, first, the FreeSpace point types of the real vehicle are labeled, and for the static target object types that cannot be output by the simulation software, such as walls, columns and road edges, the construction is performed in the extensible markup language scene file (.xml) of the simulation software. Based on the interface of the simulation software, the target object size and type information are extracted, the target object types in the software are matched with the actual required output target object types, and the position, posture and other information of the static and dynamic target objects in the simulation process are accurately calculated. According to the actual sensor parameters and the mounting position of the vehicle, the corresponding sensors are simulated at the same position of the vehicle model in the simulation module, and the real vehicle FreeSpace point cloud radiation shape and resolution are labeled according to the real vehicle FreeSpace point cloud radiation shape and resolution. By calculating the intersection points between each sensor ray and the target object, it is judged whether the FreeSpace point can detect the target object, if the intersection point is within the FreeSpace detection shape range of the ego vehicle, and there is no target object shielding, the FreeSpace point information of the target object is output, including the position, ID, type, motion state and source. If the intersection point is not within the FreeSpace detection shape range of the ego vehicle, no FreeSpace point is output or a FreeSpace point of unknown type is output according to different sensor components (such as visual FreeSpace point output and fused FreeSpace point output). At the same time, in order to realize the simulation output more combined with the actual sensor and avoid the idealization of the FreeSpace point cloud output, the FreeSpace point is subjected to noise and random disturbance processing combined with the real vehicle data.

[0080] The application realizes the high consistency of the FreeSpace point cloud data of the simulation test and the actual driving environment, and significantly improves the effectiveness and reliability of the simulation test.

[0081] Those skilled in the art can clearly understand that the method according to the above-mentioned embodiments can be realized by means of software on a general hardware platform as necessary, and of course, can also be realized by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method described in each embodiment of the present application.

[0082] Embodiment 2

[0083] In this embodiment, a point cloud simulation device for a passable area is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0084] Figure 7 is a structural block diagram of a point cloud simulation device for a passable area according to an embodiment of the present application, as shown in Figure 7 , the device comprises:

[0085] A first construction module 71 is configured to construct a virtual environment of a vehicle model and add a set of obstacles in the virtual environment, wherein the set of obstacles comprises a plurality of static obstacles and a plurality of dynamic obstacles, and wherein the vehicle model is configured with simulation sensors identical to the positions of real vehicle sensors.

[0086] A transmitting module 72 is configured to transmit detection rays in all directions according to the resolution of the simulation sensors.

[0087] A first calculation module 73 is configured to calculate the intersection points between the detection rays and each obstacle in the set of obstacles.

[0088] A judgment module 74 is configured to judge whether the intersection points meet predetermined conditions.

[0089] A first output module 75 is configured to output the intersection points as a simulation point cloud of the passable area boundary of the vehicle model if the intersection points meet the predetermined conditions, to obtain an original simulation point set of the passable area.

[0090] Optionally, the judging module comprises: a first determining unit, configured to determine a passable area detection range of the vehicle model; a judging unit, configured to judge whether the intersection point is within the passable area detection range and whether the intersection point is blocked by an obstacle in the obstacle set; and a second determining unit, configured to determine that the intersection point meets a preset condition if the intersection point is within the passable area detection range and the intersection point is not blocked by an obstacle in the obstacle set.

[0091] Optionally, the device further comprises: a first obtaining module, configured to obtain a farthest detection point of the detection ray if the intersection point does not meet the preset condition after the judging module judges whether the intersection point meets the preset condition; and a second output module, configured to output the farthest detection point as a simulation point cloud of a passable area of the vehicle model.

[0092] Optionally, the first constructing module comprises: an editing unit, configured to edit the dynamic obstacle by using a software scene editor of the virtual environment and store in a road network file of the virtual environment; an adding unit, configured to add a static obstacle in the virtual environment by using a file tag in a preset format, wherein the preset format is the same as a format of the road network file; and a configuring unit, configured to configure attribute information of the static obstacle and record corner point coordinates of the static obstacle by using a sub-tag, wherein the corner point coordinates are used to represent a shape of the static obstacle.

[0093] Optionally, the device further comprises: a second obtaining module, configured to obtain self-vehicle pose information of the vehicle model before the first calculating module calculates the intersection point between the detection ray and the dynamic obstacle; a second constructing module, configured to construct a vehicle coordinate system based on the self-vehicle pose information; and a second calculating module, configured to calculate relative coordinates of each obstacle in the obstacle set in the virtual environment based on the vehicle coordinate system.

[0094] Optionally, the device further comprises: a selecting module, configured to select a vehicle obstacle in the obstacle set before the first calculating module calculates the intersection point between the detection ray and the dynamic obstacle, wherein an initial boundary shape of each obstacle in the obstacle set in a top view angle is a quadrilateral; a converting module, configured to convert the boundary of the vehicle obstacle from a quadrilateral to an octagon; and a configuring module, configured to respectively configure corresponding attribute information for each boundary line of the octagon.

[0095] Optionally, the device further comprises: a collection module, configured to collect real vehicle data of the vehicle model after the first output module obtains the original simulation point set of the passable area; an extraction module, configured to extract real vehicle point cloud data and cluster sensor data based on millimeter wave radar of the passable area of the real vehicle data; and a disturbance module, configured to disturb the original simulation point set randomly based on the real vehicle point cloud data and the cluster sensor data to obtain a target simulation point set of the passable area.

[0096] Optionally, the disturbance module comprises: a construction unit, configured to construct a random distribution function based on an intersection point of the cluster sensor data and a detection ray as a reference point and a distance between the real vehicle point cloud data and the reference point as an error on each detection ray of the simulation sensor; and a disturbance unit, configured to disturb the original simulation point set randomly by using the random distribution function.

[0097] Optionally, the device further comprises: a conversion module, configured to convert the original simulation point set into effective point cloud data in a proto format after the first output module obtains the original simulation point set of the passable area; and a test module, configured to input the effective point cloud data into an automatic driving module of a real vehicle controller and perform automatic parking test.

[0098] It should be noted that each of the above modules can be implemented by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: all the above modules are located in the same processor; or the above modules are located in different processors in any combination.

[0099] Embodiment 3

[0100] Embodiments of the present application also provide a storage medium having a computer program stored therein, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0101] Optionally, in the present embodiment, the above storage medium can be configured to store a computer program for executing the following steps:

[0102] S1, a virtual environment of a vehicle model is constructed, and a set of obstacles is added in the virtual environment, wherein the set of obstacles comprises a plurality of static obstacles and a plurality of dynamic obstacles, and a simulation sensor identical to a real vehicle position is configured on the vehicle model;

[0103] S2, detection rays are emitted in all directions according to a resolution of the simulation sensor;

[0104] S3, an intersection point between each detection ray and each obstacle in the set of obstacles is calculated;

[0105] S4, judging whether the intersection point meets preset conditions;

[0106] S5, if the intersection point meets the preset conditions, outputting the intersection point as a simulation point cloud of a passable area boundary of the vehicle model to obtain an original simulation point set of the passable area.

[0107] Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various computer program storage media.

[0108] The embodiment of the application further provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to perform the steps in any of the method embodiments.

[0109] Optionally, the electronic device can further include a transmission device and an input-output device, wherein the transmission device is connected with the processor, and the input-output device is connected with the processor.

[0110] Optionally, in the embodiment, the processor can be configured to execute the following steps through the computer program:

[0111] S1, constructing a virtual environment of a vehicle model and adding a set of obstacles in the virtual environment, wherein the set of obstacles includes a plurality of static obstacles and a plurality of dynamic obstacles, and a simulation sensor identical to a real vehicle position is configured on the vehicle model;

[0112] S2, emitting a detection ray in all directions according to a resolution of the simulation sensor;

[0113] S3, calculating an intersection point between the detection ray and each obstacle in the set of obstacles;

[0114] S4, judging whether the intersection point meets preset conditions;

[0115] S5, if the intersection point meets the preset conditions, outputting the intersection point as a simulation point cloud of a passable area boundary of the vehicle model to obtain an original simulation point set of the passable area.

[0116] Optionally, specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, and the embodiment will not be described here again.

[0117] The apparatus embodiments described above are only illustrative, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the embodiments or some parts of the embodiments.

[0119] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order in which they are described, unless specifically identified as an order dependent step. It is also to be understood that additional or alternative steps can be employed.

[0120] The above description is merely illustrative of the application and should not be taken as limiting. Numerous modifications and variations underlying the general principles of the applications can be made by those of ordinary skill in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.< / object> < / cornerroad> < / object>

Claims

1. A point cloud simulation method for a traversable area, characterized in that: include: Constructing a virtual environment for the vehicle model and adding an obstacle set to the virtual environment, wherein the obstacle set includes a plurality of static obstacles and a plurality of dynamic obstacles, wherein the vehicle model is configured with simulated sensors in the same positions as the actual vehicle; emitting detection rays in all directions according to the resolution of the analog sensor; Calculating an intersection point between the detection ray and each obstacle in the obstacle set; Determining whether the intersection meets a preset condition; If the intersection meets the preset conditions, the intersection is output as a simulated point cloud of the boundary of the passable area of ​​the vehicle model to obtain an original simulated point set of the passable area.

2. The method according to claim 1, characterized in that Determining whether the intersection meets the preset conditions includes: Determining a traversable area detection range of the vehicle model; Determining whether the intersection is within the detection range of the passable area, and determining whether the intersection is blocked by an obstacle in the obstacle set; If the intersection is within the detection range of the passable area and the intersection is not blocked by an obstacle in the obstacle set, it is determined that the intersection meets the preset condition.

3. The method according to claim 1, characterized in that After determining whether the intersection meets the preset conditions, the method further includes: If the intersection point does not meet the preset conditions, obtaining the farthest detection point of the detection ray; The farthest detection point is output as a simulated point cloud of the traversable area of ​​the vehicle model.

4. The method according to claim 1, wherein Adding an obstacle set to the virtual environment includes: Editing the dynamic obstacles using a software scene editor of the virtual environment and storing the dynamic obstacles in a road network file of the virtual environment; Adding static obstacles in the virtual environment using a file tag in a preset format, wherein the preset format is the same as the format of the road network file; The attribute information of the static obstacle is configured, and the corner coordinates of the static obstacle are recorded through sub-tags, wherein the corner coordinates are used to represent the shape of the static obstacle.

5. The method according to claim 1, wherein Before calculating the intersection point between the detection ray and the dynamic obstacle, the method further includes: Obtaining vehicle pose information of the vehicle model; Constructing a vehicle coordinate system based on the vehicle position information; The relative coordinates of each obstacle in the obstacle set in the virtual environment are calculated based on the vehicle coordinate system.

6. The method according to claim 1, characterized in that Before calculating the intersection point between the detection ray and the dynamic obstacle, the method further includes: Selecting a vehicle obstacle from the obstacle set, wherein an initial boundary shape of each obstacle in the obstacle set in a top-down perspective is a quadrilateral; Converting the boundary of the vehicle obstacle from a quadrilateral to an octagon; For each boundary line of the octagon, corresponding attribute information is configured respectively.

7. The method according to claim 1, characterized in that After obtaining the original simulation point set of the traversable area, the method further includes: Collecting real vehicle data of the vehicle model; Extracting real vehicle point cloud data of a traversable area of ​​the real vehicle data and cluster sensor data based on millimeter wave radar; The original simulation point set is randomly perturbed based on the real vehicle point cloud data and the cluster sensor data to obtain a target simulation point set in the traversable area.

8. The method according to claim 7, characterized in that The random perturbation of the original simulation point set based on the real vehicle point cloud data and the cluster sensor data includes: On each detection ray of the simulated sensor, a random distribution function is constructed with the intersection of the cluster sensor data and the detection ray as a reference point and the distance between the real vehicle point cloud data and the reference point as an error; The random distribution function is used to randomly perturb the original simulation point set.

9. The method according to claim 1, characterized in that After obtaining the original simulation point set of the traversable area, the method further includes: Convert the original simulation point set into valid point cloud data in proto format; The valid point cloud data is input into the automatic driving module of the real vehicle controller, and an automatic parking test is performed.

10. A point cloud simulation device for a traversable area, characterized in that: include: a first construction module, configured to construct a virtual environment for the vehicle model and add an obstacle set to the virtual environment, wherein the obstacle set includes a plurality of static obstacles and a plurality of dynamic obstacles, and wherein the vehicle model is configured with simulated sensors having the same positions as those of the actual vehicle; A transmitting module, configured to transmit detection rays in all directions according to the resolution of the analog sensor; A first calculation module, configured to calculate an intersection point between the detection ray and each obstacle in the obstacle set; A judging module, configured to judge whether the intersection point meets a preset condition; The first output module is used to output the intersection as a simulated point cloud of the boundary of the passable area of ​​the vehicle model if the intersection meets the preset conditions, so as to obtain the original simulated point set of the passable area.

11. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 9 when executed.

12. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 9.

Images