Road environment modeling method and device, storage medium, terminal and mobile device

Abstract

The present invention discloses a road environment modeling method and device, storage medium, terminal and mobile device, wherein the modeling method comprises the following steps: receiving upstream input data and internal historical data; performing data fusion and data reconstruction based on the received upstream input data and internal historical data, and respectively constructing a static data model, a dynamic data model and a predictive data model. The beneficial effects of the present invention are as follows: 1. The present invention can effectively organize the environmental data around the vehicle, which can not only provide stable environmental data to the downstream use module, but also effectively avoid the problem of inconsistent data obtained by the downstream module; 2. The present invention can cover various current road scenes, can express environmental data more completely, and effectively avoid data loss; 3. The modeling process of the present invention is clear, and the data hierarchy is clear, which can be selected by the downstream module on demand, thereby improving the development and use efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving environment modeling, and in particular to a road environment modeling method and device, a storage medium, a terminal and a mobile device. Background Art

[0002] With the strong support of the government and related enterprises for the artificial intelligence industry in recent years, autonomous driving technology has developed rapidly. Various regions have successively opened dedicated test sections for autonomous driving. More and more autonomous driving companies have started large-scale road tests with the goal of promoting the iteration of technology and products. As for the current development status of unmanned driving, most autonomous driving vehicles are not only highly dependent on the road topology information of high-precision maps, but also need to use various sensors to obtain real-time obstacle information around the vehicle. This surrounding environment information is the basis of all upper-level technologies. Therefore, it is very important to model the road environment around the vehicle, or to digitize the surrounding environment.

[0003] As we all know, computer software systems do not have the ability to directly process world information like humans. They need clear digital input to perform logical calculations and judgments. Therefore, it is necessary to highly abstract the real world environment and convert it into corresponding environmental data. This process is environmental modeling. Just like the importance of foundation to a house, the quality of the environmental model will directly affect the difficulty and efficiency of implementing the upper-level technology. Therefore, it is imperative to establish an efficient and complete environmental model.

[0004] Currently, there are two main environmental modeling methods: one is based on map topology, and the other is based on a three-lane model.

[0005] The method based on map topology is derived from the topology of HD map, and its essence is the further processing and encapsulation of HD map. This method usually relies on positioning to obtain road topology information (including road level and lane level information) within a certain range around the vehicle from HD map, then associates the perceived obstacle information with the lane level information, and finally performs obstacle analysis based on the associated information. For users of environmental results (such as decision modules and planning modules), if they want to use the environment model based on map topology, they first need to associate the position of the obstacle of interest or the vehicle with the road topology information, and then use the topology of the map itself to obtain the required information and association relationship. In this modeling structure, the HD map topology can be considered as the medium connecting all substances, and the vehicle and the surrounding obstacles must determine their relationship with each other through this medium; the method based on map topology mainly relies on HD map, but the map itself is an objective description of the physical world, while the user needs the subjective position relationship of the left, right or front and back. There is a certain difference between "supply and demand", and users often need to perform secondary abstraction. Secondly, the model access process is cumbersome and inefficient. When you need to find the relationship between any two obstacles, you must first project them onto the high-precision map, and then determine the relationship between the two obstacles based on the relationship between the roads. This process requires multiple projections and transformations to achieve. When the map is large, the efficiency of relationship search through map topology becomes extremely low, and even limits the efficiency of the algorithm itself. Finally, when projecting obstacles onto the map, there may be misprojections (matching the wrong road) or no projection results can be found, which will result in real obstacles that cannot be abstracted into valid data, causing the user to obtain errors or missing environmental data. It is difficult to make a comprehensive judgment based on such modeling results, which may eventually lead to safety accidents.

[0006] The modeling method based on the three-lane model is another commonly used modeling method. This method is different from directly relying on the topological relationship between high-definition maps. Instead, it abstracts the surrounding environment into a left, middle, and right three-lane model that can only be seen visually. This modeling method first obtains the current vehicle position based on the high-precision map and positioning, and then obtains the center line of a certain length of lane, as well as the center line of the parallel lane on the left side of the current lane of equal length and the center line of the parallel lane on the right side of equal length; then the current obstacle information provided by perception is associated with these three center lines respectively. In this model, the vehicle and the surrounding obstacles are only related to the left, middle, and right lanes, which can achieve rapid relationship positioning. The method based on the three-lane model is currently a relatively mainstream method, which can basically meet the functional requirements of simple autonomous driving. However, since the model only covers the data within the three lanes around the vehicle, it is equivalent to artificially losing part of the valid data, and the result is too subjective; in addition, the three-lane model cannot express some complex traffic scenes, such as road merging, divergence, intersections, etc., so the model cannot cope with complex traffic environments. Finally, the three-lane model cannot effectively describe the relationship between obstacles in special scenarios, such as obstacles in the opposite lane and obstacles in the merging lane. This is an inherent defect of the model and the main reason why it cannot be widely applied. Summary of the invention

[0007] In view of the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a road environment modeling method and device, storage medium, terminal and mobile device to better meet the user's usage needs.

[0008] To achieve the purpose of the present invention, the present invention provides a road environment modeling method and device, storage medium, terminal and mobile device. The technical solutions are as follows:

[0009] First aspect

[0010] An embodiment of the present invention provides a road environment modeling method, comprising:

[0011] Receive upstream input data and internal historical data;

[0012] Data fusion and data reconstruction are performed based on the received upstream input data and internal historical data to respectively construct a static data model, a dynamic data model and a predictive data model.

[0013] Furthermore, constructing the static data model includes:

[0014] Step S11: determining the surrounding lane relationship according to the vehicle position;

[0015] Step S12: forming a lane reference line according to the lane relationship determined in step S11;

[0016] Step S13: Associating the road facilities and sign information with the lane reference line formed in step S12, thereby obtaining static data;

[0017] Step S14: Perform static semantic analysis on the static data obtained in step S13, and superimpose the analysis result on the static data.

[0018] Furthermore, when lane splitting is performed, the process of constructing the static data model further includes:

[0019] When lane splitting, if the vehicle is before the split point, the left lane of the vehicle includes the route of the vehicle before the split point and the split route;

[0020] If the vehicle is behind the diverging point, when the vehicle is on the main road, the diverging road is the left lane of the vehicle, and obstacles on the diverging road are included in the range of obstacles to be paid attention to; when the vehicle is on the diverging road, the main road is the right lane of the vehicle, and obstacles on the main road are included in the range of obstacles to be paid attention to.

[0021] Furthermore, constructing the dynamic data model includes the following steps:

[0022] Step S21: Projecting the obstacle onto a high-precision map to determine its location;

[0023] Step S22: determining whether the obstacle will conflict with the current lane reference line, and selecting the obstacle to focus on;

[0024] Step S23: Associating the obstacle of interest selected in step S22 with the lane reference line, thereby obtaining dynamic data;

[0025] Step S24: Perform dynamic semantic analysis on the dynamic data obtained in step S23, and superimpose the analysis results on the dynamic data.

[0026] Furthermore, when performing lane merging, the process of constructing the dynamic data model further includes:

[0027] If the vehicle is on the main road, the obstacles on the merging lane are associated with the right or left lane by giving a free entry point;

[0028] If the vehicle is in the merging lane, the obstacles on the main road are divided into two cases for association: one is that there is an intersection with the vehicle's current lane in the road structure, and the obstacle is associated with the vehicle's current lane as a free entry point; the other is that there is no intersection with the vehicle's current lane in the road structure, but there is an intersection with the vehicle's future road, and the obstacle is included in the obstacle of interest in the vehicle's current lane as a forced entry point.

[0029] Furthermore, when traveling at an intersection, the process of constructing the dynamic data model further includes:

[0030] If the vehicle is going straight at the intersection, obstacles turning left are included in the range of obstacles to be concerned in the form of forced entry points; obstacles turning right and merging into the vehicle's current lane are projected into the vehicle's current lane in the form of free entry points;

[0031] If the vehicle turns left at an intersection, obstacles in the opposite lane turning left or going straight are included in the range of obstacles to be concerned in the form of mandatory entry points; obstacles turning right and merging into the vehicle's current lane are projected into the vehicle's current lane in the form of free entry points;

[0032] If the vehicle turns right at an intersection, the left-turning obstacle in the opposite lane is included in the range of obstacles in the form of a forced entry point; the obstacle going straight into the vehicle's current lane is projected into the vehicle's current lane as a free entry point.

[0033] Furthermore, constructing the prediction data model includes:

[0034] Step S31: Associating the predicted obstacle trajectory with the corresponding dynamic obstacle and superimposing it on the static data;

[0035] Step S32: Associating the predicted obstacle semantic information with the corresponding dynamic obstacle and superimposing it on the static data.

[0036] Furthermore, the upstream input data includes data information provided by high-precision maps, positioning devices, perception devices, business layer modules, vehicle bottom layer modules, and prediction modules;

[0037] The internal historical data includes historical frame data of the environment modeling module, historical decision behaviors of the decision module, and local paths released by the planning module.

[0038] Second aspect

[0039] An embodiment of the present invention provides a road environment modeling device, the device comprising a data receiving unit and a model building unit,

[0040] The data receiving unit is used to receive upstream input data and internal historical data;

[0041] The model building unit is used to perform data fusion and data reconstruction based on the received upstream input data and internal historical data, and to build a static data model, a dynamic data model and a predictive data model respectively.

[0042] The third aspect

[0043] An embodiment of the present invention provides a storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the road environment modeling method.

[0044] The fourth aspect

[0045] An embodiment of the present invention provides a terminal device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the road environment modeling method.

[0046] Fifth Aspect

[0047] An embodiment of the present invention provides a mobile device, including the terminal device.

[0048] Compared with the prior art, the present invention has the following technical advantages:

[0049] 1. The technical solution provided by the present invention fully combines the ideas based on the map topology structure and the three-lane model to establish a more complete multi-lane model, which effectively reduces the complexity of use while ensuring the integrity of the model data; it can not only provide stable environmental data to the downstream use module, but also effectively avoid the problem of inconsistent data obtained by the downstream module;

[0050] 2. The construction process of the technical solution method provided by the present invention is divided into three levels and steps: static data and its semantic construction, dynamic data and its semantic construction, and predictive data and its semantic construction. Users can extract data and semantics at different levels according to their different needs. The modeling process is clear and the data hierarchy is clear, which can be selected by downstream modules on demand, thereby improving development and use efficiency;

[0051] 3. The multi-lane model provided by the present invention can describe almost all current structured road scenes, can express environmental data more completely, and effectively avoid data loss. The model is more universal and practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flow chart of a road environment modeling method in an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the interactive relationship of road environment modeling data in an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of a multi-lane road structure in an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a process for constructing a data model in an embodiment of the present invention;

[0056] Figure 5 Schematic diagram of a situation in which the vehicle is on a main road in a vehicle merging scenario in an embodiment of the present invention;

[0057] Figure 6 Schematic diagram of a situation in which the vehicle is on a merging lane in a vehicle merging scenario in an embodiment of the present invention;

[0058] Figure 7 It is a schematic diagram of a situation in which the vehicle is before the diversion point in a vehicle diversion scenario in an embodiment of the present invention;

[0059] Figure 8 It is a first schematic diagram of a situation where the vehicle is behind a diversion point in a vehicle diversion scenario in an embodiment of the present invention;

[0060] Fig. 9 A second schematic diagram of a situation in which the vehicle is behind a diversion point in a vehicle diversion scenario in an embodiment of the present invention;

[0061] Fig.10 This is a schematic diagram of a situation in which a vehicle is traveling straight at an intersection in a crossroads scene according to an embodiment of the present invention;

[0062] Fig.11 This is a schematic diagram of a situation in which a vehicle turns left at an intersection in a crossroads scene according to an embodiment of the present invention;

[0063] Fig.12 This is a schematic diagram of a vehicle turning right at an intersection in a crossroads scene in an embodiment of the present invention;

[0064] Fig.13 4 is a structural block diagram of a road environment modeling device in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] like Figure 1 The embodiment of the present invention provides a road environment modeling method, comprising:

[0067] Receive upstream input data and internal historical data;

[0068] Data fusion and data reconstruction are performed based on the received upstream input data and internal historical data to respectively construct a static data model, a dynamic data model and a predictive data model.

[0069] It should be noted that the environmental modeling module is at the upstream of the entire planning node (planning), and its main function is to integrate and reconstruct the upstream input data of the planning and the internal historical data of the planning. Figure 2 As shown in the figure, the upstream input data mainly includes road topology, reference paths and boundaries provided by high-precision maps; vehicle position and posture information provided by positioning; obstacles, traffic lights, road signs and other information within a certain range around the vehicle provided by perception; tasks and modes issued by the business layer; real-time vehicle status information fed back by the vehicle bottom layer, and obstacle prediction lines and intentions provided by the prediction module. The internal historical data mainly includes historical frame data of the environment modeling module, historical decision-making behavior of the decision module, and local paths released by the planning module. The environment modeling module further abstracts and reconstructs the above data to form a digital environment model, and provides it to downstream modules (such as decision-making and planning) for use, thereby ensuring the consistency and efficiency of downstream module data.

[0070] It should be noted that, in the embodiment of the present invention, the fused data is divided into three levels: static data and its semantics, dynamic data and its semantics, and predicted data and its semantics.

[0071] Among them, static data and its semantics: Static data mainly refers to the road topology, lane reference lines and boundaries provided by high-definition maps, and other data that exist in a static form in the physical world. Static semantics refers to a more abstract expression of these static data after certain processing, such as the lane ahead becoming narrower or wider, etc.;

[0072] Dynamic data and its semantics: Dynamic data mainly refers to the data that changes dynamically, such as obstacles, traffic lights, road signs, etc. within a certain range around the vehicle. Dynamic semantic information is a higher-level abstract expression obtained by processing dynamic data, such as the traffic flow of the lane, whether the lane is congested, and the safety factor of the lane;

[0073] Prediction data and its semantics: Prediction data mainly refers to the trajectory of obstacles in the future provided by the prediction module. Prediction semantic information refers to the intention information given by the prediction module based on the historical trajectory of obstacles and the prediction line, such as vehicles changing lanes to the left, pedestrians going straight or crossing.

[0074] like Figure 4 As shown, the steps of constructing a static data model, a dynamic data model, and a predictive data model in an embodiment of the present invention. The modeling process is explained below using a common multi-lane road structure as an example.

[0075] Multi-lane road structures, such as Figure 3 As shown, assuming that the vehicle (green polygon) is in the middle (current) lane of the five-lane road, it is easy to determine its left and right lanes, as well as the left lane of its left lane (left-left) and the right lane of its right lane (right-right) based on the topological relationship of the map.

[0076] Wherein, constructing the static data model includes:

[0077] Step S11: determining the surrounding lane relationship according to the vehicle position;

[0078] It should be noted that the relationship between lanes around the vehicle is determined based on the position of the vehicle, including the lane where the vehicle is located, the left and right lanes, and the relationship between the preceding and succeeding lanes.

[0079] Step S12: forming a lane reference line according to the lane relationship determined in step S11.

[0080] It should be noted that, secondly, based on the relationship between the current, left lane, and right lane's successor lanes, a forward path and a backward path of a certain length are combined to form a three-lane reference line.

[0081] Step S13: Associating the road facilities and sign information with the lane reference line formed in step S12, thereby obtaining static data;

[0082] It should be noted that the road facilities and sign information (speed bumps, intersection locations, traffic light locations, etc.) within a certain length of the current lane are associated with the current lane reference line, and the left and right lanes are processed in the same way, and finally a three-lane static data based on the vehicle position is obtained.

[0083] Step S14: Perform static semantic analysis on the static data obtained in step S13, and superimpose the analysis results on the static data, thereby completing the construction of the entire static data model.

[0084] In a preferred embodiment, when lane divergence is performed, the process of constructing the static data model also includes: when lane divergence is performed, if the vehicle is before the divergence point, the left lane of the vehicle includes the route of the vehicle before the divergence point and the divergence route; if the vehicle is after the divergence point, when the vehicle is on the main road, the divergence road is the left lane of the vehicle, and obstacles on the divergence road are included in the range of obstacles of concern; when the vehicle is on the divergence road, the main road is the right lane of the vehicle, and obstacles on the main road are included in the range of obstacles of concern.

[0085] It should be noted that: Figure 7 , 8 As shown in Figure 9, the vehicle diversion scenario is divided into two situations: the vehicle is before the diversion point and after the diversion point.

[0086] For the vehicle before the divergence point ( Figure 7 ), it should be noted that in the process of constructing static data, the left lane of the vehicle is a combination of the route of the vehicle before the divergence point and the divergence route, and the construction method of other parts remains unchanged.

[0087] For the vehicle after the divergence point ( Figure 8 , Fig. 9 ), which is divided into two scenarios: the vehicle is on the main road and the diversion road. Figure 8 ), the diversion road needs to be considered as the left lane of the vehicle, so that obstacles on the diversion road can be included in the range of obstacles of concern. Fig. 9 ), according to the previous idea, the diverging lane is the current lane of the vehicle, so the main road needs to be regarded as the right lane of the vehicle, and the obstacles on the main road should be included in the range of obstacles to be concerned, so as to deal with obstacles on the main road that may change lanes to the diverging road.

[0088] Furthermore, constructing the dynamic data model includes the following steps:

[0089] Step S21: Project each obstacle onto the high-precision map to determine its location;

[0090] Step S22: Determine whether the obstacle will conflict with the current lane reference line, and select the obstacle to be concerned; specifically, combining the prediction information of the current lane and the obstacle, it can be concluded whether the obstacle will conflict with the current lane reference line. The judgment conditions are as follows: obstacles in the vehicle lane and the left and right lanes must be set as obstacles to be concerned (the vehicle may change lanes, and the left and right vehicles may also cut into the vehicle lane at any time), and obstacles in the left-left or right-right lane of the vehicle only need to be concerned if they are cutting into the left or right lane (the obstacles maintained in the left-left and right-right lanes do not directly conflict with the vehicle, and the cutting vehicle may have conflicts in the continuous loop twice), in special scenarios, obstacles that may borrow the road in the opposite direction, obstacles such as conflicts between turning left and right and going straight in the intersection area, and obstacles merging into the vehicle lane need to be marked as obstacles to be concerned.

[0091] Step S23: Associate the obstacle of interest selected in step S22 with the lane reference line to obtain dynamic data; specifically, associate the above obstacle of interest with the current reference line in sequence. Next, perform two simulations, assuming that the left lane reference line and the right lane reference line are the current reference lines of the vehicle, select the obstacle of interest according to the above judgment conditions, and associate them with the left and right reference lines respectively.

[0092] Step S24: Finally, the dynamic data of the three lanes are comprehensively analyzed to output dynamic semantic information, and the information is superimposed, thus completing the dynamic data modeling process.

[0093] Among them, Figure 5 , Figure 6 As shown, when lane merging is performed, the process of constructing the dynamic data model further includes:

[0094] When the vehicle is on the main road, the current lane and the left and right lanes are first determined based on the map, and static data is established according to the method described in the previous section. When processing dynamic data, other obstacles on the merging lane need to be specially processed. In this solution, they are described as free entry concepts and are associated with the right lane by giving free entry points. The association methods of other obstacles remain unchanged, and the prediction information processing method is also as described above.

[0095] For the situation where the ego vehicle is in the lane to be merged, the process of establishing static data and prediction data remains unchanged. The difference is that in this case, the ego vehicle only has one current lane, and other obstacles on the main road are associated with two situations: one is that there is an intersection with the merging lane in the road structure, and this type of obstacle is associated with the current lane of the ego vehicle in the form of a free entry point; the other type is that there is no intersection with the current lane of the ego vehicle in the road structure, but because the obstacle's lane change trajectory intersects with the future road of the ego vehicle, this type of obstacle is described as a forced entry concept and is included in the obstacle of concern in the current lane of the ego vehicle by giving a forced entry point.

[0096] like Fig.10 , 11 As shown in , 12, when traveling at an intersection, the process of constructing the dynamic data model also includes:

[0097] For the scenario where the vehicle is driving straight at an intersection, obstacles in the opposite lane turning left need to be considered during the dynamic data construction process. In terms of road topology, the two roads have overlapping traffic areas, which is a unique situation at intersections. In order to express this situation, the obstacles in the opposite lane turning left are still converted into mandatory entry points and included in the scope of attention. Similarly, obstacles turning right and merging into the vehicle's current lane are projected into the current lane as free entry points. The construction method of other layers of data remains unchanged.

[0098] For the scenario where the vehicle turns left at the intersection, the obstacles in the opposite lane turning left and going straight need to be considered during the dynamic data construction process. In terms of road topology, the obstacles overlap with the lane of the vehicle. The obstacles in this case are converted into mandatory entry points and included in the scope of attention. For the obstacles that turn right and merge into the current lane of the vehicle, they are projected into the current lane in the form of free entry points. The construction method of other layers of data remains unchanged.

[0099] For the scenario where the vehicle turns right at the intersection, the left-turning obstacle in the opposite lane needs to be considered during the dynamic data construction process. In terms of road topology, the two roads have overlapping traffic areas, which are also converted into mandatory entry points and included in the scope of attention. For obstacles that go straight to the vehicle's current lane, they are projected to the current lane in the form of free entry points. The construction method of other layers of data remains unchanged.

[0100] Furthermore, constructing the prediction data model includes:

[0101] Step S31: Associating the predicted obstacle trajectory with the corresponding dynamic obstacle and superimposing it on the static data;

[0102] Step S32: Associating the predicted obstacle semantic information with the corresponding dynamic obstacle and superimposing it on the static data.

[0103] It should be noted that the predicted obstacle trajectory and intention are associated with the corresponding dynamic obstacle and superimposed on the static data, thus completing the modeling process of the predicted data.

[0104] Through the above three steps, the construction of the multi-lane model is completed, and the model data of different layers are obtained. After a certain amount of data integration and interface conversion, this part of the data is provided to the downstream modules for use. At the same time, according to the different focuses of different modules, different levels of data can be obtained as needed, which helps the downstream modules to use quickly and clearly.

[0105] like Fig.13 As shown, corresponding to the above method, an embodiment of the present invention provides a road environment modeling device, the device includes a data receiving unit and a model building unit,

[0106] The data receiving unit is used to receive upstream input data and internal historical data;

[0107] The model building unit is used to perform data fusion and data reconstruction based on the received upstream input data and internal historical data, and to build a static data model, a dynamic data model and a predictive data model respectively.

[0108] For the functions and implementation methods of the data receiving unit and the model building unit, please refer to the above description of the method and will not be described in detail here.

[0109] In addition, an embodiment of the present invention provides a storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the road environment modeling method.

[0110] In addition, an embodiment of the present invention provides a terminal device, the terminal device includes a processor and a memory, the memory stores at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the road environment modeling method. The terminal device includes but is not limited to a computer device.

[0111] The following is a further introduction to storage media and computer devices mainly in combination with application scenarios.

[0112] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, devices (equipment or system), or computer equipment, storage media. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0113] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (devices or systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0116] In a typical configuration, a computer device includes one or more processors (CPU), storage, input / output interfaces, network interfaces, and memory. The storage may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The storage is an example of a computer-readable medium.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof in the present invention are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0119] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0120] In addition, an embodiment of the present invention provides a mobile device, including the terminal device.

[0121] It should be noted that the mobile devices referred to in this application may include but are not limited to vehicles with six autonomous driving technology levels, L0-L5, as established by the Society of Automotive Engineers International (SAE International) or the Chinese national standard "Automotive Driving Automation Classification".

[0122] In some possible implementations, the mobile device may be a vehicle device or a robot device having the following various functions:

[0123] (1) Passenger-carrying function, such as family cars and buses;

[0124] (2) Cargo-carrying functions, such as ordinary trucks, van trucks, trailer trucks, closed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, trucks with special structures, etc.;

[0125] (3) Tool functions, such as logistics delivery vehicles, automated guided vehicles (AGVs), patrol cars, cranes, hoists, excavators, bulldozers, forklifts, road rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor scrubbers, water sprinklers, sweeping robots, food delivery robots, shopping guide robots, lawn mowers, golf carts, etc.;

[0126] (4) Entertainment functions, such as entertainment vehicles, amusement park self-driving devices, balance vehicles, etc.;

[0127] (5) Special rescue functions, such as fire trucks, ambulances, power repair vehicles, engineering rescue vehicles, etc.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate and explain the present invention, and are not intended to limit the present invention to the scope of the described embodiments. In addition, those skilled in the art can understand that the present invention is not limited to the above embodiments, and more variations and modifications can be made according to the teachings of the present invention, and these variations and modifications all fall within the scope of the protection claimed by the present invention.

Claims

1. A road environment modeling method, characterized in that: include: Receive upstream input data and internal historical data; Based on the received upstream input data and internal historical data, data fusion and data reconstruction are performed to respectively construct a static data model, a dynamic data model and a predictive data model; Constructing the dynamic data model includes the following steps: Step S21: Projecting the obstacle onto a high-precision map to determine its location; Step S22: determining whether the obstacle will conflict with the current lane reference line, and selecting the obstacle to focus on; Step S23: Associating the obstacle of interest selected in step S22 with the lane reference line, thereby obtaining dynamic data; Step S24: Perform dynamic semantic analysis on the dynamic data obtained in step S23, and superimpose the analysis result on the dynamic data; When performing lane merging, the process of constructing the dynamic data model further includes: If the vehicle is on the main road, the obstacles on the merging lane are associated with the right or left lane by giving a free entry point; If the vehicle is in the merging lane, the obstacles on the main road are divided into two cases for association: one is that there is an intersection with the vehicle's current lane in the road structure, and the obstacle is associated with the vehicle's current lane as a free entry point; the other is that there is no intersection with the vehicle's current lane in the road structure, but there is an intersection with the vehicle's future road, and the obstacle is included in the obstacle of interest in the vehicle's current lane as a forced entry point.

2. The road environment modeling method according to claim 1, characterized in that: Building the static data model includes: Step S11: determining the surrounding lane relationship according to the vehicle position; Step S12: forming a lane reference line according to the lane relationship determined in step S11; Step S13: Associating the road facilities and sign information with the lane reference line formed in step S12, thereby obtaining static data; Step S14: Perform static semantic analysis on the static data obtained in step S13, and superimpose the analysis result on the static data.

3. The road environment modeling method according to claim 2, characterized in that: When performing lane diversion, the process of constructing the static data model further includes: When lane splitting, if the vehicle is before the split point, the left lane of the vehicle includes the route of the vehicle before the split point and the split route; If the vehicle is behind the diverging point, when the vehicle is on the main road, the diverging road is the left lane of the vehicle, and obstacles on the diverging road are included in the range of obstacles to be paid attention to; when the vehicle is on the diverging road, the main road is the right lane of the vehicle, and obstacles on the main road are included in the range of obstacles to be paid attention to.

4. The road environment modeling method according to claim 1, characterized in that: When traveling at an intersection, the process of constructing the dynamic data model further includes: If the vehicle is going straight at the intersection, obstacles turning left are included in the range of obstacles to be concerned in the form of forced entry points; obstacles turning right and merging into the vehicle's current lane are projected into the vehicle's current lane in the form of free entry points; If the vehicle turns left at an intersection, obstacles in the opposite lane turning left or going straight are included in the range of obstacles to be concerned in the form of mandatory entry points; obstacles turning right and merging into the vehicle's current lane are projected into the vehicle's current lane in the form of free entry points; If the vehicle turns right at an intersection, the left-turning obstacle in the opposite lane is included in the range of obstacles in the form of a forced entry point; the obstacle going straight into the vehicle's current lane is projected into the vehicle's current lane as a free entry point.

5. The road environment modeling method according to claim 1, characterized in that: Constructing the prediction data model includes: Step S31: Associating the predicted obstacle trajectory with the corresponding dynamic obstacle and superimposing it on the static data; Step S32: Associating the predicted obstacle semantic information with the corresponding dynamic obstacle and superimposing it on the static data.

6. The road environment modeling method according to claim 1, characterized in that: The upstream input data includes data information provided by high-precision maps, positioning devices, perception devices, business layer modules, vehicle bottom modules and prediction modules; The internal historical data includes historical frame data of the environment modeling module, historical decision behaviors of the decision module, and local paths released by the planning module.

7. A road environment modeling device, characterized in that: The device comprises a data receiving unit and a model building unit. The data receiving unit is used to receive upstream input data and internal historical data; The model building unit is used to perform data fusion and data reconstruction based on the received upstream input data and internal historical data, and respectively build a static data model, a dynamic data model and a prediction data model; Constructing the dynamic data model includes the following steps: Step S21: Projecting the obstacle onto a high-precision map to determine its location; Step S22: determining whether the obstacle will conflict with the current lane reference line, and selecting the obstacle to focus on; Step S23: Associating the obstacle of interest selected in step S22 with the lane reference line, thereby obtaining dynamic data; Step S24: Perform dynamic semantic analysis on the dynamic data obtained in step S23, and superimpose the analysis result on the dynamic data; When performing lane merging, the process of constructing the dynamic data model further includes: If the vehicle is on the main road, the obstacles on the merging lane are associated with the right or left lane by giving a free entry point; If the vehicle is in the merging lane, the obstacles on the main road are divided into two cases for association: one is that there is an intersection with the vehicle's current lane in the road structure, and the obstacle is associated with the vehicle's current lane as a free entry point; the other is that there is no intersection with the vehicle's current lane in the road structure, but there is an intersection with the vehicle's future road, and the obstacle is included in the obstacle of interest in the vehicle's current lane as a forced entry point.

8. A storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the road environment modeling method as described in any one of claims 1 to 6.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the road environment modeling method as described in any one of claims 1 to 6.

10. A mobile device, characterized in that: Comprising the terminal device as claimed in claim 9.

Images