Object instantiation method and device

Abstract

The invention discloses an object instantiation method and device. Inputting the multi-sensor information into an obstacle detection model to obtain data of a first center point and / or raster data; wherein the data of the first center point comprises center position information of the first object, and the raster data comprises attributes of each raster; and obtaining a first polygon instance of the first object according to the data of the first center point and / or the raster data. Thus, the contour information of the first object is obtained through the first center point and / or the raster data output by the obstacle detection model, the method is a universal instantiation method and can be suitable for different types of objects or obstacles, iterative calculation of a common clustering algorithm is avoided, and the method is suitable for large-scale popularization and application. The method has the advantages of small calculation amount, small calculation power requirement on equipment and high calculation efficiency, thereby being easy to deploy at a vehicle end.

Description

Object instantiation method and device Technical Field

[0001] The present application relates to the field of smart cars, and more specifically, to a method and device for instantiating an object. Background Art

[0002] Intelligent driving technology eliminates the need for human drivers, so it can theoretically effectively avoid human driving errors, reduce traffic accidents, and improve road transport efficiency. Consequently, intelligent driving technology is gaining increasing attention. With the continuous development of intelligent driving technology, related technologies have been applied to an increasing number of vehicle models, gradually evolving from intelligent driving in specific scenarios to intelligent driving in all scenarios. Intelligent driving technology relies on the collaborative efforts of computer vision and multi-sensor perception, enabling vehicles to achieve intelligent driving without active human intervention.

[0003] Currently, perception algorithms primarily include whitelist-based obstacle detection and clustered obstacle detection based on Lidar point clouds. These two detection methods are only effective in specific scenarios. However, in real-world driving, vehicles encounter a wide variety of objects, such as odd-shaped vehicles and irregular obstacles. Consequently, these two detection methods lack universal applicability.

[0004] Furthermore, based on the above detection methods, it is also impossible to obtain universal object instantiation results. Therefore, how to improve the universality of object instantiation is an urgent problem to be solved.

[0005] Summary of the Invention

[0006] The present application provides an object instantiation method and apparatus, which can instantiate different types of objects, thereby improving the versatility of object instantiation.

[0007] In a first aspect, a method for instantiating an object is provided. The method includes: inputting multi-sensor information into an obstacle detection model to obtain first center point data and / or raster data; wherein the first center point data includes information about the center position of a first object, and the raster data includes attributes of each grid cell; and obtaining a first polygon instance of the first object based on the first center point data and / or raster data.

[0008] In the above technical solution, the contour information of the first object is obtained by using the data of the first center point and / or the raster data output by the obstacle detection model. This is not only a general instantiation method that can be applied to different types of objects or obstacles, but also avoids the iterative calculation of the general clustering algorithm. It has a small amount of calculation, low requirements on the computing power of the device and high computing efficiency, and is therefore easy to deploy on the vehicle side.

[0009] In combination with the first aspect, in certain implementations of the first aspect, a first polygon instance of the first object is obtained based on the data of the first center point and / or the grid data, including: determining the first grid around the grid where the first center point is located as the first polygon instance, and the occupancy information corresponding to the first grid indicates that the first grid is occupied.

[0010] In this way, the center point data of the object output directly by the obstacle detection model does not need to be clustered iteratively to obtain the center point, which improves the computational efficiency of object instantiation.

[0011] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: when the first polygon instance and the second polygon instance of the second object are adhered, determining, based on the first similarity and the second similarity, that the second grid belongs to the first object or the second object. The second grid is located at the location where the first polygon instance and the second polygon instance are adhered, the first similarity includes the similarity between the second grid and the grid where the first center point is located, the second similarity includes the similarity between the second grid and the grid where the second center point is located, and the data of the second center point includes information about the center position of the second object.

[0012] In this way, if the first polygon instance and the second polygon instance are stuck together, the problem of the first object and the second object sticking together can be resolved through multi-dimensional similarity. Especially for dynamic objects, clearly separating the boundaries between the dynamic first object and the second object can facilitate subsequent accurate planning and control.

[0013] In conjunction with the first aspect, in certain implementations of the first aspect, the raster data includes multiple frames of data of the first object. Obtaining a first polygon instance of the first object based on the data of the first center point and / or the raster data includes: fusing the multiple frames of data to obtain the first polygon instance.

[0014] In certain implementations, multiple frames of data are fused to complete the first polygon instance to obtain a completed first polygon instance.

[0015] In an embodiment of the present application, by using raster data for a first object at different frames in the time domain, both visible and invisible faces of the instance of the object can be fully characterized, thereby facilitating more reasonable prediction, planning, and control of the first object. For example, for a dynamic object, since the vehicle is constantly moving, multiple frames of data can be supplemented with raster data at different angles for the dynamic object, thereby obtaining a more complete first polygon instance. For another example, for a static object (e.g., a road cone), if the vehicle is constantly moving, and at the same time, there may be occlusion of a dynamic object between the vehicle and the static object, multiple frames of data can also be supplemented with raster data at different angles for the static object, thereby obtaining a more complete first polygon instance.

[0016] In conjunction with the first aspect, in certain implementations of the first aspect, the raster data includes multiple frames of data. Obtaining a first polygon instance of the first object based on the data of the first center point and / or the raster data includes: fusing the multiple frames of data to obtain multiple first polygon instances of the first object.

[0017] In this way, the grid data output by the multi-frame obstacle detection model can be used in time series to filter out noise and effectively overcome problems such as occlusion and low confidence in distant predictions (for example, when the distant point cloud is sparse or the visual features are not obvious).

[0018] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: clustering the multiple first polygon instances according to height information in the multiple frames of data to obtain different three-dimensional polygon instances.

[0019] In this way, by clustering the first polygon instances at multiple height levels, the objective and real environment can be further restored, which helps to further solve the scenes of vehicles going up and down hills and hanging obstacles.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: segmenting visible surfaces of different three-dimensional polygon instances to obtain boundaries of multiple first objects.

[0021] In this way, by segmenting the 3D polygon instances based on the visible surface information in the raster data, the boundaries of the obstacles can be represented more concisely, which further helps to improve the accuracy of subsequent ego-vehicle planning and control.

[0022] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: inputting the multiple frames of data into a ground estimation model to obtain height information of multiple first objects from the road surface; and performing a traffic area analysis on different three-dimensional polygon instances based on the height information to obtain traffic risk information corresponding to the multiple first objects.

[0023] In this way, for objects with low risk of vehicle passing, such as suspended obstacles and height limit poles during driving, the above-mentioned assessment of the passage risk information of the first object can reduce the probability of false triggering of the autonomous emergency braking system (AEB), thereby improving the user experience.

[0024] In combination with the first aspect, in certain implementations of the first aspect, when the multiple first objects are continuous in space, the method further includes: when the multiple first polygon instances belong to the same line segment, converting the multiple first polygon instances into driving boundary lines.

[0025] In this way, the continuity of continuous first polygon instances is judged, so that multiple first polygon deformation instances belonging to the same line segment can be converted into driving boundary lines. In complex traffic environments, the drivable boundaries of the vehicle can be well determined, which is of great use in scenarios such as changes in traffic reality or the absence of high-precision maps.

[0026] In a second aspect, an object instantiation device is provided, characterized in that the device includes a processing unit, which is used to input multi-sensor information into an obstacle detection model to obtain data and / or raster data of a first center point; wherein the data of the first center point includes central position information of the first object, and the raster data includes attributes of each grid; and based on the data and / or raster data of the first center point, a first polygon instance of the first object is obtained.

[0027] It should be understood that the second aspect is a device corresponding to the first aspect. The beneficial effects brought about by the solution of the second aspect can be referred to the first aspect and will not be elaborated here.

[0028] In combination with the second aspect, in some implementations of the second aspect, the processing unit is specifically used to: determine the first grid around the grid where the first center point is located as a first polygon instance, and the occupancy information corresponding to the first grid indicates that the first grid is occupied.

[0029] In conjunction with the second aspect, in certain implementations of the second aspect, the processing unit is further configured to, when the first polygon instance and the second polygon instance of the second object are adhered, determine, based on the first similarity and the second similarity, that the second grid belongs to the first object or the second object. The second grid is located at the location where the first polygon instance and the second polygon instance are adhered, the first similarity includes the similarity between the second grid and the grid where the first center point is located, the second similarity includes the similarity between the second grid and the grid where the second center point is located, and the data of the second center point includes information about the center position of the second object.

[0030] In conjunction with the second aspect, in certain implementations of the second aspect, the raster data includes multiple frames of data of the first object, and the processing unit is specifically configured to fuse the multiple frames of data to obtain the first polygon instance.

[0031] In certain implementations, multiple frames of data are fused to complete the first polygon instance to obtain a completed first polygon instance.

[0032] In conjunction with the second aspect, in certain implementations of the second aspect, the raster data includes multiple frames of data, and the processing unit is specifically configured to fuse the multiple frames of data to obtain multiple first polygon instances of the first object.

[0033] In combination with the second aspect, in some implementations of the second aspect, the processing unit is further configured to cluster the plurality of first polygon instances based on height information of the plurality of frames of data to obtain different three-dimensional polygon instances.

[0034] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is further configured to segment visible surfaces of different three-dimensional polygon instances to obtain boundaries of multiple first objects.

[0035] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is also used to input multiple frames of data into a ground estimation model to obtain height information of multiple first objects from the road surface; based on the height information, perform traffic area analysis on different three-dimensional polygon instances to obtain traffic risk information corresponding to the multiple first objects.

[0036] In combination with the second aspect, in certain implementations of the second aspect, when multiple first objects are continuous in space, the processing unit is further used to convert multiple first polygon instances into driving boundary lines when the multiple first polygon instances belong to the same line segment.

[0037] In a third aspect, an object instantiation device is provided, which includes: a memory for storing programs; a processor for executing computer program codes or instructions stored in the memory, and when the computer program codes or instructions stored in the memory are executed, the processor is used to execute the method provided in any one of the implementations of the first aspect above.

[0038] In a fourth aspect, the present application provides a processor for executing the method provided by any one of the implementation methods of the first aspect above. In the process of executing these methods, the process of sending the above information and obtaining / receiving the above information in the above methods can be understood as the process of the processor outputting the above information, and the process of the processor receiving the input above information. When outputting the above information, the processor outputs the above information to the interface and transmits it through the interface. After being output by the processor, the above information may also need to undergo other processing before reaching the interface. Similarly, when the processor receives the input above information, the interface obtains / receives the above information and inputs it into the processor. Furthermore, after the interface receives the above information, the above information may need to undergo other processing before being input into the processor.

[0039] For the operations involved, such as transmission, sending, and acquisition / reception, unless otherwise specified, or if they do not conflict with their actual functions or internal logic in the relevant descriptions, they can be understood as output and reception, input and other operations, and can also be understood as transmission, sending and receiving operations performed by radio frequency circuits and antennas. This application does not limit this.

[0040] During implementation, the processor may be a processor specifically configured to execute the methods, or may be a processor that executes computer program code or instructions in a memory to execute the methods, such as a general-purpose processor. The memory may be a non-transitory memory, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or may be disposed on separate chips. The embodiments of the present application do not limit the type of memory or the configuration of the memory and the processor.

[0041] In a fifth aspect, a computer-readable storage medium is provided, which stores program code or instructions for execution by a device, wherein the program code or instructions include a method for executing any one of the implementations of the first aspect.

[0042] In a sixth aspect, a computer program product comprising instructions is provided, which, when run on a computer, enables the computer to execute the method provided in any one of the implementations of the first aspect.

[0043] In the seventh aspect, a chip is provided, which includes a processor and a communication interface. The processor reads instructions stored in the memory through the communication interface and executes the method provided by any one of the implementation methods of the first aspect above.

[0044] Optionally, as an implementation method, the chip may also include a memory, in which computer program code or instructions are stored, and the processor is used to execute the computer program code or instructions stored on the memory. When the computer program code or instructions are executed, the processor is used to execute the method provided in any one of the implementation methods of the first aspect above.

[0045] In an eighth aspect, a mobile carrier is provided, which includes any possible device in the second aspect.

[0046] In some possible implementations, the mobile carrier is a vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is a functional block diagram of a mobile carrier provided in an embodiment of the present application;

[0048] FIG2 is a schematic diagram of sensor distribution applied to a mobile carrier according to an embodiment of the present application;

[0049] FIG3 is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0050] FIG4 is a schematic diagram of an overall process of perception and detection provided by an embodiment of the present application;

[0051] FIG5 is a schematic diagram of a process of instantiating an object provided in an embodiment of the present application;

[0052] FIG6 is a schematic diagram of another object instantiation process provided by an embodiment of the present application;

[0053] FIG7 is a two-dimensional rendering of an instantiation of an object provided in an embodiment of the present application;

[0054] FIG8 is a flowchart of another object instantiation method provided in an embodiment of the present application;

[0055] FIG9 is a two-dimensional rendering of another object instantiation provided in an embodiment of the present application;

[0056] FIG10 is a schematic diagram showing the effect of an application object instantiation method provided in an embodiment of the present application;

[0057] FIG11 is a flow chart of another object instantiation method provided in an embodiment of the present application;

[0058] FIG12 is a schematic diagram showing the effect of another method for instantiating an application object provided in an embodiment of the present application;

[0059] FIG13 is a schematic diagram of an object instantiation device provided in an embodiment of the present application;

[0060] FIG14 is a schematic structural diagram of an object instantiation device provided in an embodiment of the present application;

[0061] FIG15 is a schematic diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solution in this application will be described below with reference to the accompanying drawings.

[0063] To facilitate understanding of the embodiments of the present application, the following points are explained:

[0064] First, in this application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments based on their internal logical relationships.

[0065] Second, in this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of this application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c. Wherein a, b and c can be single or multiple, respectively.

[0066] Third, throughout this application, the terms "first," "second," and various numerical references (e.g., #1, #2, etc.) are used to distinguish between different types of data for ease of description and are not intended to limit the scope of the embodiments of this application. For example, they are used to distinguish between different types of raster data, rather than to describe a specific order or precedence. It should be understood that the terms described in this manner are interchangeable, where appropriate, to allow for the description of solutions beyond the embodiments of this application.

[0067] Fourth, in this application, expressions such as "when," "under the circumstances of," and "if" all imply that a corresponding action will be taken under certain objective circumstances. They do not limit the timeframe, do not require a judgment action to be taken when the action is taken, and do not imply any other limitations. Furthermore, the judgment action following these conditional conjunctions does not imply that the judgment action following the conditional conjunctions is the only condition for achieving the result; additional conditions may also be included to achieve the result.

[0068] Fifth, in this application, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatuses.

[0069] Sixth, in this application, "used to indicate" can include being used for direct indication and being used for indirect indication. When describing that a certain indication information is used to indicate A, it can include that the indication information directly indicates A or indirectly indicates A, and it does not mean that the indication information must carry A.

[0070] The indication methods involved in the embodiments of this application should be understood to encompass various methods that enable the party to be indicated to obtain information to be indicated. The information to be indicated can be sent as a whole or divided into multiple sub-information and sent separately. The transmission period and / or timing of these sub-information can be the same or different. This application does not limit the specific transmission method.

[0071] Seventh, in this application, "storage" may refer to storage in one or more memories. The one or more memories may be provided separately or integrated into an encoder or decoder, a processor, or a communication device. The one or more memories may also be provided in part separately and in part integrated into a decoder, a processor, or a communication device. The memory may be any type of storage medium and is not limited in this application.

[0072] FIG1 is a functional block diagram of a mobile carrier provided in an embodiment of the present application. The mobile carrier may include a perception system 120, a display device 130, and a computing platform 150. The perception system 120 may include several sensors for sensing information about the environment surrounding the mobile carrier 100. For example, the perception system 120 may include a positioning system, which may be a global positioning system (GPS), a Beidou system or other positioning system, an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera.

[0073] Some or all functions of the mobile carrier can be controlled by the computing platform 150. The computing platform 150 may include processors 151 to 15n (n is a positive integer). The processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of the hardware circuit. The logical relationship of the hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 150 can also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory and execute them to achieve corresponding functions.

[0074] The mobile carrier may include an advanced driving assistant system (ADAS). ADAS utilizes a variety of sensors on the mobile carrier (including but not limited to: lidar, millimeter-wave radar, camera, ultrasonic sensor, global positioning system, inertial measurement unit) to obtain information from the surroundings of the mobile carrier, and analyzes and processes the acquired information to achieve functions such as obstacle perception, target recognition, mobile carrier positioning, path planning, driver monitoring / reminders, etc., thereby improving the safety, automation and comfort of mobile carrier driving.

[0075] In the present application, the mobile carrier in the present application may include road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, the mobile carrier may be a vehicle, which is a vehicle in a broad sense, and may be a transportation vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a lawn mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiments of the present application do not specifically limit the type of the mobile carrier.

[0076] Figure 2 is a schematic diagram of the distribution of sensors applied to a mobile carrier provided in an embodiment of the present application. It should be understood that Figure 2 is only an example schematic diagram of a sensor distribution method. There may be other distribution methods, and the embodiment of the present application does not limit this. As shown in Figure 2, the sensors distributed on the mobile carrier include a millimeter wave radar 201, a camera device 202, and a laser radar 203. It may also include other sensors not shown in Figure 2, and the embodiment of the present application does not limit this. For example, the maximum sensing distance of the laser radar is about 150 meters, the maximum sensing distance of the camera device is about 200 meters, the maximum sensing distance of the long-range millimeter wave radar is about 250 meters, and the maximum sensing distance of the medium / short-range millimeter wave radar is about 120 meters.

[0077] Figure 3 is a schematic diagram of a system architecture provided in accordance with an embodiment of the present application. In Figure 3, data acquisition device 360 ​​is used to collect training data. For example, for the general obstacle detection (GOD) model in accordance with an embodiment of the present application, the training data may include image data and / or point cloud data as well as obstacle recognition results. For example, the obstacle recognition results may include obstacle semantic information, three-dimensional position information, etc., wherein the obstacle recognition results may be manually pre-annotated.

[0078] After collecting the training data, the data collection device 360 ​​stores the training data in the database 330 , and the training device 320 obtains the target model / rule 301 through training based on the training data maintained in the database 330 .

[0079] The following describes how the training device 320 obtains the target model / rule 301 based on the training data. The training device 320 processes the input data and compares the output value with the target value until the difference between the value output by the training device 320 and the target value is less than a certain threshold, thereby completing the training of the target model / rule 301.

[0080] It should be noted that, in actual applications, the training data maintained in database 330 may not all be collected by data acquisition device 360, but may also be existing training data received from other devices. It should also be noted that training device 320 may not train target model / rule 301 entirely based on the training data maintained by database 330, but may also obtain training data from the cloud or other places for model training. The above description should not be construed as limiting the embodiments of the present application.

[0081] Exemplarily, for the GOD model, before the training data (also referred to as semantic point cloud training samples) is input into the training device 120, the semantic point cloud training samples can be divided into a plurality of unit data volumes of preset sizes. The unit data volumes may include voxels, point cloud pillars, etc. In this way, irregular semantic point cloud training samples can be converted into regular data volumes, reducing the difficulty of subsequent data processing. The prediction result may include the predicted target of the semantic point cloud data and its three-dimensional position, as well as the probability of having the predicted target at the three-dimensional position. The iteration meeting the preset requirements may include that the difference between the prediction result and the three-dimensional position information of the labeled target is less than a difference threshold, and the difference threshold may be set to 0.01, 0.05, etc., for example. The iteration meeting the preset requirements may also include that the number of iterations is greater than a preset number threshold, and the preset number threshold may be set to 50 times, 60 times, etc., for example. The target model / rule 201 can be a convolutional neural network (CNN) and various network modules based on CNN, such as AlexNet, ResNet, ResNet1001 (pre-activation), Hourglass, Inception, Xception, SENet, etc., and this application does not impose any restrictions on this.

[0082] The target model / rule 301 obtained by training the training device 320 can be applied to different systems or devices, such as the execution device 310 shown in Figure 3. The execution device 310 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, augmented reality (AR) / virtual reality (VR), a mobile carrier, etc., or a server or cloud. For example, the execution device 310 can be the mobile carrier shown in Figure 1, or a computing platform in the mobile carrier. In Figure 3, the execution device 310 is configured with an input / output (I / O) interface 312 for data interaction with external devices. The user can input data to the I / O interface 312 through the client device 340. The input data in this embodiment of the application may include: data to be processed input by the client device. For example, the input data is obtained through the perception system 120 in the mobile carrier of Figure 1 and input to the computing platform of the mobile carrier shown in Figure 1 through the I / O interface 312.

[0083] It should be understood that the training device 310 and the execution device 320 can be deployed in the same device or in different devices. Taking the mobile carrier in Figure 1 as an example, the training device 310 and the execution device 320 can both be deployed on the computing platform of the mobile carrier. Alternatively, the training device 310 can be deployed on a cloud server, and the execution device 320 can be deployed on the computing platform of the mobile carrier. This embodiment of the present application is not limited to this.

[0084] Optionally, the execution device 310 may also include a preprocessing module 313, which is used to preprocess the input data (for example, image data and / or point cloud data) received by the I / O interface 312. If the input data does not need to be preprocessed, the input data received from the I / O interface 312 (for example, the image data and / or point cloud data directly collected in the embodiment of the present application) is directly input into the execution device 310 for processing.

[0085] When the execution device 310 preprocesses the input data, or when the computing module 311 of the execution device 310 performs calculations and other related processing, the execution device 310 can call the data, code, etc. in the data storage system 350 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 350.

[0086] Finally, the I / O interface 312 returns the output result, such as the output result of the data obtained above, to the client device 340, thereby providing it to the user. Alternatively, the I / O interface 312 outputs the output result to the next algorithm module, so that the output result is further processed. For example, in the embodiment of the present application, according to the instantiation method in the embodiment of the present application, the output result of the GOD model is:

[0087] It is worth noting that the training device 320 can generate corresponding target models / rules 301 based on different training data for different goals or different tasks. The corresponding target models / rules 301 can be used to achieve the above goals or complete the above tasks, thereby providing the user with the desired results.

[0088] In the scenario shown in FIG3 , the user can manually provide input data, which can be operated through the interface provided by the I / O interface 312. In another scenario, the client device 340 can automatically send input data to the I / O interface 312. If the client device 340 requires user authorization to automatically send input data, the user can set the corresponding permissions in the client device 340. For example, the computing platform of the mobile carrier shown in FIG1 obtains input data from the perception system. The user can view the results output by the execution device 310 on the client device 340. The specific presentation form can be a display, sound, action, etc., for example, through the display device of the mobile carrier shown in FIG1 . Alternatively, the output result can be further processed and the final processing result can be viewed. For example, in the embodiment of the present application, the result after instantiation of the output result can be displayed on the display device of the mobile carrier. The client device 340 can also serve as a data acquisition terminal, collecting the input data input to the I / O interface 312 and the output result of the output from the I / O interface 312 as new sample data, and storing it in the database 330. Of course, the data may be collected without going through the client device 340 , and the I / O interface 312 may directly store the input data of the I / O interface 312 and the output result of the I / O interface 312 as new sample data into the database 130 .

[0089] It is worth noting that Figure 3 is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 3, the data storage system 350 is an external memory relative to the execution device 310. In other cases, the data storage system 350 can also be placed in the execution device 310.

[0090] In order to better understand the solutions of the embodiments of the present application, the technical terms involved in the present application are briefly explained below.

[0091] 1. Freespace

[0092] The accessible space is also called the drivable space. It is the area around the intelligent driving vehicle where it can be driven, planned, and controlled.

[0093] 2. Voxel

[0094] Voxel is the basic unit of three-dimensional space.

[0095] At present, perception algorithms mainly rely on whitelist-based bounding box (Bbox) detection and Lidar point cloud clustering-based obstacle detection. Among them, whitelist-based Bbox detection requires manual labeling of mined data before model training. The detection model is limited by the efficiency and cost of manual labeling, and the detection model is difficult to generalize. It targets long-tail scenarios and corner cases in the field of intelligent driving. The so-called long-tail scenarios and corner cases refer to possible scenarios with a low probability of occurrence in intelligent driving. The obstacle detection model based on Lidar point cloud clustering relies heavily on the point cloud collected by Lidar. In addition to the high cost of Lidar itself, it also relies on manual processing rules, lacks robustness, and has poor generalization ability. It can only target specific expert scenarios.

[0096] However, in actual driving, the vehicle may encounter various obstacles, such as irregular vehicles and irregular obstacles. The above obstacle detection model cannot effectively cope with different obstacles, and based on the above detection method, it is also impossible to obtain universal object instantiation results.

[0097] To address the above issues, the present application provides an object instantiation method and apparatus, which will be described in detail below with reference to Figures 4 to 15. It should be understood that the "rectangular blocks" in Figures 4 and 6 to 11 represent input data or output data, and the "rounded blocks" represent data processing steps or data processing models.

[0098] FIG4 is a schematic diagram of an overall process of perception and detection provided in an embodiment of the present application.

[0099] The first step is to collect sensor data, which can include information collected by different sensors.

[0100] For example, the information collected by different sensors may include point cloud data information collected by Lidar, distance information collected by millimeter wave radar and / or ultrasonic radar, image information collected by a camera device, and posture information collected by an inertial measurement unit.

[0101] It should be understood that the information collected by different sensors can also be referred to as multimodal sensor information.

[0102] In the second step, the collected sensor data is input into the general obstacle detection (GOD) model to obtain the center point data and grid data.

[0103] The center point data includes the center point of each obstacle (god_center), and the grid data includes multidimensional attributes for each grid. Each grid's multidimensional attributes can include grid occupancy information, grid semantics (e.g., whether the object corresponding to the grid is static or dynamic; or whether the grid is visible or invisible to the object), the height of the object corresponding to the grid, the speed of the object corresponding to the grid, and the orientation of the object corresponding to the grid.

[0104] Exemplarily, when the object corresponding to the grid is a static object or a dynamic object, the semantics of the grid may further specifically include the type of the object corresponding to the grid, for example, the object corresponding to the grid is a road, a curb, a bush, others, or dynamics.

[0105] It should be understood that the grid occupancy information is used to indicate whether the grid is occupied by an object. For example, a grid occupancy information of 1 indicates that the grid is occupied by an object, and a grid occupancy information of 0 indicates that the grid is not occupied by an object. It should be understood that the specific numerical value or symbol of the occupancy information is not limited in the present embodiment of the application, and is only used for example.

[0106] It should be understood that grid data can also be referred to as voxel data, and the multidimensional attributes of each grid can also be referred to as the multidimensional attributes of each voxel. The present embodiment does not limit the name of the grid.

[0107] The third step is to obtain a polygon instance of the object through an instantiation method based on the center point data and / or grid data.

[0108] It should be understood that the specific instantiation method will be described in detail in conjunction with Figures 5 to 11.

[0109] The fourth step is to perform subsequent prediction, planning and control based on the polygonal instances of the object.

[0110] FIG5 is a schematic diagram of a process of instantiating an object provided in an embodiment of the present application.

[0111] S510: Input the multi-sensor information into the obstacle detection model to obtain data and / or grid data of the first center point.

[0112] The data of the first center point includes the center position information of the first object, and the grid data includes the attributes of each grid.

[0113] It should be understood that the multi-sensor information may include information collected by different sensors. For a detailed description, please refer to the relevant description of the first step in FIG. 4 , which will not be repeated here.

[0114] It should be understood that the obstacle detection model can be a GOD model, which can be obtained by training through the system architecture shown in Figure 3. The training process of the GOD model can refer to the relevant description of Figure 3 and will not be repeated here.

[0115] It should also be understood that the specific contents of the attributes of each grid can be found in the relevant description of the second step in FIG. 4 , which will not be elaborated here.

[0116] S520: Obtain a first polygon instance of the first object according to the data of the first center point and / or the grid data.

[0117] The first polygonal instance of the first object can be understood as the outline shape or outline information of the first object. It should be understood that the embodiment of the present application does not limit the number of first objects.

[0118] As a possible implementation, a first polygon instance of the first object is obtained based on the data of the first center point, wherein the data of the first center point also includes size data of the first center point.

[0119] It should be understood that when the data of the first center point includes size data of the first center point in addition to the center position information of the first object, the first polygon instance of the first object can be directly obtained based on the first center point.

[0120] As a possible implementation, a first grid around the grid where the first center point is located is determined as a first polygon instance, wherein occupancy information in grid data of the first grid indicates that the first grid is occupied.

[0121] As a possible implementation manner, when the first polygon instance and the second polygon instance of the second object are adhered, it is determined that the second grid belongs to the first object or the second object based on the first similarity and the second similarity.

[0122] The second grid is located at the adhesion position of the first polygon instance and the second polygon instance, the first similarity includes the similarity between the second grid and the grid where the first center point is located, the second similarity includes the similarity between the second grid and the grid where the second center point is located, and the data of the second center point includes the center position information of the second object.

[0123] It should be understood that the specific method of obtaining the first polygon instance of the first object based on the data of the first center point and the grid data will be described in detail with reference to FIG. 6 and FIG. 7 .

[0124] As a possible implementation, the raster data includes multiple frames of data of the first object, and the multiple frames of data are fused to obtain the first polygon instance. It should be understood that the specific implementation will be described in detail with reference to FIG8 to FIG10.

[0125] It should be understood that the multi-frame data may also be referred to as multi-frame first grid data, wherein the multi-frame first grid data is grid data corresponding to the first object in different frames obtained according to the obstacle detection model.

[0126] As a possible implementation manner, the raster data includes multiple frames of data, and the multiple frames of data are fused to obtain multiple first polygon instances of the first object.

[0127] It should be understood that the multiple frames of data may also be referred to as multiple frames of second raster data, wherein the multiple frames of second raster data are raster data of different frames obtained according to the obstacle detection model.

[0128] In some implementations, multiple first polygon instances are clustered based on different height information in multiple frames of data to obtain different three-dimensional polygon instances, and visible surfaces of the different three-dimensional polygon instances are segmented to obtain multiple boundaries of the first objects.

[0129] In certain implementations, multiple first polygon instances are clustered based on different height information in multiple frames of data to obtain different three-dimensional polygon instances. The multiple frames of data are input into a ground estimation model to obtain height information of multiple first objects from the road surface. Based on the height information, a traffic area analysis is performed on the different three-dimensional polygon instances to obtain traffic risk information corresponding to the multiple first objects.

[0130] In some implementations, when the plurality of first objects are spatially continuous and the plurality of first polygon instances belong to the same line segment, the plurality of first polygon instances are converted into driving boundary lines.

[0131] It should be understood that the specific method of obtaining the first polygon instance of the first object through the raster data will be described in detail with reference to FIG. 11 and FIG. 12 .

[0132] In the above technical solution, the contour information of the first object is obtained by using the data of the first center point and / or the raster data output by the obstacle detection model. This is not only a general instantiation method that can be applied to different types of objects or obstacles, but also avoids the iterative calculation of the general clustering algorithm. It has a small amount of calculation, low requirements on the computing power of the device and high computing efficiency, and is therefore easy to deploy on the vehicle side.

[0133] The specific processes of the above different instantiation methods will be described in detail below with reference to FIG. 6 to FIG. 11 .

[0134] FIG6 is a schematic diagram of another process of instantiating an object provided in an embodiment of the present application.

[0135] S610a: Based on the first center point, search for connected grids around the grid where the first center point is located to obtain a first polygon instance.

[0136] Specifically, a first grid around the grid where the first center point is located is determined as the first polygon instance, wherein the occupancy information corresponding to the first grid indicates that the first grid is occupied.

[0137] Exemplarily, occupancy information of a first grid surrounding the grid where the first center point is located is determined to indicate whether the grid is occupied by an object. If the occupancy information of the first grid indicates that the first grid is occupied by an object, the first grid is determined to be a first polygon instance.

[0138] It should be understood that the embodiment of the present application does not limit the number of first grids.

[0139] Optionally, S610b, based on the second center point, search for connected grids around the grid where the second center point is located to obtain a second polygon instance

[0140] It should be understood that the specific steps may refer to the method of obtaining the first polygon instance in S610a, which will not be described in detail here.

[0141] In this way, the center point data of the object directly output by the GOD model does not need to be clustered iteratively to obtain the center point, which improves the computational efficiency of object instantiation.

[0142] S620 : When the first polygon instance and the second polygon instance are adhered, determine, based on the first similarity and the second similarity, whether the second grid belongs to the first polygon instance or the second polygon instance.

[0143] The second grid is located at the intersection of the first polygon instance and the second polygon instance. In other words, the second grid belongs to both the first polygon instance and the second polygon instance before segmentation. It should be understood that the embodiment of the present application does not limit the number of second grids.

[0144] The first similarity includes the similarity between the second grid and the grid where the first center point is located, the second similarity includes the similarity between the second grid and the grid where the second center point is located, and the data of the second center point includes the center position information of the second object.

[0145] It should be understood that the similarity between the second grid and the grid where the first center point is located can be obtained through the attributes of the second grid and the attributes of the grid where the first center point is located; the similarity between the second grid and the grid where the first center point is located can be obtained through the attributes of the second grid and the attributes of the grid where the second center point is located.

[0146] Specifically, when the first similarity is greater than the second similarity, the second grid is determined to belong to the first polygon instance; when the first similarity is less than the second similarity, the second grid is determined to belong to the second polygon instance. This embodiment of the application does not limit the specific method of calculating the similarity. For example, the first similarity can be different attributes corresponding to the second grid and the grid where the first center point is located.

[0147] In this way, if the first polygon instance and the second polygon instance are stuck together, the problem of the first object and the second object sticking together can be resolved through multi-dimensional similarity. Especially for dynamic objects, clearly separating the boundaries between the dynamic first object and the second object can facilitate subsequent accurate planning and control.

[0148] Figure 7 is a two-dimensional rendering of an instantiation of an object provided by an embodiment of the present application. It should be understood that the grid shown in Figure 7 is not the minimum scale of the grid, and the grid where the center point is located is only for illustration and does not have any limiting effect.

[0149] As shown in FIG7( a ), the connected grids around the grid where the first center point output by the GOD model is located are searched to obtain a first polygon instance.

[0150] As shown in (b) of FIG7 , the first polygon instance and the second polygon instance are adhered. As shown in the figure, there is an adhesion position between the two instances. The first polygon instance after splitting and the second polygon instance after splitting are obtained through the first similarity and the second similarity.

[0151] The above solution is for objects with a center point, but the output of the GOD model may miss individual objects. Figure 8 is a flow chart of another object instantiation method provided in an embodiment of the present application.

[0152] S810 , fusing multiple frames of first raster data of a first object to obtain a first polygon instance.

[0153] The multiple frames of first raster data are raster data corresponding to the first object in different frames obtained according to the obstacle detection model.

[0154] It should be understood that the first object is the object corresponding to the grid data of the third grid in the grid data output by GOD. The grid data of the third grid is the grid data excluding the polygon instance obtained according to the center point, and the occupancy information in the grid data of the third grid is used to indicate whether it is occupied by the object.

[0155] In an embodiment of the present application, by using raster data for a first object at different frames in the time domain, both visible and invisible faces of the instance of the object can be fully characterized, thereby facilitating more reasonable prediction, planning, and control of the first object. For example, for a dynamic object, since the vehicle is constantly moving, multiple frames of first raster data can be supplemented with raster data at different angles for the dynamic object, thereby obtaining a more complete first polygon instance. For another example, for a static object (e.g., a road cone), if the vehicle is constantly moving, and at the same time, there may be occlusion of a dynamic object between the vehicle and the static object, multiple frames of first raster data can also be supplemented with raster data at different angles for the static object, thereby obtaining a more complete first polygon instance.

[0156] Figure 9 is a two-dimensional rendering of another object instantiation provided by an embodiment of the present application. It should be understood that the grid shown in Figure 9 is not the minimum scale of the grid, and Figure 9 uses two frames of grid data as an example.

[0157] As shown in Figure 9, the first raster data of one frame and the first raster data of another frame are both incomplete. By combining the first raster data of these two frames, a completed first polygon instance can be obtained. More frames of first raster data can make the first polygon instance more complete. The embodiment of the present application does not limit the number of frames for multi-frame fusion.

[0158] FIG10 is a schematic diagram showing the effect of an application object instantiation method provided in an embodiment of the present application.

[0159] The upper left corner of Figure 10 shows an irregular-shaped object, more specifically a moving vehicle carrying it. Using the instantiation method described above, the complete outline of the irregular-shaped object can be obtained, as shown in Figure 10. Compared to whitelist-based obstacle detection, which may suffer from unstable detection and incomplete outline detection, the instantiation method of the present embodiment can fully and accurately obtain the outline of the actual irregular-shaped object.

[0160] FIG11 is a flow chart of another object instantiation method provided in an embodiment of the present application.

[0161] S1101 , fusing multiple frames of second raster data to obtain multiple first polygon instances of multiple first objects.

[0162] The multiple frames of second raster data are raster data of different frames obtained according to the obstacle detection model.

[0163] It should be understood that the second raster data may include a plurality of first objects.

[0164] Specifically, the historical raster data in the time series and the raster data of the current frame are fused to obtain a plurality of first polygon instances of the first object.

[0165] In this way, using the raster data output by the multi-frame GOD model in time series can not only filter out noise, but also effectively overcome problems such as occlusion and low confidence in distant predictions (for example, when the distant point cloud is sparse or the visual features are not obvious).

[0166] S1102 , clustering multiple first polygon instances according to height information of multiple frames of second raster data to obtain different three-dimensional polygon instances.

[0167] It should be understood that by clustering multiple first polygon instances based on different height information, three-dimensional polygon instances at different height levels can be obtained.

[0168] In this way, by clustering the first polygon instances at multiple height levels, the objective and real environment can be further restored, which helps to further solve the scenes of vehicles going up and down hills and hanging obstacles.

[0169] S1103 , segmenting the visible surfaces of different three-dimensional polygon instances to obtain boundaries of multiple first objects.

[0170] Specifically, based on the attributes of the visible surface in the semantic information about the visible surface in the grid data of the fourth grid, the visible surface of different three-dimensional polygon instances is segmented to obtain the boundaries of the multiple first objects. The fourth grid includes grids related to different three-dimensional polygon instances in multiple frames of the second grid data.

[0171] In this way, by segmenting the 3D polygon instances based on the visible surface information in the raster data, the boundaries of the obstacles can be represented more concisely, which further helps to improve the accuracy of subsequent ego-vehicle planning and control.

[0172] Optionally, in S1104, multiple frames of second grid data are input into a ground estimation model to obtain height information of multiple first objects from the road surface.

[0173] It should be understood that the ground estimation model is trained in advance.

[0174] Optionally, S1105 performs a traffic area analysis on the different three-dimensional polygon instances obtained in S1102 based on the height information of the multiple first objects from the road surface to obtain traffic risk information corresponding to the multiple first objects.

[0175] It should be understood that when the traffic risk information indicates low risk, it means that the multiple first objects pose little communication barrier to the vehicle; when the traffic risk information indicates high risk, it means that the multiple first objects pose great communication barrier to the vehicle.

[0176] For example, if the first object is a roadside plant that invades the path of the vehicle, and if the height of the plant from the road surface does not affect the vehicle passing under the plant, then the traffic risk information corresponding to the plant indicates a low traffic risk.

[0177] For example, if the first object is a suspended wall at the entrance of an underground parking lot, a traffic area analysis is performed on the three-dimensional polygon instance corresponding to the suspended wall based on the height information of the suspended wall from the road surface, and the traffic risk information corresponding to the suspended wall indicates that the traffic risk is low.

[0178] For another example, if the first object is a height limit pole, based on the height information of the height limit pole from the road surface and the height information of the vehicle, if the three-dimensional polygon instance corresponding to the height limit pole is higher than the height information of the vehicle, the traffic risk information corresponding to the height limit pole indicates that the traffic risk is low.

[0179] In this way, for objects with low risk of vehicle passing such as suspended obstacles and height limit poles during driving, the above-mentioned evaluation of the passage risk information of the first object can reduce the probability of false triggering of AEB, thereby improving the user experience.

[0180] Optionally, after S1101 , if the multiple first objects are spatially continuous objects, S1106 is executed to determine whether the multiple first polygon instances belong to the same line segment.

[0181] As a possible implementation manner, whether the multiple first polygon instances belong to the same line segment is determined by determining whether the interval between two adjacent first polygon instances among the multiple first polygon instances is less than a preset distance.

[0182] For example, Figure 12 is a schematic diagram illustrating the effects of another object instantiation method provided in an embodiment of the present application. As shown in Figure 12, the first object is a barrier barrier on a road edge median strip. The barrier barriers are spatially continuous, the distance between each two adjacent barrier barriers is less than a preset distance, and the barrier barriers belong to the same line segment.

[0183] For another example, the first object is a spacer railing on a road edge isolation strip. The multiple spacer railings are continuous in space, but the distance between two adjacent spacer railings is greater than a preset distance, and the two adjacent spacer railings do not belong to the same line segment.

[0184] It should be understood that the embodiment of the present application does not limit whether multiple first polygon instances belong to the same line segment when the interval distance between two adjacent first polygon instances is equal to the preset distance.

[0185] Optionally, S1107 , when multiple first polygon instances belong to the same line segment, the multiple first polygon instances are converted into driving boundary lines.

[0186] Specifically, according to the driving position and driving boundary of the vehicle, traffic attributes corresponding to the plurality of first multi-variable instances are obtained.

[0187] For example, if the vehicle is on the left side of the driving boundary, the left side of the driving boundary is the passable area, and the right side of the driving boundary is the impassable area. It should be understood that this is just an example and is not limited to this distance.

[0188] In this way, the continuity of continuous first polygon instances is judged, so that multiple first polygon deformation instances belonging to the same line segment can be converted into driving boundary lines. In complex traffic environments, the drivable boundaries of the vehicle can be well determined, which is of great use in scenarios such as changes in traffic reality or the absence of high-precision maps.

[0189] The object instantiation method provided by the embodiment of the present application is described in detail above with reference to Figures 4 to 12. It is understood that in order to implement the above functions, it includes hardware structures and / or software modules corresponding to executing each function.

[0190] Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of this application.

[0191] The object instantiation apparatus provided in the embodiments of the present application is described in detail below with reference to Figures 13 to 15. It should be understood that the description of the apparatus embodiment corresponds to the description of the method embodiment. Therefore, for matters not described in detail, reference can be made to the method embodiment above. For the sake of brevity, some details are not repeated here.

[0192] Figure 13 is a schematic diagram of an object instantiation device provided by an embodiment of the present application. The device may include a processing unit 1320, which is used to perform data processing. Optionally, the device 1300 may also include a transceiver unit 1310, which may implement corresponding communication functions. The transceiver unit 1310 may also be referred to as a communication interface or a communication unit or an interface unit. It should be understood that for the operations such as sending and receiving involved in this application, if there is no special explanation, or if it does not conflict with its actual function or internal logic in the relevant description, it can be more generally understood as operations such as output and input, rather than sending and receiving operations directly performed by the radio frequency circuit and antenna.

[0193] Optionally, the device may further include a storage unit, which may be used to store instructions and / or data. The processing unit 1320 may read the instructions and / or data in the storage unit so that the device implements the aforementioned method embodiment.

[0194] The device can be used to perform the actions in the above method embodiments, the transceiver unit 1310 is used to perform the acquisition-related operations in the above method embodiments, and the processing unit 1320 is used to perform the processing-related operations in the above method embodiments.

[0195] As a design, the device is configured to perform the actions of the method embodiments shown in Figures 4, 5, 6, 8, or 11 above. The execution entity may be a chip, chip system, or processor that supports the object instantiation device in implementing the corresponding method, or a logic module or software that implements all or part of the functionality of the object instantiation device.

[0196] Specifically, the processing unit 1320 is used to input multi-sensor information into the obstacle detection model to obtain data of the first center point and / or raster data; wherein the data of the first center point includes the center position information of the first object, and the raster data includes the attributes of each grid; based on the data of the first center point and / or the raster data, a first polygon instance of the first object is obtained.

[0197] For details not described in detail, please refer to the above method embodiment.

[0198] It should be understood that the specific process of each unit executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0199] The processing unit 1320 in the above embodiment may be implemented by at least one processor or processor-related circuits. The transceiver unit 1310 may be implemented by a transceiver or transceiver-related circuits. The storage unit may be implemented by at least one memory.

[0200] FIG14 is a schematic structural diagram of an object instantiation device provided in an embodiment of the present application.

[0201] As shown in FIG14 , an embodiment of the present application further provides an object instantiation device. The device includes a processor 1410. Optionally, the device also includes a memory 1420. The processor 1410 is coupled to the memory 1420. The memory 1420 is configured to store computer programs, instructions, and / or data. The processor 1410 is configured to execute the computer programs, instructions, and / or data stored in the memory 1420, thereby executing the method described in the above method embodiment.

[0202] Optionally, the device includes one or more processors 1410.

[0203] Optionally, as shown in FIG14 , the device 1400 may further include a memory 1420 .

[0204] Optionally, the device may include one or more memories 1420.

[0205] Optionally, the memory 1420 may be integrated with the processor 1410 or provided separately.

[0206] Optionally, as shown in Figure 14, the device may further include a transceiver 1430, which is used to receive and / or send signals. For example, the processor 1410 is used to control the transceiver 1430 to receive and / or send signals.

[0207] As a solution, the device is used to implement the operations performed by the object instantiation device in the above method embodiment.

[0208] For example, the processor 1410 is used to implement the processing-related operations performed by the object instantiation device in the above method embodiment, and the transceiver 1430 is used to implement the sending and receiving-related operations performed by the object instantiation device in the above method embodiment.

[0209] Figure 15 is a schematic diagram of a chip system provided by an embodiment of the present application, as shown in Figure 15. The chip system (or it can also be called a processing system) includes a logic circuit 1510 and an input / output interface (input / output interface) 1520, and the logic circuit is used to couple with the input interface and transmit data parameters through the input / output interface to execute the method in the above method embodiment. The device installed with the chip system can implement the method and function of the embodiment of the present application. For example, the logic circuit 1510 can be a processing circuit in the chip system to realize the control of the device installed with the chip system, and can also be coupled to a storage unit to call instructions in the storage unit so that the device can implement the method and function of the embodiment of the present application. The input / output interface 1520 can be an input and output circuit in the chip system to output information processed by the chip system, or input data or signaling information to be processed into the chip system for processing.

[0210] As a solution, the chip system is used to implement the operations performed by the object instantiation device in the above method embodiment.

[0211] For example, the logic circuit 1510 is used to implement the processing-related operations in the above method embodiments, and the input / output interface 1520 is used to implement the acquisition-related operations in the above method embodiments.

[0212] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions for implementing the method executed by the object instantiation device in the above method embodiment.

[0213] For example, when the computer program is executed by a computer, the computer can implement the method performed by the object instantiation device in the above method embodiment.

[0214] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to implement the method performed by the object instantiation apparatus in the above method embodiment.

[0215] An embodiment of the present application further provides a mobile carrier, which may include the above-mentioned object instantiation device.

[0216] Optionally, the mobile carrier may be a vehicle.

[0217] An embodiment of the present application also provides a cloud server, which may include the above-mentioned object instantiation device.

[0218] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, which will not be repeated here.

[0219] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0220] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM may include the following forms: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0221] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0222] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0223] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0224] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0226] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0227] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0228] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk, or an optical disk.

[0229] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An object instantiation method, characterized in that, Including: Inputting multi-sensor information into an obstacle detection model to obtain data of a first center point and / or grid data, where the data of the first center point includes the center position information of a first object, and the grid data includes the attributes of each grid; Obtaining a first polygon instance of the first object according to the data of the first center point and / or the grid data.

2. The method according to claim 1, wherein The obtaining the first polygon instance of the first object according to the data of the first center point and / or the grid data includes: Determining a first grid around the grid where the first center point is located as the first polygon instance, and the occupancy information corresponding to the first grid indicates that the first grid is occupied.

3. The method according to claim 2, wherein The method further includes: When the first polygon instance and a second polygon instance of a second object are adhered, determining whether a second grid belongs to the first object or the second object according to a first similarity and a second similarity; wherein, the second grid is located at the adhesion position of the first polygon instance and the second polygon instance, the first similarity includes the similarity between the second grid and the grid where the first center point is located, the second similarity includes the similarity between the second grid and the grid where a second center point is located, and the data of the second center point includes the center position information of the second object.

4. The method according to any one of claims 1 to 3, characterized in that, The grid data includes multiple frames of data of the first object, The obtaining the first polygon instance of the first object according to the data of the first center point and / or the grid data includes: Fusing the multiple frames of data to obtain the first polygon instance.

5. The method according to claim 1, characterized in that The grid data includes multiple frames of data, The obtaining the first polygon instance of the first object according to the data of the first center point and / or the grid data includes: Fusing the multiple frames of data to obtain multiple first polygon instances of the first object.

6. The method according to claim 5, wherein The method further includes: Clustering the multiple first polygon instances according to the height information in the multiple frames of data to obtain different three-dimensional polygon instances.

7. The method according to claim 6, characterized in that, The method further includes: Dissecting the visible surfaces of the different three-dimensional polygon instances to obtain the boundaries of the multiple first objects.

8. The method according to claim 6, characterized in that, The method further includes: Inputting the multiple frames of data into a ground estimation model to obtain the height information of the multiple first objects from the road surface; Performing a passing area analysis on the different three-dimensional polygon instances according to the height information to obtain the passing risk information corresponding to the multiple first objects.

9. The method according to claim 5, wherein When the multiple first objects are continuous in space, the method further includes: When the multiple first polygon instances belong to the same line segment, converting the multiple first polygon instances into a driving boundary line.

10. An object instantiation device, characterized in that, The device includes a processing unit, The processing unit is configured to input multi-sensor information into an obstacle detection model to obtain data of a first center point and / or grid data; wherein, the data of the first center point includes the center position information of a first object, and the grid data includes the attributes of each grid; Obtaining a first polygon instance of the first object according to the data of the first center point and / or the grid data.

11. The device according to claim 10, characterized in that, Specifically, the processing unit is used for: A first grid around the grid where the first center point is located is determined as the first polygon instance, and the occupancy information corresponding to the first grid indicates that the first grid is occupied.

12. The device according to claim 11, characterized in that, The processing unit is further configured to: When the first polygon instance and the second polygon instance of the second object are adhered, determining, according to the first similarity and the second similarity, that the second grid belongs to the first object or the second object; The second grid is located at the adhesion position of the first polygon instance and the second polygon instance, the first similarity includes the similarity between the second grid and the grid where the first center point is located, and the second similarity includes the similarity between the second grid and the grid where the first center point is located. The similarity between the grids where the center point is located, the data of the second center point includes the center position information of the second object.

13. The device according to any one of claims 10 to 12, characterized in that, The raster data includes multiple frames of data of the first object; The processing unit is specifically configured to fuse the multiple frames of data to obtain the first polygon instance.

14. The device according to claim 10, characterized in that, The grid data includes multiple frames of data; The processing unit is specifically configured to fuse the multiple frames of data to obtain multiple first polygon instances of the first object.

15. The device according to claim 14, characterized in that, The processing unit is further configured to cluster the plurality of first polygon instances according to the height information of the plurality of frames of data to obtain different three-dimensional polygon instances.

16. The device according to claim 15, characterized in that, The processing unit is further configured to segment the visible surfaces of the different three-dimensional polygon instances to obtain boundaries of a plurality of the first objects.

17. The device according to claim 15, characterized in that, The processing unit is further configured to: Inputting the plurality of frames of data into a ground estimation model to obtain a plurality of height information of the first objects from the road surface; According to the height information, a traffic area analysis is performed on the different three-dimensional polygon instances to obtain traffic risk information corresponding to the plurality of first objects.

18. The device according to claim 14, characterized in that, In the case that the plurality of first objects are continuous in space, the processing unit is further configured to: When a plurality of the first polygon instances belong to the same line segment, the plurality of the first polygon instances are converted into driving boundary lines.

19. An object instantiation device, characterized in that, The device comprises a processor, the processor is coupled to a memory, the memory is used to store computer program codes or instructions, and the processor is used to execute the computer program codes or instructions in the memory, so that the device performs the method according to any one of claims 1 to 9.

20. A mobile carrier, characterized in that, The mobile carrier comprises the apparatus according to any one of claims 10 to 19.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program codes or instructions, and when the computer program codes or instructions are executed on a computer, the computer is enabled to perform the method according to any one of claims 1 to 9.

22. A computer program product, characterized in that, When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 9.

23. A chip, characterized in that, The chip is coupled to a memory and is used to read and execute program instructions stored in the memory to implement the method according to any one of claims 1 to 9.

Images