Simulation test method and device based on sensor and electronic equipment

Abstract

The invention provides a simulation test method and device based on a sensor, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a global envelope for simulating a test scene; performing region division processing on the global envelope according to a plurality of sensors deployed to the vehicle to obtain local envelopes corresponding to the plurality of sensors respectively; performing envelope fitting processing on the local envelopes respectively corresponding to the plurality of sensors according to detection geometries of the sensors to obtain fitting envelopes respectively corresponding to the plurality of sensors; and performing simulation test processing in the simulation test scene according to the fitting envelopes corresponding to the plurality of sensors. Through the method and the device, the real envelope in the real world can be accurately fitted, so that the simulation test effect is improved.

Description

Technical Field

[0001] The present application relates to vehicle technology, and in particular to a sensor-based simulation test method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] Vehicle simulation testing refers to the process of simulating the static and dynamic traffic environments of the real world through computer technology, allowing autonomous vehicles and their algorithms to be tested in virtual scenarios (simulation test scenarios). Simulation testing is designed to verify and evaluate the performance and safety of autonomous driving systems while reducing the cost, time and risk of actual road testing.

[0003] Vehicle simulation testing requires simulating the detection characteristics of sensors. In the solutions provided by related technologies, the sensor envelope shape provided by the simulation software is usually used for simulation. However, the sensor envelope shape is too simple and rigid, and cannot meet the various complex envelope requirements in the real world, resulting in poor simulation test results. Summary of the invention

[0004] The present application provides a sensor-based simulation test method, device, electronic device, computer-readable storage medium and computer program product, which can accurately fit the real envelope in the real world, thereby improving the simulation test effect.

[0005] The technical solution of this application is implemented as follows:

[0006] The present application provides a sensor-based simulation test method, comprising:

[0007] Get the global envelope for simulating test scenarios;

[0008] Performing region division processing on the global envelope according to a plurality of sensors deployed to the vehicle to obtain local envelopes corresponding to the plurality of sensors respectively;

[0009] Perform envelope fitting processing on the local envelopes respectively corresponding to the multiple sensors according to the detection geometry of the sensors to obtain the fitting envelopes respectively corresponding to the multiple sensors;

[0010] A simulation test process is performed in the simulation test scenario according to the fitting envelopes respectively corresponding to the multiple sensors.

[0011] The present application provides a sensor-based simulation test device, comprising:

[0012] An acquisition module, used to acquire a global envelope for a simulation test scenario;

[0013] A region division module, used for performing region division processing on the global envelope according to a plurality of sensors deployed to the vehicle, to obtain local envelopes corresponding to the plurality of sensors respectively;

[0014] An envelope fitting module, used for performing envelope fitting processing on the local envelopes respectively corresponding to the multiple sensors according to the detection geometry of the sensors, to obtain fitting envelopes respectively corresponding to the multiple sensors;

[0015] The simulation test module is used to perform simulation test processing in the simulation test scenario according to the fitting envelopes respectively corresponding to the multiple sensors.

[0016] The present application provides an electronic device, including:

[0017] A memory for storing executable instructions;

[0018] The processor is used to implement the sensor-based simulation test method provided in the present application when executing the executable instructions stored in the memory.

[0019] The present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute instructions to implement the sensor-based simulation test method provided by the present application.

[0020] The present application provides a computer program product, which includes executable instructions for causing a processor to execute and implement the sensor-based simulation test method provided by the present application.

[0021] This application has the following beneficial effects:

[0022] This application obtains a global envelope for a simulation test scenario; performs regional division processing on the global envelope according to multiple sensors deployed to the vehicle to obtain local envelopes corresponding to the multiple sensors; performs envelope fitting processing on the local envelopes corresponding to the multiple sensors according to the detection geometry of the sensors to obtain fitting envelopes corresponding to the multiple sensors; performs simulation test processing in a simulation test scenario according to the fitting envelopes corresponding to the multiple sensors. This application performs envelope fitting processing on the basis of the local envelope, so that the shape of the obtained fitting envelope conforms to the detection geometry of the sensor, that is, it can accurately fit the real envelope in the real world, thereby improving the simulation test effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a schematic diagram of the architecture of a sensor-based simulation test system provided in an embodiment of the present application;

[0025] Figure 2 It is a structural diagram of a server provided in an embodiment of the present application;

[0026] Figure 3A It is a first flow chart of the sensor-based simulation test method provided in an embodiment of the present application;

[0027] Figure 3B is a second flow chart of the sensor-based simulation test method provided in an embodiment of the present application;

[0028] Figure 3C is a third flow chart of the sensor-based simulation test method provided in an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of a global envelope provided by an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of a detection geometry provided in an embodiment of the present application;

[0031] Figure 6 is a first schematic diagram of a fitting envelope provided in an embodiment of the present application;

[0032] Figure 7 is a second schematic diagram of a fitting envelope provided in an embodiment of the present application;

[0033] Figure 8 is a schematic diagram of a temporary envelope provided in an embodiment of the present application;

[0034] Fig. 9 is a schematic diagram of detecting a target object by an ultrasonic radar sensor provided in an embodiment of the present application;

[0035] Fig.10 is a schematic diagram of a flow chart of calculating object coordinates provided in an embodiment of the present application;

[0036] Fig.11 It is a flowchart of outputting or discarding object coordinates provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0038] In the following description, reference is made to "some embodiments", which describes a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. In the following description, reference is made to the term "multiple" to mean at least two.

[0039] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0041] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0042] 1) Sensor: In the embodiments of the present application, it specifically refers to the vehicle-mounted sensor. The sensor is an important part of the vehicle's electronic control system, which is used to sense various parameters of the vehicle or the outside world and convert them into electrical signals and input them into the Electronic Control Unit (ECU), thereby realizing the control and monitoring of the vehicle. The sensor involved in the embodiments of the present application can be a real sensor or a simulated sensor, depending on the specific simulation test method.

[0043] 2) Envelope: Also known as detection range, it refers to the spatial range in which the sensor can effectively detect and identify a specific object during its operation. The envelope is used to reflect the detection characteristics of the sensor.

[0044] 3) Detection geometry: It is the geometric parameterization of the envelope, which is used to define the shape of the envelope. For example, the envelope can be abstractly described by a polygonal pyramid or a cone.

[0045] 4) Simulation test scenario: It is a virtual scenario simulated by a computer, which may include, for example, a static environment (such as roads, traffic lights, etc.) and a dynamic traffic environment (such as other vehicles and pedestrians).

[0046] 5) Hardware-In-the-Loop (HIL): It is a simulation test method that combines real hardware devices (such as ECU, sensors, actuators, etc.) with simulated test scenarios (usually implemented through simulators). In HIL simulation testing, the hardware devices are connected to the simulator, which simulates the simulation test scenarios, so as to conduct comprehensive and systematic testing on the hardware devices.

[0047] The embodiments of the present application provide a sensor-based simulation test method, device, electronic device, computer-readable storage medium, and computer program product, which can accurately fit the real envelope in the real world, thereby improving the simulation test effect. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of terminal devices or as a server.

[0048] See also Figure 1 , Figure 1 It is an architectural diagram of a sensor-based simulation test system 100 provided in an embodiment of the present application, wherein a terminal device 400 is connected to a server 200 via a network 300, wherein the network 300 may be a wide area network or a local area network, or a combination of the two.

[0049] In some embodiments, taking the electronic device as a terminal device as an example, the sensor-based simulation test method provided in the embodiments of the present application can be implemented by the terminal device. For example, the terminal device 400 obtains a global envelope for a simulation test scenario; performs regional division processing on the global envelope according to multiple sensors deployed to the vehicle to obtain local envelopes corresponding to multiple sensors; performs envelope fitting processing on the local envelopes corresponding to the multiple sensors according to the detection geometry of the sensor to obtain fitting envelopes corresponding to the multiple sensors; performs simulation test processing in the simulation test scenario according to the fitting envelopes corresponding to the multiple sensors. The above method is suitable for situations where the simulation test requirements are relatively simple, and the simulation test can be implemented locally on the terminal device 400.

[0050] In some embodiments, taking the electronic device as a server as an example, the sensor-based simulation test method provided in the embodiments of the present application can be implemented by the server. For example, the server 200 obtains a global envelope for a simulation test scenario; performs regional division processing on the global envelope according to multiple sensors deployed to the vehicle to obtain local envelopes corresponding to multiple sensors; performs envelope fitting processing on the local envelopes corresponding to the multiple sensors according to the detection geometry of the sensor to obtain fitting envelopes corresponding to the multiple sensors; performs simulation test processing in the simulation test scenario according to the fitting envelopes corresponding to the multiple sensors. In the above manner, the server 200 has powerful computing power and storage capacity, and can process large amounts of data and computing tasks, so it can meet a variety of simulation test needs.

[0051] In some embodiments, the sensor-based simulation test method provided in the embodiments of the present application can be implemented collaboratively by a terminal device and a server. For example, user operations or hardware device behaviors are simulated by the terminal device 400, or simulation test results are displayed by the terminal device 400; simulation test scenarios are constructed and data and computing tasks are processed by the server 200. In this way, on the one hand, the flexibility of the simulation test process can be improved, that is, the various parameters involved in the simulation test process can be set by the terminal device 400; on the other hand, the efficiency of the simulation test can be improved, that is, efficient simulation testing can be achieved through the powerful computing power and storage capacity of the server 200.

[0052] In some embodiments, the terminal device 400 or the server 200 can implement the sensor-based simulation test method provided in the embodiment of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as simulation test software; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP, and the small program can be controlled to run or closed by the user. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0053] Take the electronic device provided in the embodiment of the present application as an example, which is a server. Figure 2 , Figure 2 is a schematic diagram of the structure of the server 200 provided in an embodiment of the present application, Figure 2The server 200 shown includes: at least one processor 210, a memory 250 and at least one network interface 220. The various components in the server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2 Various buses are labeled as bus system 240 .

[0054] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0055] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 210.

[0056] The memory 250 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiment of the present application is intended to include any suitable type of memory.

[0057] In some embodiments, memory 250 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0058] Operating system 251, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0059] A network communication module 252, used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0060] In some embodiments, the sensor-based simulation test device provided in the embodiments of the present application can be implemented in a software manner. Figure 2 A sensor-based simulation test device 255 stored in the memory 250 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: an acquisition module 2551, a region division module 2552, an envelope fitting module 2553, and a simulation test module 2554. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0061] The sensor-based simulation test method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the electronic device provided in the embodiment of the present application.

[0062] See also Figure 3A , Figure 3A is a flow chart of a sensor-based simulation test method provided in an embodiment of the present application. The sensor-based simulation test method provided in an embodiment of the present application can be implemented by an electronic device, such as a terminal device and / or a server. Figure 3A The steps shown are explained.

[0063] In step 101, a global envelope for simulating a test scenario is obtained.

[0064] Here, a global envelope (or initial envelope) for the simulation test scene is obtained. For example, the global envelope can be an envelope provided by the simulation test software. The global envelope covers a 360-degree field of view (FOV) angle centered on the vehicle. The field of view angle is also called the detection angle. The embodiment of the present application does not limit the shape of the global envelope. For example, the plane projection of the global envelope can be an octagon.

[0065] It is worth noting that the simulation test scene is a virtual scene obtained by simulating the scene in the real world through computer technology. The simulation test scene can include static environment and dynamic traffic environment. The static environment includes fixed objects such as roads and traffic lights, and the dynamic traffic environment includes objects participating in traffic such as other vehicles (or surrounding vehicles) and pedestrians.

[0066] In step 102 , the global envelope is divided into regions according to a plurality of sensors deployed on the vehicle to obtain local envelopes corresponding to the plurality of sensors.

[0067] Here, when multiple sensors for deployment to a vehicle (here, a real vehicle) are known, the global envelope is divided into regions according to the multiple sensors for deployment to the vehicle to obtain local envelopes corresponding to the multiple sensors. For example, the global envelope is divided into regions according to the deployment position of each sensor and the field of view angle of each sensor, wherein the deployment position and the field of view angle can be arbitrarily set according to actual conditions.

[0068] It is worth noting that the embodiments of the present application do not limit the type of sensor, for example, it can be a radar sensor (which can be further subdivided into various types of radar sensors), a camera sensor, etc. The "multiple sensors" involved in the embodiments of the present application can refer to multiple sensors of the same type, for example, multiple sensors are ultrasonic radars. The sensors involved in the embodiments of the present application can be real sensors or simulated sensors, depending on the specific simulation test method.

[0069] It is worth noting that the local envelopes corresponding to multiple sensors jointly occupy part or all of the global envelope. For example, multiple sensors are only used to detect the front of the vehicle, then the local envelopes corresponding to the multiple sensors jointly occupy part of the global envelope; for another example, multiple sensors together constitute a panoramic sensor for achieving 360-degree panoramic field of view coverage around the vehicle, then the local envelopes corresponding to the multiple sensors jointly occupy the entire global envelope.

[0070] In step 103, envelope fitting processing is performed on the local envelopes corresponding to the multiple sensors respectively according to the detection geometry of the sensors to obtain the fitting envelopes corresponding to the multiple sensors respectively.

[0071] The detection geometry is the geometric parameterization of the envelope and is used to define the shape of the envelope. For example, there are two types of detection geometries: polygonal pyramid and cone.

[0072] Considering that the local envelope obtained in step 102 may not be consistent with the real envelope in the real world in shape, if simulation test processing is performed directly in the simulation test scene according to the local envelope, the simulation test effect will be poor. For example, an object in the simulation test scene is located in the local envelope corresponding to a sensor, but is actually not located in the real envelope corresponding to the sensor; for another example, an object in the simulation test scene is located in the real envelope corresponding to a sensor, but is not located in the local envelope corresponding to the sensor.

[0073] Therefore, in the embodiment of the present application, envelope fitting processing is performed on the local envelopes corresponding to the multiple sensors according to the detection geometry of the sensor to obtain the fitted envelopes corresponding to the multiple sensors. For example, the local envelope corresponding to each sensor is expanded (or extended) so that the shape of the expanded envelope conforms to the detection geometry. In the above manner, it is ensured that the shape of the fitted envelope conforms to the detection geometry, and the fitted envelope is consistent with the real envelope as much as possible.

[0074] In step 104, simulation test processing is performed in a simulation test scenario according to the fitting envelopes corresponding to the multiple sensors respectively.

[0075] For example, the parameters that multiple sensors can sense in a simulation test scenario are determined based on the fitting envelopes corresponding to the multiple sensors, and the vehicle (here refers to the virtual self-vehicle) in the simulation test scenario is controlled to perform automatic driving based on these parameters, and the automatic driving algorithm is optimized through continuous iteration and debugging, wherein the automatic driving algorithm can be used for Advanced Driver Assistance Systems (ADAS), and of course, can also be used for other systems. Since the fitting envelope can accurately fit the real envelope, it can improve the effect of the simulation test, making the authenticity of the simulation test as close as possible to the real vehicle test.

[0076] like Figure 3A As shown, the embodiment of the present application obtains a global envelope for a simulation test scenario; performs regional division processing on the global envelope according to multiple sensors deployed to the vehicle to obtain local envelopes corresponding to the multiple sensors; performs envelope fitting processing on the local envelopes corresponding to the multiple sensors according to the detection geometry of the sensors to obtain fitting envelopes corresponding to the multiple sensors; performs simulation test processing in a simulation test scenario according to the fitting envelopes corresponding to the multiple sensors. The embodiment of the present application further performs envelope fitting processing on the basis of the local envelope, so that the shape of the obtained fitting envelope conforms to the detection geometry of the sensor, that is, it can accurately fit the real envelope in the real world, thereby improving the simulation test effect.

[0077] In some embodiments, see Figure 3B , Figure 3B is a flow chart of a sensor-based simulation test method provided in an embodiment of the present application. Figure 3A The illustrated step 103 can be implemented through step 201 or step 202, and will be described in conjunction with each step.

[0078] In step 201, when the plane projection of the local envelope corresponding to the target sensor is a trapezoid or a convex quadrilateral whose two sets of opposite sides are not parallel, the local envelope corresponding to the target sensor is subjected to envelope fitting processing according to a polygonal pyramid to obtain a fitting envelope corresponding to the target sensor.

[0079] For ease of understanding, an example of performing envelope fitting processing on a local envelope corresponding to any sensor (named as a target sensor) is used for explanation.

[0080] It is known that the detection geometric bodies of sensors include polygonal pyramids and cones. When the plane projection of the local envelope corresponding to the target sensor is a trapezoid or a convex quadrilateral (trapezoid-like) with two sets of opposite sides that are not parallel, it is proved that the plane projection has a set of opposite sides that can be extended to intersect and form a triangular plane projection. Therefore, the local envelope corresponding to the target sensor is envelope fitted according to the polygonal pyramid to obtain the fitting envelope corresponding to the target sensor, that is, the shape of the fitting envelope corresponding to the target sensor conforms to the polygonal pyramid, and the plane projection of the fitting envelope corresponding to the target sensor is a triangle.

[0081] In some embodiments, the above-mentioned envelope fitting processing of the local envelope corresponding to the target sensor according to the polygonal pyramid to obtain the fitting envelope corresponding to the target sensor can be achieved in the following manner: the local envelope corresponding to the target sensor is extended along a set of opposite sides in the plane projection toward the target sensor until the extension lines intersect to obtain a polygonal pyramid envelope whose plane projection is a triangle; the polygonal pyramid envelope is determined as the fitting envelope corresponding to the target sensor.

[0082] Here, a set of opposite sides facing the target sensor (i.e., facing the emission source) is determined in the plane projection of the local envelope corresponding to the target sensor, and the set of opposite sides is extended until the extension lines intersect, so that the plane projection is a triangular envelope, and the shape of the envelope conforms to a polygonal pyramid, so the envelope is called a polygonal pyramid envelope. Finally, the polygonal pyramid envelope is determined as the fitting envelope corresponding to the target sensor, and the envelope fitting process for the target sensor is completed.

[0083] The above method can easily and quickly construct a fitting envelope that conforms to the shape of a polygonal pyramid by extending the edge line without performing a lot of calculations.

[0084] In step 202, when the plane projection of the local envelope corresponding to the target sensor is a rectangle, envelope fitting processing is performed on the local envelope corresponding to the target sensor according to a cone to obtain a fitting envelope corresponding to the target sensor.

[0085] When the plane projection of the local envelope corresponding to the target sensor is a rectangle, the two sets of opposite sides of the plane projection are parallel and cannot be extended to intersect, that is, the local envelope corresponding to the target sensor cannot be fitted according to the polygonal pyramid. Therefore, the local envelope corresponding to the target sensor is fitted according to the cone to obtain the fitting envelope corresponding to the target sensor.

[0086] In some embodiments, the above-mentioned envelope fitting processing of the local envelope corresponding to the target sensor according to the cone can be implemented in the following manner to obtain the fitting envelope corresponding to the target sensor: construct a temporary envelope that covers the local envelope corresponding to the target sensor and whose plane projection is a semicircle; delete the fitting envelopes corresponding to other sensors in the temporary envelope to obtain the remaining cone envelope; wherein the other sensors are distinguished from the target sensor; and determine the cone envelope as the fitting envelope corresponding to the target sensor.

[0087] Here, firstly, a temporary envelope is constructed that covers the local envelope corresponding to the target sensor and whose plane projection is a semicircle. The temporary envelope will inevitably cover the fitting envelopes corresponding to other sensors different from the target sensor. In order to avoid the unreasonableness caused by the temporary envelope being too large, the area of ​​the temporary envelope can be constrained to be as small as possible when constructing the temporary envelope, that is, the plane projection of the temporary envelope intersects with the plane projection of the local envelope corresponding to the target sensor. Then, the fitting envelopes corresponding to other sensors are deleted from the temporary envelope, and the shape of the remaining envelope conforms to the cone. For the convenience of distinction, the remaining envelope is named the cone envelope, wherein the fitting envelopes corresponding to other sensors can be obtained by envelope fitting processing according to a polygonal pyramid. Finally, the cone envelope is determined as the fitting envelope corresponding to the target sensor.

[0088] The above method first constructs a temporary envelope whose plane projection is a semicircle, and then excludes the fitting envelopes corresponding to other sensors in the temporary envelope through the elimination method. Through geometric transformation, a fitting envelope with a shape that conforms to the cone is conveniently and quickly constructed without a lot of calculations.

[0089] like Figure 3B As shown, in the embodiment of the present application, when the detection geometry includes a polygonal pyramid and a cone, a suitable detection geometry is selected for envelope fitting processing according to the shape of the local envelope corresponding to the target sensor, thereby improving the success rate and accuracy of the envelope fitting processing and ensuring that the shape of the obtained fitting envelope conforms to the detection geometry.

[0090] In some embodiments, see Figure 3C , Figure 3C is a flow chart of a sensor-based simulation test method provided in an embodiment of the present application. Figure 3AThe illustrated step 104 can be implemented through steps 301 to 304 , which will be described in conjunction with each step.

[0091] In step 301, detection processing is performed in a simulation test scenario by a target sensor.

[0092] For ease of understanding, an arbitrary sensor (named as a target sensor) is taken as an example to illustrate the process of performing simulation test processing in a simulation test scenario.

[0093] First, detection processing is performed in a simulated test scenario using the target sensor.

[0094] In step 302, object coordinates of a target object detected by the detection process are determined.

[0095] When an object (named as a target object) is detected through the detection process, the object coordinates of the target object are determined. The type of the target object is not limited, that is, it can be any object in the simulation test scene, such as another vehicle, a pedestrian, etc.

[0096] In some embodiments, the above-mentioned determination of the object coordinates of the target object detected by the detection processing can be achieved in the following manner: determining the intersecting surface covered by the detection signal of the target sensor among the multiple surfaces of the target object; determining the lowest vertex with the smallest vertical coordinate and the highest vertex with the largest vertical coordinate among the multiple vertices of the intersecting surface; determining the vertical coordinate difference between the highest vertex and the lowest vertex as the object vertical coordinate; constructing an object line segment based on the lowest vertex and the vertex with the smallest vertical coordinate difference with the lowest vertex; merging the horizontal coordinates of the points in the object line segment, the vertical coordinates of the points in the object line segment, and the object vertical coordinate to obtain the object coordinates of the target object.

[0097] Here, the intersecting surfaces covered by the detection signal of the target sensor are determined among the multiple surfaces of the target object. Generally speaking, the detection signal of the target sensor cannot cover all surfaces of the target object, and the intersecting surface may be only a part of a certain surface. Therefore, in the embodiment of the present application, further calculations are performed based on multiple vertices (usually 4 vertices) of the intersecting surface to obtain object coordinates that can accurately describe the target object.

[0098] For example, among the multiple vertices of the intersecting surface, the vertex with the smallest vertical coordinate (named as the lowest vertex) and the vertex with the largest vertical coordinate (named as the highest vertex) are determined, wherein the coordinates of each vertex include a horizontal coordinate (x coordinate), a vertical coordinate (y coordinate) and a vertical coordinate (z coordinate). Then, on the one hand, the vertical coordinate difference between the highest vertex and the lowest vertex is determined as the object vertical coordinate, that is, the object vertical coordinate = the vertical coordinate of the highest vertex - the vertical coordinate of the lowest vertex, and the obtained object vertical coordinate is used to describe the height of the target object. On the other hand, an object line segment is constructed based on the lowest vertex and the vertex with the smallest vertical coordinate difference with the lowest vertex (the positive or negative value is not considered here, that is, the absolute value of the vertical coordinate difference is taken and then it is determined whether it is the smallest). Finally, the horizontal coordinate of a point in the object line segment, the vertical coordinate of the point in the object line segment and the object vertical coordinate are merged to obtain the object coordinate of the target object. It can be understood that the object line segment includes multiple points, so the obtained object coordinates also include multiple points.

[0099] The above method comprehensively considers the vertical coordinates of the object and the object line segments, and the obtained multiple object coordinates can achieve comprehensive and complete coverage of the target object, that is, can accurately describe the target object.

[0100] In some embodiments, the number of intersecting surfaces includes multiple; after determining the intersecting surfaces covered by the detection signal of the target sensor among the multiple surfaces of the target object, the sensor-based simulation test method also includes: determining the target intersecting surface with the largest coverage area covered by the detection signal of the target sensor among the multiple intersecting surfaces; the above-mentioned determination of the lowest vertex with the smallest vertical coordinate and the highest vertex with the largest vertical coordinate among the multiple vertices of the intersecting surface can be achieved in the following manner: determining the lowest vertex with the smallest vertical coordinate and the highest vertex with the largest vertical coordinate among the multiple vertices of the target intersecting surface.

[0101] Here, when there are multiple intersection surfaces, the coverage area of ​​each intersection surface covered by the detection signal of the target sensor is determined, and the intersection surface with the largest coverage area is determined as the target intersection surface, so that the object coordinates can be calculated based on multiple vertices of the target intersection surface. Since the target intersection surface has the largest coverage area, the target intersection surface contains richer and more accurate information, which can more comprehensively describe the target object, thereby helping to improve the reliability of the object coordinates calculated based on the target intersection surface.

[0102] In step 303, when the object coordinates are within the fitting envelope corresponding to the target sensor, the automatic driving process is performed according to the object coordinates.

[0103] Since the coordinates of the fitting envelope corresponding to the target sensor in the simulation test scene are known, it can be determined whether the object coordinates are within the fitting envelope corresponding to the target sensor.

[0104] When the object coordinates are within the fitting envelope corresponding to the target sensor, it is proved that the object coordinates are valid data, so the autonomous driving process is performed according to the object coordinates. For example, the object coordinates are used as the input of the autonomous driving algorithm to obtain the autonomous driving strategy, and the vehicle movement in the simulation test scene is controlled according to the autonomous driving strategy to achieve autonomous driving processing. The results of the autonomous driving process (such as whether the vehicle has collided) can be used to optimize the autonomous driving algorithm.

[0105] In some embodiments, before performing autonomous driving processing according to the object coordinates, the sensor-based simulation test method further includes: converting the object coordinates from the world coordinate system to the sensor coordinate system.

[0106] Here, the object coordinates are described in the world coordinate system in the simulation test scenario. Therefore, it is necessary to convert the object coordinates from the world coordinate system to the sensor coordinate system, and then perform autonomous driving processing based on the object coordinates in the sensor coordinate system. For example, the object coordinates in the sensor coordinate system are used as the input of the autonomous driving algorithm, thereby avoiding unreliable input due to coordinate system confusion and ensuring the accuracy of the input of the autonomous driving algorithm.

[0107] In step 304 , when the object coordinates are not within the fitted envelope corresponding to the target sensor, the object coordinates are discarded.

[0108] When the object coordinates are not within the fitting envelope corresponding to the target sensor, it is proved that the object coordinates are invalid data, and thus the object coordinates are discarded.

[0109] like Figure 3C As shown, the embodiment of the present application obtains an accurate conclusion on whether the object coordinates are valid by judging whether the object coordinates are within the fitting envelope corresponding to the target sensor. If the object coordinates are valid, autonomous driving processing is performed according to the object coordinates. If the object coordinates are invalid, the object coordinates are discarded, thereby making the simulation test as close as possible to the actual vehicle test, and effectively improving the effect of the simulation test.

[0110] Below, an exemplary application of the embodiment of the present application in an actual application scenario will be described. For ease of understanding, an example will be given in which the sensor is an ultrasonic radar sensor and a simulation test is implemented in a hardware-in-the-loop (HIL) manner.

[0111] Since the envelope shape (i.e., the global envelope) provided in the simulation test software is single and rigid and cannot accurately simulate the sensor characteristics, in an embodiment of the present application, the envelope shape provided in the simulation test software is divided into regions, and then the envelope fitting process is performed to obtain the fitting envelopes corresponding to the multiple ultrasonic radar sensors, thereby being able to accurately fit the real envelope of the ultrasonic radar sensor.

[0112] The present application embodiment provides Figure 4 The envelope diagram (i.e., plane projection) of the global envelope of the divided area is shown. Figure 4 The local envelopes corresponding to the 12 probes (i.e., ultrasonic radar sensors) are shown in FIG. Figure 4 This is just an example. In other cases, there may be more or fewer probes, and the area division method is not limited to Figure 4 shown.

[0113] On the basis of the area division process, the local envelopes corresponding to the multiple ultrasonic radar sensors are respectively subjected to envelope fitting process according to the detection geometry of the ultrasonic radar sensor, so as to obtain the fitting envelopes corresponding to the multiple ultrasonic radar sensors. For ultrasonic radar sensors, the detection geometry usually includes two types: polygonal pyramid and cone (or ice cream type), such as Figure 5 Based on this, according to the shape of the local envelope itself, a suitable detection geometry is selected to perform envelope fitting processing to obtain a fitting envelope.

[0114] For example, Figure 4 The plane projections of the local envelopes corresponding to probes 1, 2, 7, and 8 are all trapezoidal (a convex quadrilateral with two sets of opposite sides that are not parallel). Therefore, a polygonal pyramid is selected for envelope fitting to obtain a fitting envelope, that is, the local envelope with a trapezoidal plane projection is extended to obtain a fitting envelope with a triangular plane projection. Figure 6 The fitting envelope corresponding to probe 1 is shown, where the FOV angles of probes 1, 2, 7, and 8 are all 16 degrees. Figure 4 The plane projections of the local envelopes corresponding to the probes 3, 6, 9, and 12 in the figure are all trapezoidal. The envelope fitting process is similar to the above-mentioned trapezoidal-like process, that is, the local envelope with a trapezoidal plane projection is extended to obtain a fitting envelope with a triangular plane projection. Figure 7 The fitting envelope corresponding to the probe 12 is shown, wherein the FOV angles of the probes 3, 6, 9, and 12 are all 65 degrees. Figure 4 The plane projections of the local envelopes corresponding to probes 4, 5, 10, and 11 are all rectangular, so a cone is selected for envelope fitting to obtain a fitting envelope. Taking the envelope fitting processing of the local envelopes corresponding to probes 10 and 11 as an example, a semicircular envelope with a FOV angle of 180 degrees is first constructed (corresponding to the temporary envelope above), such as Figure 8 As shown, the fitting envelopes corresponding to probes 1, 8, 9, and 12 are removed from the semicircular envelope, and the remaining is the envelope jointly constituted by the fitting envelopes corresponding to probes 10 and 11. Further, the intersection lines of the local envelopes corresponding to probes 10 and 11 can be evenly divided to obtain the fitting envelopes corresponding to probes 10 and 11.

[0115] After obtaining the fitting envelopes corresponding to the multiple ultrasonic radar sensors, simulation test processing can be performed in the simulation test scenario, which will be explained in the following steps.

[0116] Step 1) performing detection processing in a simulation test scene through a target sensor (referring to any ultrasonic radar sensor), and determining the object coordinates of the target object detected through the detection processing.

[0117] like Fig. 9 As shown in the figure, the surface of the target object detected by the target sensor includes four surfaces, namely Front (1), Rear (2), Right (3) and Left (4). The detection signal of the target sensor and the target object have two intersection surfaces, one of which is located at the Rear (2) surface and the other is located at the Right (3) surface. Considering that the intersection surface located at the Right (3) surface has a larger coverage area, the intersection surface located at the Right (3) surface is taken as the target intersection surface, and the object coordinates are calculated based on the four vertices of the target intersection surface.

[0118] The present application embodiment provides Fig.10 A flowchart of calculating the object coordinates is shown in FIG. First, the coordinates of the four vertices of the target intersecting surface are determined, and the coordinates can be expressed in the form of (x, y, z); then, the vertex with the smallest z coordinate is determined among the four vertices as vertex N0 (corresponding to the lowest vertex above); the absolute value of the difference between the z coordinates of other vertices and the z coordinate of vertex N0 is calculated, that is, abs|Zn-Z0|, where Zn represents the z coordinates of other vertices and Z0 represents the z coordinate of vertex N0; vertex N1 that makes the value of abs|Zn-Z0| the smallest is selected from other vertices; and the object line segment is constructed based on vertex N0 and vertex N1. At the same time, the absolute value of the difference between the largest z coordinate and the smallest z coordinate among the coordinates of the four vertices is calculated as the object height (corresponding to the object vertical coordinate above). Finally, the x coordinate of the point in the object line segment, the y coordinate of the point in the object line segment, and the object height (used as the z coordinate) are merged into the object coordinates. In this way, it is ensured that the target object can be accurately and comprehensively described by the obtained multiple object coordinates.

[0119] Step 2) Determine whether the object coordinates are within the fitting envelope corresponding to the target sensor.

[0120] like Fig.11As shown in FIG. 1 , when the object coordinates and the fitting envelope corresponding to the target sensor (multiple coordinates within the fitting envelope) are known, the inpolygon function can be used to determine whether the object coordinates are within the fitting envelope corresponding to the target sensor. When the object coordinates are within the fitting envelope corresponding to the target sensor, the object coordinates are output, that is, the process proceeds to step 3); when the object coordinates are not within the fitting envelope corresponding to the target sensor, the object coordinates are discarded.

[0121] Step 3) Use the object coordinates as input to the autonomous driving algorithm.

[0122] For example, the object coordinates are first converted from the world coordinate system to the sensor coordinate system, and then the object coordinates in the sensor coordinate system are sent to the domain controller, so that the domain controller calculates the autonomous driving strategy based on the object coordinates in the sensor coordinate system. The autonomous driving strategy is used to control the movement of the ego vehicle (the ego vehicle is virtual) in the simulation test scenario.

[0123] The embodiments of the present application can solve the dilemma that the simulation test software has a single envelope shape and cannot meet the complex envelope requirements. The embodiments of the present application can analyze the sensor installation position and design the FOV angle by self-partitioning, and then realize the accurate fitting of the real envelope of the real vehicle through envelope fitting processing, thereby finally achieving the purpose of accurate simulation.

[0124] The following is a description of an exemplary structure of a sensor-based simulation test device 255 provided in an embodiment of the present application implemented as a software module. In some embodiments, for example, Figure 2 As shown, the software modules stored in the sensor-based simulation test device 255 of the memory 250 may include: an acquisition module 2551, used to acquire a global envelope for a simulation test scenario; an area division module 2552, used to perform area division processing on the global envelope according to multiple sensors deployed to the vehicle, and obtain local envelopes corresponding to the multiple sensors; an envelope fitting module 2553, used to perform envelope fitting processing on the local envelopes corresponding to the multiple sensors according to the detection geometry of the sensor, and obtain the fitting envelopes corresponding to the multiple sensors; a simulation test module 2554, used to perform simulation test processing in a simulation test scenario according to the fitting envelopes corresponding to the multiple sensors.

[0125] In some embodiments, the detection geometry of the sensor includes a polygonal pyramid and a cone; the envelope fitting module 2553 is also used for: when the plane projection of the local envelope corresponding to the target sensor is a trapezoid or a convex quadrilateral whose two sets of opposite sides are not parallel, the local envelope corresponding to the target sensor is subjected to envelope fitting processing according to the polygonal pyramid to obtain the fitting envelope corresponding to the target sensor; when the plane projection of the local envelope corresponding to the target sensor is a rectangle, the local envelope corresponding to the target sensor is subjected to envelope fitting processing according to the cone to obtain the fitting envelope corresponding to the target sensor; wherein the target sensor represents any one of a plurality of sensors.

[0126] In some embodiments, the envelope fitting module 2553 is also used to: extend a set of opposite sides of the local envelope corresponding to the target sensor in the plane projection in the direction of the target sensor until the extended lines intersect to obtain a polygonal pyramid envelope with a plane projection of a triangle; and determine the polygonal pyramid envelope as the fitting envelope corresponding to the target sensor.

[0127] In some embodiments, the envelope fitting module 2553 is also used to: construct a temporary envelope that covers the local envelope corresponding to the target sensor and whose plane projection is a semicircle; delete the fitting envelopes corresponding to other sensors in the temporary envelope to obtain the remaining cone envelope; wherein the other sensors are different from the target sensor; and determine the cone envelope as the fitting envelope corresponding to the target sensor.

[0128] In some embodiments, the simulation test module 2554 is also used to: perform detection processing in a simulation test scene through a target sensor; determine the object coordinates of the target object detected through the detection processing; when the object coordinates are within the fitting envelope corresponding to the target sensor, perform autonomous driving processing according to the object coordinates; when the object coordinates are not within the fitting envelope corresponding to the target sensor, discard the object coordinates; wherein the target sensor represents any one of multiple sensors.

[0129] In some embodiments, the simulation test module 2554 is also used to: determine the intersecting surfaces covered by the detection signal of the target sensor among the multiple surfaces of the target object; determine the lowest vertex with the smallest vertical coordinate and the highest vertex with the largest vertical coordinate among the multiple vertices of the intersecting surfaces; determine the vertical coordinate difference between the highest vertex and the lowest vertex as the object vertical coordinate; construct an object line segment based on the lowest vertex and the vertex with the smallest vertical coordinate difference with the lowest vertex; merge the horizontal coordinate of the point in the object line segment, the vertical coordinate of the point in the object line segment, and the object vertical coordinate to obtain the object coordinate of the target object.

[0130] In some embodiments, the number of intersecting surfaces includes multiple; the simulation test module 2554 is also used to: determine the target intersection surface with the largest coverage area covered by the detection signal of the target sensor among the multiple intersecting surfaces; determine the lowest vertex with the smallest vertical coordinate and the highest vertex with the largest vertical coordinate among the multiple vertices of the target intersection surface.

[0131] The embodiment of the present application provides a computer program product or a computer program, which includes executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the executable instructions from the computer-readable storage medium, and the processor executes the executable instructions, so that the electronic device performs the sensor-based simulation test method described in the embodiment of the present application.

[0132] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the sensor-based simulation test method provided in the embodiment of the present application.

[0133] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0134] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0135] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0136] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0137] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A sensor-based simulation test method, characterized in that: include: Get the global envelope for simulating test scenarios; Performing region division processing on the global envelope according to a plurality of sensors deployed to the vehicle to obtain local envelopes corresponding to the plurality of sensors respectively; Perform envelope fitting processing on the local envelopes respectively corresponding to the multiple sensors according to the detection geometry of the sensors to obtain the fitting envelopes respectively corresponding to the multiple sensors; A simulation test process is performed in the simulation test scenario according to the fitting envelopes respectively corresponding to the multiple sensors.

2. The method according to claim 1, characterized in that: The detection geometry of the sensor includes a polygonal pyramid and a cone; performing envelope fitting processing on the local envelopes corresponding to the multiple sensors respectively according to the detection geometry of the sensor to obtain the fitting envelopes corresponding to the multiple sensors respectively includes: When the plane projection of the local envelope corresponding to the target sensor is a trapezoid or a convex quadrilateral whose two sets of opposite sides are not parallel, envelope fitting processing is performed on the local envelope corresponding to the target sensor according to a polygonal pyramid to obtain a fitting envelope corresponding to the target sensor; When the plane projection of the local envelope corresponding to the target sensor is a rectangle, performing envelope fitting processing on the local envelope corresponding to the target sensor according to a cone to obtain a fitting envelope corresponding to the target sensor; The target sensor refers to any one of the multiple sensors.

3. The method according to claim 2, characterized in that The performing envelope fitting processing on the local envelope corresponding to the target sensor according to the polygonal pyramid to obtain the fitting envelope corresponding to the target sensor includes: Extending a set of opposite sides of the local envelope corresponding to the target sensor in the plane projection in the direction of the target sensor until the extended lines intersect to obtain a polygonal pyramid envelope whose plane projection is a triangle; The polygonal pyramid envelope is determined as a fitting envelope corresponding to the target sensor.

4. The method according to claim 2, characterized in that: The step of performing envelope fitting processing on the local envelope corresponding to the target sensor according to the cone to obtain the fitting envelope corresponding to the target sensor includes: Constructing a temporary envelope that covers the local envelope corresponding to the target sensor and whose plane projection is a semicircle; Deleting the fitting envelopes corresponding to other sensors in the temporary envelope to obtain the remaining cone envelope; wherein the other sensors are different from the target sensor; The cone envelope is determined as a fitting envelope corresponding to the target sensor.

5. The method according to any one of claims 1 to 4, characterized in that: The performing simulation test processing in the simulation test scenario according to the fitting envelopes respectively corresponding to the multiple sensors includes: Performing detection processing in the simulation test scenario by means of a target sensor; determining object coordinates of a target object detected by the detection process; When the object coordinates are within the fitting envelope corresponding to the target sensor, performing autonomous driving processing according to the object coordinates; When the object coordinates are not within the fitting envelope corresponding to the target sensor, discarding the object coordinates; The target sensor refers to any one of the multiple sensors.

6. The method according to claim 5, characterized in that Determining the object coordinates of the target object detected by the detection process includes: determining, among the plurality of surfaces of the target object, an intersecting surface covered by the detection signal of the target sensor; Determine, among the plurality of vertices of the intersecting surface, a lowest vertex with the smallest vertical coordinate and a highest vertex with the largest vertical coordinate; Determine the vertical coordinate difference between the highest vertex and the lowest vertex as the object vertical coordinate; Constructing an object line segment according to the lowest vertex and the vertex with the smallest vertical coordinate difference from the lowest vertex; The horizontal coordinate of the point in the object line segment, the vertical coordinate of the point in the object line segment and the object vertical coordinate are combined to obtain the object coordinate of the target object.

7. The method according to claim 6, characterized in that The number of the intersecting surfaces includes a plurality; after determining the intersecting surface covered by the detection signal of the target sensor among the plurality of surfaces of the target object, the method further includes: Determine, among a plurality of intersecting surfaces, a target intersecting surface having a largest coverage area covered by the detection signal of the target sensor; The step of determining the lowest vertex with the smallest vertical coordinate and the highest vertex with the largest vertical coordinate among the multiple vertices of the intersecting surface comprises: A lowest vertex with the smallest vertical coordinate and a highest vertex with the largest vertical coordinate are determined among the multiple vertices of the target intersection surface.

8. A sensor-based simulation test device, characterized in that: include: An acquisition module, used to acquire a global envelope for simulating a test scenario; A region division module, used for performing region division processing on the global envelope according to a plurality of sensors deployed to the vehicle, to obtain local envelopes corresponding to the plurality of sensors respectively; An envelope fitting module, used for performing envelope fitting processing on the local envelopes respectively corresponding to the multiple sensors according to the detection geometry of the sensors, to obtain fitting envelopes respectively corresponding to the multiple sensors; The simulation test module is used to perform simulation test processing in the simulation test scenario according to the fitting envelopes respectively corresponding to the multiple sensors.

9. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the executable instructions stored in the memory.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the method described in any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The method comprises executable instructions for implementing the method according to any one of claims 1 to 7 when executed by a processor.

Images