A vehicle, a simulation method and device of a sensor

Abstract

The application provides a vehicle, a simulation method and device of a sensor, the method comprising: inputting position information and speed information of a first target vehicle relative to a simulation vehicle and road environment information of the simulation vehicle into a sensor model to obtain a sensor feature prediction value of the first target vehicle; wherein the sensor model is used for simulating a sensor in the simulation vehicle, the first target vehicle is a vehicle in a test environment of the simulation vehicle, the position information and speed information of the first target vehicle relative to the simulation vehicle and the road environment information of the simulation vehicle are determined according to the test environment, and the sensor model is obtained by training according to measurement data of the sensor and labeled road environment information; inputting the sensor feature prediction value of the first target vehicle into a decision module of the simulation vehicle to obtain a simulation decision result of the simulation vehicle; wherein the decision module is used for outputting a vehicle driving decision determined based on the sensor feature prediction value.

Description

Technical Field

[0001] This application relates to the field of intelligent connected vehicle technology, and in particular to a simulation method and apparatus for vehicles and sensors. Background Technology

[0002] Autonomous driving is a technology in which computer systems drive motor vehicles instead of humans. It includes functional modules such as environmental perception, location positioning, path planning, decision-making and control, and power systems. Environmental perception is achieved in two ways: through high-precision, low-dimensional sensors such as LiDAR and millimeter-wave radar, and through high-dimensional, low-precision sensors such as monocular / multi-view high-definition cameras.

[0003] To ensure the safety of autonomous driving, intelligent vehicles need to undergo extensive road testing to fully verify their safety. However, this incurs significant time and economic costs. Therefore, before conducting road tests, intelligent vehicles can be tested and verified through virtual simulation, improving testing efficiency and reducing costs. In intelligent vehicle simulation, sensor simulation is a crucial step, as the simulation data obtained from sensor simulation directly impacts the reliability of the vehicle simulation results.

[0004] However, current sensor testing processes are complex and lack real-time performance, making it difficult to meet the needs of real-time simulation for intelligent vehicles. Summary of the Invention

[0005] This application provides a vehicle and sensor simulation method and apparatus to improve the simulation effect of vehicles.

[0006] Firstly, this application provides a vehicle simulation method. This simulation method can be applied to a testing device and may include hardware devices that support running simulation software, such as personal computers, servers, in-vehicle mobile terminals, industrial control computers, embedded devices, etc. For example, the testing device can be implemented by a cloud server or virtual machine. The testing device can also be a chip that supports running simulation software. This testing device is used to test simulated vehicles and may include sensor models used to simulate sensors in the simulated vehicle. For example, the testing device may be a server for testing simulated vehicles or a chip on a server. The method may include:

[0007] The position and speed information of the first target vehicle relative to the simulated vehicle, as well as the road environment information of the simulated vehicle, are input into the sensor model to obtain the sensor feature prediction value of the first target vehicle. The sensor feature prediction value includes at least one of the following: radar cross section (RCS) prediction value and signal-noise ratio (SNR) prediction value. The first target vehicle is a vehicle in the test environment where the simulated vehicle is located. The position and speed information of the first target vehicle relative to the simulated vehicle, as well as the road environment information of the simulated vehicle, are determined based on the test environment. The sensor model is trained based on the sensor measurement data and labeled road environment information. The sensor feature prediction value of the first target vehicle is input into the decision module of the simulated vehicle to obtain the simulation decision result of the simulated vehicle. The decision module is used to output a vehicle driving decision determined based on the sensor feature prediction value.

[0008] In the above method, the sensor model can obtain predicted sensor features of the target object in the test environment, such as RCS and SNR predictions. This makes the relevant information input to the decision module for simulating the sensor's acquisition more closely resemble the output of a real millimeter-wave radar sensor, improving the realism of the sensor model's simulation. This allows the decision module to better simulate the possible decisions made by the vehicle based on the relevant information collected by the sensors in real-world scenarios, thereby improving the vehicle simulation effect. Furthermore, since the sensor model is trained based on the sensor's measurement data and labeled road environment information, it can output corresponding predicted sensor features based on different test environments (which can correspond to different road environment information), effectively improving vehicle simulation performance and the robustness of the simulation results.

[0009] One possible implementation is that the first target vehicle is a vehicle within the detection range of the sensor, determined from among the candidate vehicles based on the position and speed information of the candidate vehicles relative to the simulated vehicle; the position and speed information of the candidate vehicles relative to the simulated vehicle are determined based on the test environment, and the candidate vehicles are vehicles in the test environment where the simulated vehicle is located.

[0010] Using the above method, candidate vehicles can be screened based on vehicles within the detection range of the sensors of the simulated vehicle. This ensures that the determined first target vehicle can be any of the possible candidate vehicles within the sensor's detection range. Considering the possibility of occlusion between multiple first target vehicles relative to the simulated vehicle, the sensor can still collect measurement data from these multiple first target vehicles due to multipath effects. Therefore, the multipath effect of the sensor can be better simulated, allowing the sensor model to output predicted sensor feature values ​​for multiple first target vehicles in this scenario. This enables the sensor model to reflect the multipath effect of the sensor, improving the simulation effect.

[0011] One possible implementation is to determine that the predicted SNR value of the first target vehicle is greater than a visibility threshold.

[0012] Considering that sensors can determine whether the collected measurement data is a target object or noise by checking if the signal-to-noise ratio is greater than a preset threshold, the sensor may exhibit four different behaviors: when a target is present, it is correctly identified as having a target (this is called "detection"); when a target is present, it is incorrectly identified as not having a target (this is called "false negative"); when no target is present, it is correctly identified as not having a target (this is called "correct detection without detection"); and when no target is present, it is incorrectly identified as having a target (this is called "false alarm").

[0013] Therefore, introducing this method into the sensor model allows the model to simulate the physical characteristics that might lead to misjudgments of target objects based on the signal-to-noise ratio (SNR). In this application, the predicted SNR values ​​of candidate vehicles are filtered to determine whether a candidate vehicle is the first target vehicle. When the predicted SNR value of a candidate vehicle is greater than a visibility threshold, the candidate vehicle is determined to be the first target vehicle. When the predicted SNR value of a candidate vehicle is less than or equal to the visibility threshold, the candidate vehicle is determined to be noise by the sensor. This effectively reflects the physical characteristics that might cause sensor misjudgments, improving the sensor model's simulation performance.

[0014] In one possible implementation, the first target vehicle includes a first candidate vehicle and a second candidate vehicle; the sensor feature prediction value of the first target vehicle is determined based on the sensor feature prediction values ​​of the first candidate vehicle and the second candidate vehicle; the first candidate vehicle and the second candidate vehicle satisfy the following condition: the relative position of the first position to the second position is less than the first position threshold; the first position is the position of the first candidate target vehicle relative to the simulated vehicle, and the second position is the position of the second candidate target vehicle relative to the simulated vehicle.

[0015] Considering that when the sensor outputs measurement data for the target vehicle, it may mistakenly identify two or more vehicles as a single vehicle, for example, if the relative position of the first candidate vehicle and the second candidate vehicle is less than a first position threshold, it can be determined that the sensor will misidentify the first candidate vehicle and the second candidate vehicle as the first target vehicle.

[0016] Therefore, in this application, by using the above method, the sensor model outputs the first candidate vehicle and the second candidate vehicle as the first target vehicle when it determines that the relative position of the first position to the second position is less than the first position threshold. In this way, the physical characteristics of the sensor may not be able to distinguish between multiple candidate vehicles, thereby improving the sensor model's ability to simulate the sensor.

[0017] In one possible implementation, the first candidate vehicle and the second candidate vehicle further satisfy the following: the relative speed of the first speed to the second speed is less than a first speed threshold; the first speed is the speed of the first candidate target vehicle relative to the simulated vehicle, and the second speed is the speed of the second candidate target vehicle relative to the simulated vehicle.

[0018] Considering that when a sensor outputs measurement data for a target vehicle, it may output two or more vehicles as a single vehicle, this can be determined based on relative position and relative speed. Therefore, in this application, by determining that the relative position of the first candidate vehicle and the second candidate vehicle satisfies a first position threshold relative to a second position and a first speed threshold relative to a second speed, the first candidate vehicle and the second candidate vehicle can be output as the first target vehicle. This better simulates the physical characteristics that the sensor may not be able to distinguish between multiple candidate vehicles, improving the sensor model's simulation effect.

[0019] One possible implementation involves training the sensor model based on the sensor's measurement data and labeled road environment information. This includes: acquiring sensor measurement data; the measurement data includes: position information and speed information of a second target vehicle relative to the sensor, and sensor feature values ​​of the second target vehicle collected by the sensor; the sensor feature values ​​include: RCS measurement values ​​and SNR measurement values; the sensor is located in a measuring vehicle, and the second target vehicle is a vehicle near the measuring vehicle; training is performed based on the sensor's measurement data and the obtained labeling information to obtain the sensor model; the labeling information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, road environment information labeled when the sensor collected data, and vehicle information where the sensor is located; the input to the sensor model is the position information and speed information of the first target vehicle relative to the sensor, and the labeling information; the output of the sensor model is the predicted sensor feature values ​​of the first target vehicle.

[0020] Using the above method, the sensor model can be trained based on the sensor feature values ​​of the second target vehicle collected by the sensor, the position information and speed information relative to the sensor, and the labeled road environment information. This allows the trained sensor model to output the sensor prediction value of the target vehicle. The sensor prediction value is obtained by training based on the sensor feature values ​​of the second target vehicle collected by the sensor. Therefore, the sensor model can more closely approximate the actual measurement data output by the sensor. In addition, since the road environment information when the sensor collected the measurement data is also considered in the training samples, the sensor feature prediction value output by the sensor model can better reflect the sensor output under different road environment information, thereby improving the sensor model's simulation effect of the sensor.

[0021] Secondly, this application provides a sensor simulation method, including:

[0022] Acquire sensor measurement data; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature measurement values ​​of the second target vehicle collected by the sensor; the sensor feature measurement values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measuring vehicle; the second target vehicle is a vehicle near the measuring vehicle; train a sensor model based on the sensor measurement data and obtained annotation information; the sample input of the sensor model is the position information, speed information, and annotation information of the second target vehicle relative to the sensor, and the output of the sensor model is the sensor feature prediction value of the second target vehicle; the sensor feature prediction value of the second target vehicle includes at least one of the following: RCS prediction value and SNR prediction value; the annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located.

[0023] Using the above method, the sensor model can be trained based on the sensor feature values ​​of the second target vehicle collected by the sensor, the position information and speed information relative to the sensor, and the labeled road environment information. This allows the trained sensor model to output the sensor prediction value of the target vehicle. This sensor prediction value is obtained based on the sensor feature values ​​of the second target vehicle collected by the sensor. Therefore, the sensor model can more closely approximate the actual measurement data output by the sensor. In addition, since the road environment information when the sensor collected the measurement data is also considered in the training samples, the sensor feature prediction value output by the sensor model can better reflect the sensor output under different road environment information, improve the sensor model's simulation effect, and thus help improve the vehicle simulation effect.

[0024] Thirdly, this application provides a vehicle simulation device, comprising:

[0025] The sensor feature prediction module is used to input the position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle into the sensor model to obtain the sensor feature prediction value of the first target vehicle. The sensor feature prediction value includes at least one of the following: radar cross section (RCS) prediction value and signal-to-noise ratio (SNR) prediction value. The sensor model is used to simulate the sensors in the simulated vehicle. The first target vehicle is a vehicle in the test environment where the simulated vehicle is located. The position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle are determined based on the test environment. The sensor model is trained based on the sensor measurement data and labeled road environment information.

[0026] An output module is used to input the sensor feature prediction values ​​of the first target vehicle into the decision module of the simulated vehicle to obtain the simulation decision result of the simulated vehicle; wherein, the decision module is used to output the vehicle driving decision determined based on the sensor feature prediction values.

[0027] In one possible implementation, the device may further include:

[0028] The first determining module is used to determine the vehicle within the detection range of the sensor as the first target vehicle from among the candidate vehicles based on the position information and speed information of the candidate vehicle relative to the simulated vehicle; the position information and speed information of the candidate vehicle relative to the simulated vehicle are determined based on the test environment; the candidate vehicle is a vehicle in the test environment where the simulated vehicle is located.

[0029] In one possible implementation, the device may further include: a second determining module, configured to determine that the predicted SNR value of the first target vehicle is greater than a visible threshold.

[0030] One possible implementation further includes: a third determining module, configured to determine the sensor feature prediction value of the first target vehicle based on the sensor feature prediction value of the first candidate vehicle and the sensor feature prediction value of the second candidate vehicle; the first target vehicle includes the first candidate vehicle and the second candidate vehicle; the first candidate vehicle and the second candidate vehicle satisfy the following: the relative position of the first position to the second position is less than the first position threshold; the first position is the position of the first candidate target vehicle relative to the simulated vehicle, and the second position is the position of the second candidate target vehicle relative to the simulated vehicle.

[0031] In one possible implementation, the first candidate vehicle and the second candidate vehicle also satisfy the following:

[0032] The relative speed of the first speed to the second speed is less than the first speed threshold; the first speed is the speed of the first candidate target vehicle relative to the simulated vehicle, and the second speed is the speed of the second candidate target vehicle relative to the simulated vehicle.

[0033] In one possible implementation, the device further includes: a sensor model training module, the sensor model training module comprising:

[0034] An acquisition module is used to acquire measurement data from a sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature values ​​of the second target vehicle collected by the sensor; the sensor feature values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measurement vehicle, and the second target vehicle is a vehicle near the measurement vehicle;

[0035] The training module is used to train a sensor model based on the measurement data and the obtained annotation information of the sensor. The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information of the measuring vehicle. The input of the sensor model is the position information and speed information of the first target vehicle relative to the sensor and the annotation information. The output of the sensor model is the sensor feature prediction value of the first target vehicle.

[0036] Fourthly, this application provides a sensor simulation device, comprising:

[0037] An acquisition module is used to acquire measurement data from a sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature measurement values ​​of the second target vehicle collected by the sensor; the sensor feature measurement values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measurement vehicle; the second target vehicle is a vehicle near the measurement vehicle;

[0038] The training module is used to train a sensor model based on the measurement data and the obtained annotation information of the sensor. The sample input of the sensor model is the position information, speed information and annotation information of the second target vehicle relative to the sensor. The output of the sensor model is the sensor feature prediction value of the second target vehicle. The sensor feature prediction value of the second target vehicle includes at least one of the following: RCS prediction value and SNR prediction value.

[0039] The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located.

[0040] Fifthly, this application provides a vehicle simulation device, comprising: a processor and an interface circuit; wherein the processor is coupled to a memory through the interface circuit, and the processor is used to execute program code in the memory to implement the method described in the first aspect or any possible implementation thereof.

[0041] In a sixth aspect, this application provides a sensor simulation device, comprising: a processor and an interface circuit; wherein the processor is coupled to a memory through the interface circuit, and the processor is used to execute program code in the memory to implement the method described in the second aspect above.

[0042] In a seventh aspect, this application provides a computer-readable storage medium including computer instructions that, when executed by a processor, cause a vehicle simulation device to perform the method described in any one of the first aspects or the method described in the second aspect.

[0043] Eighthly, this application provides a computer program product that, when run on a processor, causes the vehicle simulation device to perform the method described in any of the first aspects or the method described in the second aspect.

[0044] Ninthly, embodiments of this application provide a vehicle-to-everything (V2X) communication system, which includes an in-vehicle system and a device as described in the third or fourth aspect, wherein the in-vehicle system is communicatively connected to the device.

[0045] In a tenth aspect, embodiments of this application provide a chip system including a processor for calling a computer program or computer instructions stored in a memory, such that the processor performs the method as described in any possible implementation of the first or second aspect.

[0046] In one possible implementation, the processor is coupled to the memory via an interface.

[0047] In one possible implementation, the chip system also includes a memory that stores computer programs or computer instructions.

[0048] This application also provides a processor for calling a computer program or computer instructions stored in a memory to cause the processor to perform the method as described in any possible implementation of the first or second aspect.

[0049] Furthermore, the technical effects of any of the implementation methods in the third to tenth aspects can be found in the technical effects of the different implementation methods in the first to second aspects, and will not be repeated here. Attached Figure Description

[0050] Figure 1a A schematic diagram of a vehicle system architecture provided in this application embodiment;

[0051] Figure 1b A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0052] Figure 1c A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0053] Figure 2 A schematic diagram of the principle of a radar sensor provided in an embodiment of this application;

[0054] Figure 3a This is a flowchart illustrating a vehicle simulation method.

[0055] Figure 3b A schematic diagram of the test environment for a vehicle simulation method provided in an embodiment of this application;

[0056] Figure 3c This is a schematic diagram of a vehicle occlusion scenario provided in an embodiment of this application;

[0057] Figure 4a This is a schematic diagram of a scenario for collecting measurement data from a vehicle, provided in an embodiment of this application.

[0058] Figure 4b A schematic flowchart illustrating a sensor simulation method provided in an embodiment of this application;

[0059] Figure 4c This is a schematic diagram of measurement data collected by a vehicle, provided in an embodiment of this application.

[0060] Figure 4d A schematic diagram illustrating a sensor simulation method provided in an embodiment of this application;

[0061] Figure 4e A schematic diagram illustrating a sensor simulation method provided in an embodiment of this application;

[0062] Figure 5a A schematic diagram of a simulated vehicle structure provided in an embodiment of this application;

[0063] Figure 5b A schematic flowchart illustrating a vehicle simulation method provided in this application embodiment;

[0064] Figure 6a A schematic diagram illustrating the detection range of a vehicle sensor provided in an embodiment of this application;

[0065] Figure 6b This is a schematic diagram illustrating the determination of a target object according to an embodiment of this application;

[0066] Figures 7a-7d This is a schematic diagram illustrating the determination of a target object according to an embodiment of this application;

[0067] Figure 8a A schematic diagram of a simulated vehicle structure provided in an embodiment of this application;

[0068] Figure 8b A schematic flowchart illustrating a vehicle simulation method provided in this application embodiment;

[0069] Figure 9 A schematic diagram of the structure of a vehicle simulation device provided in an embodiment of this application;

[0070] Figure 10 A schematic diagram of the structure of a vehicle simulation device provided in an embodiment of this application;

[0071] Figure 11 A schematic diagram of the structure of a sensor simulation device provided in an embodiment of this application;

[0072] Figure 12 This is a schematic diagram of the structure of a sensor simulation device provided in an embodiment of this application. Detailed Implementation

[0073] The terms "first," "second," etc., used in the specification, embodiments, claims, and drawings of this application are for distinguishing purposes only and should not be construed as indicating or implying relative importance or order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0074] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0075] Figure 1aThis is an exemplary functional block diagram of a vehicle 100 according to an embodiment of this application. In one embodiment, the vehicle 100 can be configured in a fully or partially autonomous driving mode. For example, while in autonomous driving mode, the vehicle 100 can simultaneously control itself and determine the current state of the vehicle and its surrounding environment through human intervention, determine the possible behaviors of at least one other vehicle in the surrounding environment, determine the confidence level corresponding to the probability of that other vehicle performing the possible behavior, and control the vehicle 100 based on the determined information. When the vehicle 100 is in autonomous driving mode, the vehicle 100 can be set to operate without human interaction.

[0076] like Figure 1a As shown, components coupled to or included in vehicle 100 may include a propulsion system 110, a sensor system 120, a control system 130, peripheral devices 140, a power supply 150, a computer system 160, and a user interface 170. Components of vehicle 100 may be configured to operate in a manner interconnected with each other and / or with other components coupled to the respective systems. For example, power supply 150 may provide power to all components of vehicle 100. Computer system 160 may be configured to receive data from and control the propulsion system 110, sensor system 120, control system 130, and peripheral devices 140. Computer system 160 may also be configured to generate an image display on user interface 170 and receive input from user interface 170.

[0077] It should be noted that in other examples, vehicle 100 may include more, fewer, or different systems, and each system may include more, fewer, or different components. Furthermore, the systems and components shown can be combined or divided in any manner, and this application does not impose any specific limitations on this.

[0078] The propulsion system 110 can provide power for the movement of the vehicle 100. For example... Figure 1a As shown, the propulsion system 110 may include an engine / motor 114, an energy source 113, a transmission 112, and wheels / tires 111. Additionally, the propulsion system 110 may additionally or alternatively include components other than those shown. Figure 1a Other components besides those shown. This application does not specifically limit this.

[0079] The sensor system 120 may include several sensors for sensing information about the environment in which the vehicle 100 is located. For example... Figure 1aAs shown, the sensor system 120 includes sensors such as a Global Positioning System (GPS) 126, an Inertial Measurement Unit (IMU) 125, a lidar 122, a camera sensor 123, a millimeter-wave radar 124, and a brake 121 for modifying the position and / or orientation of the sensors. The millimeter-wave radar 124 can use radio signals to sense targets in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing targets, the millimeter-wave radar 124 can also be used to sense the speed and / or direction of travel of the targets. The lidar 122 can use laser light to sense targets in the environment in which the vehicle 100 is located. In some embodiments, the lidar 122 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. The camera sensor 123 can be used to capture multiple images of the surrounding environment of the vehicle 100. The camera sensor 123 can be a still camera or a video camera.

[0080] GPS 126 can be any sensor used to estimate the geographic location of vehicle 100. For this purpose, GPS 126 may include a transceiver that estimates the position of vehicle 100 relative to the Earth based on satellite positioning data. In the example, computer system 160 can be used to combine map data with GPS 126 to estimate the road traveled by vehicle 100. IMU 125 can be used to sense changes in the position and orientation of vehicle 100 based on inertial acceleration and any combination thereof. In some examples, the combination of sensors in IMU 125 may include, for example, an accelerometer and a gyroscope. Other combinations of sensors in IMU 125 are also possible.

[0081] The sensor system 120 may also include sensors from the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a critical function for the safe operation of the vehicle 100. The sensor system 120 may also include other sensors. This application does not specifically limit this.

[0082] Control system 130 controls the operation of vehicle 100 and its components. Control system 130 may include various elements, including a steering unit 136, a throttle 135, a braking unit 134, a sensor fusion algorithm 133, a computer vision system 132, a route control system 131, and an obstacle avoidance system 137. Steering system 136 is operable to adjust the forward direction of vehicle 100. For example, in one embodiment, it may be a steering wheel system. Throttle 135 controls the operating speed of engine 114 and thus the speed of vehicle 100. Control system 130 may additionally or alternatively include, in addition to... Figure 1a Other components besides those shown. This application does not specifically limit this.

[0083] Braking unit 134 is used to control the deceleration of vehicle 100. Braking unit 134 may use friction to slow down wheels 111. In other embodiments, braking unit 134 may convert the kinetic energy of wheels 111 into electrical current. Braking unit 134 may also take other forms to slow down the rotational speed of wheels 111 to control the speed of vehicle 100. Computer vision system 132 may operate to process and analyze images captured by camera sensor 123 to identify targets and / or features in the environment surrounding vehicle 100. The targets and / or features may include traffic signals, road boundaries, and obstacles. Computer vision system 132 may use target recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, computer vision system 132 may be used to map the environment, track targets, estimate the speed of targets, etc. Route control system 131 is used to determine the driving route of vehicle 100. In some embodiments, route control system 131 may combine data from sensor system 120, GPS 126, and one or more predetermined maps to determine the driving route of vehicle 100. The obstacle avoidance system 137 is used to identify, assess, and avoid or otherwise traverse potential obstacles in the environment of the vehicle 100. Of course, in one instance, the control system 130 may add or alternatively include components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0084] Peripheral device 140 can be configured to allow vehicle 100 to interact with external sensors, other vehicles, and / or users. For this purpose, peripheral device 140 may include, for example, a wireless communication system 144, a touchscreen 143, a microphone 142, and / or a speaker 141. Peripheral device 140 may additionally or alternatively include, in addition to... Figure 1a Other components besides those shown. This application does not specifically limit this.

[0085] In some embodiments, peripheral device 140 provides a means for a user of vehicle 100 to interact with user interface 170. For example, touchscreen 143 may provide information to a user of vehicle 100. User interface 170 may also operate touchscreen 143 to receive user input. In other cases, peripheral device 140 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 142 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 100. Similarly, speaker 141 may output audio to a user of vehicle 100.

[0086] The wireless communication system 144 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 144 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. The wireless communication system 144 can communicate with a wireless local area network (WLAN) using wireless fidelity (WiFi). In some embodiments, the wireless communication system 144 can communicate directly with devices using wireless protocols such as infrared links, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, are also possible. For example, the wireless communication system 144 may include one or more dedicated short range communications (DSRC) devices that can enable public and / or private data communication between vehicles and / or roadside stations.

[0087] Power source 150 can be configured to provide power to some or all of the components of vehicle 100. For this purpose, power source 150 may include, for example, rechargeable lithium-ion or lead-acid batteries. In some examples, one or more battery packs may be configured to provide power. Other power materials and configurations are also possible. In some examples, power source 150 and energy source 113 may be implemented together, as in some fully electric vehicles. Components of vehicle 100 can be configured to operate in a manner interconnected with other components within and / or outside their respective systems. For this purpose, the components and systems of vehicle 100 can be communicatively linked together via system buses, networks, and / or other connectivity mechanisms.

[0088] Some or all of the functions of vehicle 100 are controlled by computer system 160. Computer system 160 may include at least one processor 161, which executes instructions 1631 stored in a non-transitory computer-readable medium such as memory 163. Computer system 160 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0089] Processor 161 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a special-purpose device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1a The processor, memory, and other elements of the computer system 160 within the same block are functionally illustrated; however, those skilled in the art will understand that the processor, computer, or memory may or may not be stored in the same physical enclosure. For example, memory may be a hard disk drive or other storage media located in an enclosure different from that of computer system 160. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0090] In the various aspects described herein, the processor may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0091] In some embodiments, memory 163 may contain instructions 1631 (e.g., program logic) that can be executed by processor 161 to perform various functions of vehicle 100, including those described above. Memory 214 may also contain additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the propulsion system 110, sensor system 120, control system 130, and peripheral devices 140.

[0092] In addition to instruction 1631, memory 163 may also store data such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information can be used by vehicle 100 and computer system 160 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0093] User interface 170 is used to provide information to or receive information from a user of vehicle 100. Optionally, user interface 170 may include one or more input / output devices within a set of peripheral devices 140, such as wireless communication system 144, touch screen 143, microphone 142, and speaker 141.

[0094] Computer system 160 can control the functions of vehicle 100 based on input received from various subsystems (e.g., propulsion system 110, sensor system 120, and control system 130) and from user interface 170. For example, computer system 160 can utilize input from control system 130 to control steering unit 136 to avoid obstacles detected by sensor system 120 and obstacle avoidance system 137. In some embodiments, computer system 160 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0095] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 163 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0096] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1a This should not be construed as a limitation on the embodiments of this application.

[0097] Autonomous vehicles traveling on roads, such as vehicle 100 above, can identify targets in their surrounding environment to determine adjustments to their current speed. These targets can be other vehicles, traffic control equipment, or other types of targets. In some examples, each identified target can be considered independently, and based on the target's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0098] Optionally, the autonomous vehicle 100 or the computing device associated with the autonomous vehicle 100 (such as...) Figure 1aThe computer system 160, computer vision system 132, and memory 163 can predict the behavior of the identified target based on the characteristics of the identified target and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified target depends on the behavior of each other, so all identified targets can also be considered together to predict the behavior of a single identified target. The vehicle 100 can adjust its speed based on the predicted behavior of the identified targets. In other words, the autonomous vehicle can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the targets. In this process, other factors can also be considered in determining the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road, the curvature of the road, the proximity of static and dynamic targets, etc.

[0099] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from targets near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0100] The aforementioned vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0101] Furthermore, it should also be noted that the radar system described in this application embodiment can be applied to a variety of fields. For example, the radar system in this application embodiment includes, but is not limited to, vehicle-mounted radar, roadside traffic radar, and drone radar.

[0102] The sensor system will be described in detail below.

[0103] Automotive sensors can be divided into two main categories based on their sensing methods: passive sensing sensors and active sensing sensors.

[0104] Among them, passive sensing sensors rely on radiation information from the external environment.

[0105] For example, a typical passive sensing sensor is a camera. A camera's sensing does not involve emitting and receiving energy waves; the accuracy of its sensing results mainly depends on image processing and classification algorithms.

[0106] Camera sensor 123 may include any camera (e.g., a still camera, video camera, etc.) for acquiring images of the environment in which vehicle 100 is located. For this purpose, camera sensor 123 may be configured to detect visible light, or may be configured to detect light from other parts of the spectrum (such as infrared or ultraviolet light). Other types of camera sensor 123 are also possible. Camera sensor 123 may be a two-dimensional detector, or may have three-dimensional spatial range detection capabilities. In some examples, camera sensor 123 may be, for example, a distance detector configured to generate a two-dimensional image indicating the distance from camera sensor 123 to several points in the environment. For this purpose, camera sensor 123 may use one or more distance detection techniques. For example, camera sensor 123 may be configured to use structured light technology, wherein vehicle 100 illuminates objects in the environment using a predetermined light pattern, such as a grid or checkerboard pattern, and uses camera sensor 123 to detect reflections from the predetermined light pattern from the objects. Based on the distortion in the reflected light pattern, vehicle 100 may be configured to detect the distance to points on the objects. The predetermined light pattern may include infrared light or light of other wavelengths. Camera sensor 123 may include any camera (e.g., a still camera, video camera, etc.) for acquiring images of the environment in which vehicle 100 is located. In some examples, camera sensor 123 may be a distance detector configured to generate a two-dimensional image indicating the distances from camera sensor 123 to several points in the environment. For this purpose, camera sensor 123 may use one or more distance detection techniques. When the camera sensor detects the presence of a target in the image sensing area, it transmits the image information to a processing module for further processing.

[0107] The camera sensor 123 can be one or more of the following camera sensors, for example: 1) an infrared camera sensor (infrared radiation-red green blue image sensor, IR-RGB image sensor), which uses a CCD unit (charge-coupled device) or a standard CMOS unit (complementary metal-oxide semiconductor), and filters the light through a filter to allow only the color wavelength band and the set infrared wavelength band to pass through. In the image signal processor, the IR (infrared radiation) image data stream and the RGB (red green blue, the three primary colors) image data stream are separated. The IR image data stream is the image data stream obtained in a low-light environment. The two separated image data streams are used for other application processing. 2) a visible light camera sensor, which uses a CCD unit (charge-coupled device) or a standard CMOS unit (complementary metal-oxide semiconductor) to obtain visible light data images.

[0108] Active sensing sensors perceive the environment by actively emitting energy waves. For example, a radar sensor can be an active sensing sensor. Onboard radar sensors transmit detection signals (electromagnetic waves) through antennas and receive signals reflected from targets. They amplify and down-convert the reflected signals to obtain information such as the relative distance, relative speed, and angle between the vehicle and the target. Based on this information, they perform target tracking and classification, and after making reasonable decisions, can achieve functions such as obstacle measurement, collision prediction, and adaptive cruise control. For instance, after tracking and classifying targets based on the information obtained, radar sensors can inform or warn the driver through various means such as sound, light, and touch, or intervene actively in the vehicle in a timely manner. This effectively reduces driving difficulty, driver burden, and the incidence of accidents, thereby ensuring driving safety and comfort, and is therefore widely used in the automotive industry.

[0109] Radar sensors can be classified into long-range radar (LRR), medium-range radar (MRR), and short-range radar (SRR) based on their different measurement ranges.

[0110] The LRR (Long-Range Detector) features distance measurement and collision avoidance capabilities, and is widely used in adaptive cruise control (ACC), forward collision warning (FCW), and automatic emergency braking (AEB). For example, installing the LRR at the center of the front bumper with an azimuth angle of 0°, setting the elevation angle to 1.5° when the height is below 50cm and 0° when the height exceeds 50cm, allows for moving target detection up to 150 meters for trucks, 100 meters for cars, and 60 meters for pedestrians. The LRR's ACC, FCW, and AEB functions provide significant safety alerts when drivers are distracted, fatigued, or using mobile phones and fail to notice the road ahead.

[0111] MRR and SRR feature blind spot detection (BSD), lane change assist (LCA), rear cross traffic alert (RCTA), exit assist function (EAF), and forward cross traffic alert (FCTA), accurately detecting targets within a certain range in front of, behind, to the sides of the vehicle. As a typical application in ADAS systems, SRR effectively reduces the risk of accidents caused by poor visibility in adverse weather conditions such as nighttime, fog, and heavy rain, and avoids potential collisions with adjacent lanes and blind spots during lane changes.

[0112] Different application scenarios have different requirements for radar detection range. LRR, MRR and SRR all play important roles in Advanced Driving Assistant Systems (ADAS).

[0113] The following example uses a specific radar sensor to illustrate this.

[0114] An ultrasonic radar sensor uses ultrasound, which refers to mechanical waves with frequencies higher than 20 kHz. To use ultrasound as a detection method, it is necessary to generate and receive ultrasonic waves. The device that performs this function is an ultrasonic radar. Ultrasonic radar has a transmitter and a receiver, but a single ultrasonic radar can also perform the dual function of transmitting and receiving sound waves. Ultrasonic radar utilizes the piezoelectric effect to convert electrical energy and ultrasonic waves into each other; that is, when transmitting ultrasonic waves, electrical energy is converted into ultrasonic waves; and when receiving the echo, the ultrasonic vibration is converted into an electrical signal.

[0115] Millimeter-wave radar sensors are radars that operate in the millimeter-wave band. Millimeter waves typically refer to the 30–300 GHz frequency range (wavelength 1–10 mm). Since the wavelength of millimeter waves falls between microwaves and centimeter waves, millimeter-wave radar combines some advantages of microwave radar and electro-optical radar. It features small size, light weight, and high spatial resolution, and has strong penetration capabilities through fog, smoke, and dust, making it widely used in navigation systems for vehicles, aircraft, and other applications. Millimeter-wave radar sensors provide depth information, allowing them to determine the target's distance. Furthermore, due to the significant Doppler effect, millimeter-wave radar sensors are highly sensitive to velocity, allowing direct acquisition of target velocity. The target's velocity can be extracted by detecting its Doppler frequency shift. Currently, the two mainstream automotive millimeter-wave radar application frequency bands are 24GHz and 77GHz, respectively. The former has a wavelength of about 1.25cm and is mainly used for short-range perception, such as the vehicle's surrounding environment, blind spots, parking assistance, lane change assistance, etc.; the latter has a wavelength of about 4mm and is used for medium and long-range measurement, such as automatic following, adaptive cruise control (ACC), emergency braking (AEB), etc.

[0116] A lidar sensor can be viewed as an object detection system that uses light sensing to detect objects in the environment in which the vehicle 100 is located. Lidar, operating in the infrared and visible light bands, is a type of radar that uses laser light as its working beam. The working principle of lidar is to emit a detection signal (laser beam) towards the target, then compare the received signal reflected back from the target (target echo) with the emitted signal. After appropriate processing, information about the target can be obtained, such as target distance, azimuth, altitude, speed, attitude, and even shape. Typically, lidar sensors utilize optical remote sensing technology to measure the distance to a target or other attributes of the target by illuminating it with light. As an example, a lidar sensor may include a laser source and / or laser scanner configured to emit laser pulses, and a detector for receiving the reflections of the laser pulses. For instance, a lidar sensor may include a laser rangefinder reflected by a rotating mirror, scanning the laser in one or two dimensions around a digitized scene to acquire distance measurements at specified angular intervals. In the example, a lidar sensor may include components such as a light (e.g., laser) source, a scanner and optical system, photodetectors and receiver electronics, and a positioning and navigation system. A lidar sensor determines the distance to an object by scanning the laser light reflected back from it, creating a 3D environmental map with centimeter-level accuracy. A lidar sensor can be viewed as an object detection system that uses light to illuminate a target to measure its distance.

[0117] like Figure 1b The diagram illustrates one possible application scenario provided by this application. In this scenario, the radar sensor can be installed on a vehicle. For example, the sensor in this application can be applied to advanced driver assistance systems (ADAS) (e.g., autonomous driving), robots, drones, connected vehicles, security monitoring, and other fields. In another scenario, the radar sensor can be installed on mobile devices. For instance, the radar sensor can be installed on motor vehicles (e.g., driverless cars, intelligent cars, electric cars, digital cars, etc.) as vehicle-mounted radar; or, for example, the radar can be installed on a drone as airborne radar, and so on. Figure 1a Examples of vehicles, such as Figure 1bAs shown, the radar sensor deployed at the front of the vehicle can sense a fan-shaped area as indicated by the solid line box. This fan-shaped area can be considered the radar sensing area. When the radar sensor detects a target within the radar sensing area, it transmits the radar signal information to the processing module for further processing. After receiving the information from the radar sensor, the processing module outputs the target radar's measurement information (e.g., the target object's relative distance, angle, and relative speed). It should be noted that the processing module here can be either a computer independent of the radar sensor or a software module within a computer, such as the processing module in computer system 160, or a computer or a software module deployed within the radar sensor; no limitation is made here.

[0118] As can be seen, by mounting the aforementioned sensors on the vehicle body, it is possible to acquire measurement information such as the vehicle's latitude and longitude, speed, orientation, and distance to surrounding objects in real time or periodically. This measurement information can then be used to achieve assisted driving or autonomous driving. For example, latitude and longitude can be used to determine the vehicle's position, speed and orientation can be used to determine the vehicle's driving direction and purpose over a future period of time, or the distance to surrounding objects can be used to determine the number and density of obstacles around the vehicle.

[0119] like Figure 1c The image shows another possible application scenario provided by this application. The radar sensor involved in this application can also be installed on fixed devices, such as roadside units (RSUs), rooftops, or base stations. For example, as shown... Figure 1c Radars 1, 2, 3, and 4 are shown. In scenarios where radars are installed on fixed devices, they require the assistance of other devices within the fixed device to determine their current position and orientation information, ensuring the availability of measurement data. For example, the fixed device may also include a Global Positioning System (GPS) and an Inertial Measurement Unit (IMU). The radar can combine the measurement data from the GPS and IMU to obtain characteristic quantities such as the target's position and velocity. For instance, the radar can obtain the fixed device's geographical location information through the GPS device and record the fixed device's attitude and orientation information through the IMU device. After determining the distance to the target based on the echo signal and the emitted laser beam, the target's measurement point can be converted from a relative coordinate system to a position point in an absolute coordinate system using at least one of the geographical location information provided by the GPS device or the attitude and orientation information provided by the IMU device, thus obtaining the target's geographical location information. This allows the radar to be used in fixed devices.

[0120] The radar sensor used in this application may be a lidar, a microwave radar, or a millimeter-wave radar; the embodiments of this application do not limit this.

[0121] For ease of explanation, the following description uses lidar as an example to illustrate the working process of radar sensors. It should be noted that the electromagnetic waves emitted by lidar are called laser beams, the electromagnetic waves emitted by microwave radar are called microwaves, and the laser beams emitted by millimeter-wave radar are called millimeter waves. In other words, lidar can be replaced by millimeter-wave radar, and electromagnetic waves can be replaced by millimeter waves; similarly, lidar can also be replaced by microwave radar, and electromagnetic waves can be replaced by microwaves.

[0122] It should be noted that this application does not limit the number of radar sensors or targets included in each scenario. For example, a scenario may include multiple radar sensors mounted on sensors and movable targets, and this application can also be applied to other possible scenarios. For example... Figure 1c The examples shown are vehicle-road cooperative (or intelligent vehicle-road cooperative system) scenarios. Another example is the automated guided vehicle (AGV) scenario, where an AGV is a transport vehicle equipped with electromagnetic or optical automatic navigation devices, capable of traveling along a predetermined navigation path, and possessing safety protection and various transfer functions. Another example is remote interaction and real-world scenario reproduction, such as remote medical care or training, interactive games (e.g., multiple people playing games, training, or participating in other activities in a virtual environment), or training in dangerous scenarios. Yet another example is facial recognition scenarios. These are not all listed here.

[0123] like Figure 2 The diagram shown illustrates the principle of a radar target detection system provided in this application. The radar may include a transmitter and a receiver. The transmitter emits an electromagnetic wave energy beam. The electromagnetic wave is transmitted to an antenna via a transmit / receive switch. The antenna then transmits the electromagnetic wave into the air along a certain direction and angle. If a target exists within a certain distance along the transmission direction of the electromagnetic wave energy beam, the electromagnetic wave energy beam is reflected by the target. When the electromagnetic wave encounters the target, a portion of its energy is reflected and received by the antenna of the millimeter-wave radar, and then transmitted to the receiver via the transmit / receive switch. A target exists along the transmission direction of the electromagnetic wave energy beam (e.g., ...). Figure 1a Taking a car as an example, the electromagnetic wave energy beam emitted by the transmitter is reflected off the target's surface after reaching it. The reflected signal returns to the receiver as an echo signal. The receiver uses the received echo signal and the emitted electromagnetic wave energy beam to determine information related to the target, such as the distance to the target and the target's point cloud density. The radar sensor emits an electromagnetic wave energy beam through the transmitter, which is further processed by a signal processor to obtain the relative distance, angle, and relative speed of the target object.

[0124] The following uses a millimeter-wave radar sensor as an example to illustrate the implementation of a radar sensor. For instance, a millimeter-wave radar sensor may include devices such as an oscillator, transmitting antenna, receiving antenna, mixer, processor, and controller. Specific steps may include:

[0125] Step 1: The waveform generator in the radar generates the transmission signal, which is then transmitted through the transmit antenna.

[0126] For example, an oscillator generates a radar signal whose frequency increases linearly with time; this radar signal is typically a frequency-modulated continuous wave. Radar detection devices generally transmit radar signals for multiple frequency sweep cycles over a continuous period of time. Here, a frequency sweep cycle refers to the period for transmitting a complete radar signal waveform. At the beginning of a transmission cycle, the radar detection device transmits a radar signal at a frequency called the initial frequency of the radar detection device. The transmission frequency of the radar detection device varies within the transmission cycle based on this initial frequency.

[0127] A portion of the radar signal is output to a mixer via a directional coupler as a local oscillator signal, while another portion is transmitted through the transmitting antenna. The transmitted signal is typically a linear frequency modulated (LFM) signal with a carrier frequency. T The expression for (t) can be:

[0128]

[0129] Where f T Indicates the carrier frequency, B sw T represents the bandwidth of the transmitted signal. CPI Indicates the duration of the transmitted signal.

[0130] Step 2: After the transmitted signal is reflected by an obstacle, it is received by the receiving antenna. For example, the receiving antenna receives the radar signal reflected back after the transmitted radar signal encounters an object in front of the vehicle. The received signal is a delayed version of the transmitted signal, denoted as s. R The expression for (t) is:

[0131] s R (t)=s T [t-τ(t)] (2)

[0132] Where τ(t) represents the delay between the transmitted signal being sent from the transmitting antenna, reflected by an obstacle, and received by the receiving antenna.

[0133] Step 3: Mix / down-convert the delayed signal of the transmitted signal with the transmitted signal, and then obtain the received signal through sampling.

[0134] For example, a mixer mixes the received radar signal with the local oscillator signal to obtain the intermediate frequency (IF) signal. Specifically, a portion of the frequency-modulated continuous wave signal generated by the oscillator is used as the local oscillator signal, and the portion is transmitted as the transmitted signal through the transmitting antenna. The reflected signal of the transmitted signal received by the receiving antenna is mixed with the local oscillator signal to obtain the IF signal. The IF signal contains information such as the relative distance, velocity, and angle between the target object and the radar system. After passing through a low-pass filter and amplification, the IF signal is sent to the processor. The processor processes the received signal, typically performing a fast Fourier transform and spectral analysis to obtain information such as the target object's distance, velocity, and angle relative to the radar system.

[0135] The distance between the target (ground object) and the radar can be determined by the difference between the transmission time of the transmitted signal and the reception time of the echo scattering of different ground objects, thereby determining the location of the target.

[0136] Among these, position information can be the target object's position relative to the current radar, velocity information can be the target object's velocity relative to the current radar, and angle information can be the target object's angle relative to the current radar. Furthermore, the frequency of the intermediate frequency signal is called the intermediate frequency (IF).

[0137] Step 4: The processor can output the obtained information to the controller to control the vehicle's behavior.

[0138] In the simulation testing of intelligent vehicles, a crucial part is verifying the decision-making and control algorithms for autonomous driving. This includes verifying the vehicle's lane-changing capabilities and identifying whether a vehicle is too close to the front. The verification process requires constructing different scenarios, and in each scenario, verifying whether the vehicle can achieve the corresponding autonomous driving decision-making and control capabilities.

[0139] Therefore, in the process of verifying the decision-making and control of autonomous driving, it is necessary to obtain information such as the vehicle speed, position, distance, and azimuth of the target vehicle relative to the vehicle, as determined by the sensors, as input parameters in the autonomous driving decision-making and control simulation. That is, in the sensor simulation process, the sensor model can take traffic participants in the test environment as input, and the output parameters of this sensor model can be the relative distance, relative speed, and angle of detectable objects within the sensor's perception range (determined based on a geometric occlusion filtering method). Thus, the output parameters of this sensor model can serve as the input parameters for the corresponding sensor modules required in the autonomous driving decision-making and control simulation.

[0140] Since the primary focus is on validating decision-making and control algorithms for autonomous driving, one possible approach to constructing the sensor model is to use information such as the perceived targets constructed in the scene, their speed relative to the vehicle, their relative position, distance, and azimuth angle, as well as other relevant data, as the output of the sensor module to the perceived targets. For example... Figure 3a As shown, the specific process may include:

[0141] Step 301: Identify traffic participants in the test environment.

[0142] The testing environment can be determined based on the specific testing scenarios. For example, ... Figure 3b As shown, it includes: simulated vehicles (including sensors to be simulated), other vehicles, non-motorized vehicles, pedestrians, road environment, traffic environment, buildings, bridges, roadblocks, etc.

[0143] Step 302: Use the parameters of traffic participants in the test environment as input parameters for the sensor model.

[0144] Traffic participants can include: vehicles, pedestrians, roads, roadblocks, etc.

[0145] The parameters of traffic participants can include modeling data such as location, speed, and size.

[0146] Step 303: Use the geometric occlusion method to filter the perceptible targets of the sensors on the simulated vehicle.

[0147] One possible implementation is to determine the maximum ranging distance of the sensor based on its specific model, thereby allowing the radar sensor's detection range to be determined. This data can be based on the sensor's factory specifications or derived from experience; no limitation is made here.

[0148] The maximum ranging distance of the radar detection device, or the maximum detection range of the radar detection device, is a parameter related to the configuration of the radar detection device (e.g., related to the factory settings of the radar detection device). For example, if the radar detection device is a radar, the maximum ranging distance of a long-range adaptive cruise control (ACC) radar is 142m, and the maximum ranging distance of a medium-range radar is 70-150m.

[0149] For example, such as Figure 3b As shown, the radar sensor deployed at the front of vehicle 1 can detect a fan-shaped area as indicated by the solid line box, which is the radar's detection range. Vehicles within this detection range can be considered as detectable targets by the sensor.

[0150] In some embodiments, vehicles that are obscured can be excluded based on the geometric occlusion relationship between them.

[0151] For example, such as Figure 3c As shown, vehicle 1 is the vehicle under test, and vehicles 2 and 3 are vehicles in front of vehicle 1. Based on the geometrical positional relationship between vehicles 2, 3, and vehicle 1, it can be determined that vehicle 3 is occluded by vehicle 2. Therefore, vehicle 3 can be removed. Thus, vehicle 2 can be determined as a perceptible target of vehicle 1.

[0152] Step 304: Determine the output parameters of the sensor model based on the parameters of the sensor's perceptible targets.

[0153] The parameters of the sensor's perceptible target can be determined based on the parameters of traffic participants in the test environment. Referring to the example in step 303, the output parameters of the sensor model can include relevant parameters of vehicle 2 relative to vehicle 1. For example, the position of vehicle 2 relative to vehicle 1, the velocity of vehicle 2 relative to vehicle 1, the angular velocity of vehicle 2 relative to vehicle 1, and the angle of vehicle 2 relative to vehicle 1.

[0154] Optionally, the output parameters of the sensor model can also be output parameters with appropriate noise added, used to simulate measurement errors. For example, the output parameters of the sensor model may include the relative position and relative speed parameters of vehicle 2 relative to vehicle 1.

[0155] Step 305: Input the parameters of the perceptible target into the decision module.

[0156] The above method has low requirements for sensor models, simple structure, and can ensure high efficiency during simulation.

[0157] However, the methods described above consider ideal scenarios for the sensors. Since active sensing sensors perceive the environment by actively emitting energy waves, the accuracy of their results depends on factors such as the reflection intensity of the target object, the propagation of the energy waves, and the emission and reception of the energy waves. In other words, the perception results of active sensing sensors are affected by multiple factors, including the material of the target object, its location, distance, and environmental weather. Furthermore, the methods described above only consider an ideal scenario where the sensor can detect objects within a geometrically defined perceptible range. The relative distance, relative speed, and angle of the detectable objects are not obtained from the actual sensor but are set during scenario simulation. Therefore, this sensor simulation cannot reflect the impact of different environmental conditions on the measurement results. Directly using ideal data as the output of the sensor model will lead to significant deviations between the simulation and real-world results. For example, in a possible scenario, in a real environment, the sensor can detect a vehicle ahead. However, based on the above model and the geometrically defined perceptible area, the sensor model might determine that the vehicle ahead is obstructed. This could potentially introduce more unpredictable impacts on subsequent autonomous driving decision-making and control algorithms, failing to achieve the purpose of simulating and testing intelligent vehicles.

[0158] Therefore, the quality of the sensor model determines the realism of the target objects perceived by the intelligent vehicle in the simulation test. In other words, whether the sensor model can realistically reflect the impact of sensor measurement results under different environments directly affects the reliability of the simulation test results of the intelligent vehicle.

[0159] Based on the aforementioned problems, another possible method for sensor simulation is to model the radar sensor based on the physical characteristics described above. For example, this involves detailed modeling of the actual physical processes of millimeter-wave radar, such as energy wave transmission and reception, propagation, and target reflection. Mathematical models can be established for each hardware module involved in the millimeter-wave radar's operation to simulate the entire process. For instance, modeling the transceiver loop involves oscillators, filters, amplifiers, and mixers. This model can reflect the internal workings of the millimeter-wave radar and the details of electromagnetic wave propagation, yielding high-precision simulation results. However, this method is complex, consumes significant computational resources, and has poor real-time performance, making it difficult to guarantee simulation efficiency and meet the needs of real-time simulation testing for intelligent vehicles. Especially for large-scale scenario simulations based on cloud platforms, this sensor model consumes substantial computational resources and cannot guarantee simulation efficiency. Furthermore, it is unsuitable for the development of intelligent vehicle decision-making and control algorithms, particularly in the early stages. The limited parameters of the sensor models considered in these early stages make it difficult to effectively utilize the sensor parameters simulated using the aforementioned method (simulating each module of the millimeter-wave radar), leading to resource waste.

[0160] Therefore, this application provides a sensor simulation method. Figure 4a The application scenarios shown can include measurement devices and testing devices. The measurement device can be a vehicle equipped with sensors, such as millimeter-wave radar, cameras, and lidar. It can also include cloud-based testing devices, which can include hardware supporting simulation software, such as personal computers, servers, in-vehicle mobile terminals, industrial control computers, and embedded devices. For example, the testing device can be implemented using a cloud server or virtual machine. The testing device can also be a chip that supports running simulation software.

[0161] This explanation will use the modeling of a millimeter-wave radar sensor as an example. Figure 4a In the scenario shown, the sensor is a radar sensor, the measuring device is a vehicle, and the testing device is a server. This application provides a sensor simulation method, such as... Figure 4b As shown, it may include:

[0162] S401: Acquire sensor measurement data.

[0163] The measurement data includes: position information and speed information of the second target vehicle relative to the sensor, as well as sensor feature measurement values ​​of the second target vehicle collected by the sensor;

[0164] The sensor characteristic measurements include: RCS measurement and SNR measurement; the sensor is located in the measuring vehicle; the second target vehicle is a vehicle near the measuring vehicle.

[0165] S402: Train the sensor model based on the measurement data and the obtained annotation information from the sensor.

[0166] The sensor model's sample input consists of the second target vehicle's position information, speed information, and labeling information relative to the sensor, and the sensor model's output consists of the second target vehicle's sensor feature prediction values. The second target vehicle's sensor feature prediction values ​​include at least one of the following: RCS prediction value and SNR prediction value.

[0167] The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located.

[0168] This application, when modeling the sensor, does not require modeling each module of the sensor, but considers the physical characteristics of the radar sensor when measuring the target, optimizes the output results of the sensor model, and thus effectively improves the simulation effect.

[0169] The following example illustrates the physical characteristics of a target when a radar sensor measures it.

[0170] Millimeter-wave radar detects targets and can obtain the distance and speed between the moving target and the radar sensor. If the millimeter-wave radar is installed on a vehicle and the target is another vehicle, the vehicle speed, relative position, relative distance, and azimuth of the target vehicle can be determined based on the echo signal collected by the radar.

[0171] Furthermore, by receiving signals, the target's RCS information can be obtained. RCS information can be used to describe the target's backscattering characteristics under radar illumination. The RCS sequence of a space target is related to factors such as the target's shape and structure, the frequency of the electromagnetic wave, the polarization of the incident field, the polarization of the receiving antenna, and the target's angular position (attitude angle) relative to the direction of the incoming wave. For the same measuring radar, the frequency of the electromagnetic wave, the polarization of the incident field, the polarization of the receiving antenna, and the target's angular position (attitude angle) relative to the direction of the incoming wave can be determined. Therefore, the target's mean RCS value can be correlated with the target's structure and attitude.

[0172] For example, the information about the target object output by the sensor can also include structural information such as the width of the target object.

[0173] When considering a vehicle as the target object, the relative target attitude between the sensor and the vehicle is usually relatively stable; for example, the sensor can detect the rear, front, and sides of the vehicle. Therefore, the mean RCS of the target can be used as a feature to identify the target's structure, thereby classifying the reflection intensity of different targets and thus classifying their structures. For example, the type of vehicle, such as a sedan, truck, or bus, can be distinguished based on its length and shape.

[0174] In map scenarios, the pose of targets in space is usually relatively stable, and the RCS (Radar Cross Section) measurements of spatial targets are stable. Therefore, the mean RCS of a target can be used as a feature to identify its structure, thereby classifying the reflection intensity of different targets and thus classifying their structures. For example, targets can be distinguished as lane boundaries, lane lines or curbs, road obstacles, tunnels, bridges, etc.

[0175] Therefore, considering the physical characteristics of the target object when the radar sensor measures it, it can be determined that the sensor model needs to possess at least the following physical characteristics:

[0176] The information about the target object output by the sensor model can include: the pose state information of the target object relative to the sensor, and the feature information of the target object.

[0177] The pose information between the target object and the sensor may include: the relative distance between the target object and the sensor, the relative velocity between the target object and the sensor, the azimuth angle between the target object and the sensor, the width information of the target object and other structural information, and the yaw angle between the target object and the sensor.

[0178] The characteristic information of the target object may include: the RCS information of the target object, the SNR information of the target object, the polarization information of the target object, etc.

[0179] In some embodiments of S401, the sensor's measurement data can be measurement information collected by the vehicle's radar sensor during actual use. In this application, the measurement information can include at least one of the following: sensor-collected measurement data, environmental information, and positioning information. The environmental information can include the number and location of pedestrians in the surrounding environment, pedestrian density, vehicle density, road information, weather information, etc., and the positioning information can include the latitude and longitude of the current location or its marking on a map. The sensor can periodically perform measurements and then report the measurement information to the testing device.

[0180] For example, the preset areas of the vehicle's sensors, such as Figure 4aThe area circled in the dashed box represents the region centered on the vehicle with a preset distance as its radius. This preset distance can be a value less than or equal to the radius of the radar signal coverage area emitted by vehicle A. It can also be an area determined by other methods, such as... Figure 1b The sector-shaped area shown is not limited here. Figure 4a As shown, the vehicle's sensors can be located within a preset area, and the target objects they measure can be vehicles, obstacles, lane lines, etc. The vehicle's sensors can determine the measurement information of target objects within the preset range.

[0181] For example, when the vehicle's sensors are working, they collect measurement data of the target object output by the sensors.

[0182] Taking a vehicle as the target object as an example, the measurement data may include the target object's position information relative to the sensor (e.g., ...). Figure 4c As shown, the target object is the distance r of vehicle 2 relative to sensor 1 on vehicle 1, the angle θ of target vehicle 2 relative to sensor 1, and the velocity information of the target object relative to the sensor (e.g., the velocity of the target object relative to the sensor, the angular velocity of the target object relative to the sensor).

[0183] Optionally, considering the varying radar signal reflection intensities at different locations of the target (e.g., the rear of a vehicle reflects radar signals more strongly than its sides), the target's yaw angle α data can also be collected. The target's position information relative to the sensor can also include the target's yaw angle α data relative to the sensor. For example... Figure 4c As shown, the position information of vehicle 2 relative to sensor 1 may also include the yaw angle α data of vehicle 2 relative to sensor 1. The yaw angle of the target can be manually labeled or obtained by measurement through other sensors, which is not limited here. The yaw angle of the target object itself can reflect the different radar reflection intensities of different parts of itself, so the sensor model trained by the measurement information has higher accuracy.

[0184] In some embodiments, the measurement data may further include: measured values ​​of feature information. For example, the measured values ​​of feature information may be sensor feature values ​​of the target object acquired by the sensor, such as measured values ​​of the signal-to-noise ratio (SNR) information, the RCS information, and the polarization information of the target object acquired by the sensor.

[0185] One possible implementation is that the measured values ​​of SNR, RCS, and polarization information in the echo signal can be stored through imaging. This means imaging information can be generated based on the echo signal. Imaging information can be understood as the target's response to the transmitted signal, primarily the image information formed by the target's backscattering. Imaging information can include various types of information, such as RCS, phase, amplitude, and polarization information from the echo signal. Another possible implementation for generating imaging information from the target's reflected echo signal is to process the received echo signal, such as performing down-conversion or analog-to-digital conversion, and then using a synthetic aperture radar (SAR) imaging algorithm to obtain the imaging information. Another possible implementation is that the imaging information can be stored as point cloud data. Point cloud data can include radar characteristic information such as the target's range, azimuth, elevation, and velocity. For example, this measurement data can be transmitted to the vehicle's processor via the CAN bus, allowing the processor to make decisions based on the obtained measurement data.

[0186] In some embodiments, taking a vehicle as an example, the sensors measuring the vehicle can collect characteristic information of the echo signal returned by the second target vehicle.

[0187] Considering that environmental information can reflect the impact of environmental factors such as rain, snow, and road material on radar reflection intensity, optionally, environmental information (e.g., weather, road conditions) corresponding to the time when the test device collects measurement data of the target can also be obtained. Therefore, sensor models trained with measurement information that incorporates environmental information have higher accuracy. This measurement information can be obtained through manual annotation or other methods, such as road information stored in the current map server.

[0188] Among them, the weather information in the environment can be divided into four categories: sunny days, rainy days, foggy days, and snowy days. Of course, it can also include other types of information.

[0189] Taking a road as the target object as an example, the sensors of the measuring vehicle can collect feature information of the echo signals on the road. For example, under different road environments, such as conditions with obstructions (e.g., fallen leaves), water accumulation, or snow accumulation, the polarization information of the echo signals can be used to determine the polarization characteristics of the obstructions, water accumulation, or snow accumulation on the target. This allows for the determination of the boundary and material characteristics of the obstructions, water accumulation, or snow accumulation on the target's echo signal, thereby determining the impact of obstructions, water accumulation, or snow accumulation on the target's echo signal. This leads to a more accurate identification of the target as an obstructed vehicle. Therefore, the polarization information collected by the vehicle can be used to train a sensor model, enabling the sensor model to predict feature information under different scenarios. This provides more information for the subsequent decision-making module, resulting in a more realistic simulation that improves the simulation effect of the decision-making module.

[0190] For example, when a lane is obstructed, such as lane lines or lane boundaries, the decision-making module can remove the obstruction based on the polarization characteristics of the obstruction, water accumulation, or snow accumulation measured by sensors, thereby improving the decision-making effect. Correspondingly, when simulating sensors using a sensor model, the polarization information collected by the sensors can be used as the output parameters predicted by the sensor model. Thus, the decision-making module can obtain more simulation information from the real sensors based on this predicted polarization information, potentially improving the simulation effect of the decision-making module.

[0191] In another possible approach, the boundaries of the target may change due to different road conditions. For example, on rainy or snowy days, water or snow on the vehicle may cause changes in the echo signal of the target vehicle. Based on the polarization information in the echo signal, it can be determined whether the material characteristics of the target vehicle are affected by rain or snow, identify whether the vehicle has water or snow accumulation, and further determine the boundary features of water accumulation and road boundary features to improve the decision-making effect of the decision module.

[0192] For example, in a flooded road surface, the size of the flooded area may change the passable road conditions. Therefore, based on the polarization information in the echo signal, the material characteristics of the floodwater and lanes can be determined, identifying whether a lane is flooded and further determining the boundary features of the floodwater and the road. This allows for a more accurate determination of the road's characteristics under flooded conditions, based on the feature information predicted by the sensor model. Subsequent decision-making modules can then use this predicted feature information to determine the current flooding situation of a lane, such as the boundary information of the floodwater, thus improving navigation or route planning simulations. For instance, if a flooded area occupies a lane, the polarization information of the echo signal detected on one lane corresponds to the polarization characteristics of the lane under floodwater (e.g., boundary features and material characteristics of the floodwater), while the polarization information generated on other lanes corresponds to the polarization characteristics of lanes without floodwater (e.g., boundary features and material characteristics of the floodwater). Therefore, it can be determined that the lane is covered by floodwater, while other lanes are passable.

[0193] In addition, road types can be divided into four categories: ordinary asphalt pavement, ordinary concrete pavement, bridge deck, and tunnel. Of course, other types of information can also be included.

[0194] In some embodiments, the environment can be further divided according to the attributes of the environmental objects, which is beneficial to provide more training information (environmental information) when constructing the sensor model. As a result, the trained sensor model can obtain simulation results that are closer to the real-world environmental objects in different scenarios, thereby improving the performance of the sensor model. This is beneficial to subsequent decision-making using the prediction data obtained from the sensor model, so as to achieve the purpose of simulation and improve the simulation effect.

[0195] For example, environmental objects can be distinguished based on lane or non-lane boundaries to determine their boundary information, thereby enabling object identification. For instance, the boundary information of an environmental object can refer to key points or lines describing the boundaries of obstacles in a road, or to the boundary information describing a lane. For example, based on lane boundaries, lanes can be categorized into various environmental objects. The types of environmental object boundaries can include, but are not limited to, any one or more of the following: lane lines, curbs, road obstacles, etc. Lanes can be categorized as: single lanes, two lanes, multiple lanes, starting lanes, middle lanes, merging lanes, forking lanes, intersections, etc. A starting lane can be a lane corresponding to several lane lines on a road, including a starting point. The boundary of a starting and ending lane can be the starting line of the lane. An ending lane can be a lane corresponding to several lane lines on a road, including an ending point. The boundary of an ending lane is the stop line of the lane. Generally, in practical applications, the starting line of a lane and the stop line of the opposite lane are on a straight line. Merging and branching lanes can be identified by lane change points. These points can be forks created by adding turning lanes near intersections, merging points where one lane is removed when entering a new road from an intersection, or exit lanes of highways / elevated roads, or merging points of entering lanes of highways / elevated roads. Lanes can also be further classified based on obstacles present, such as tunnel lanes, elevated road entrance lanes, elevated road exit lanes, and bridges.

[0196] Optionally, in other embodiments, measurement information from different sensors can be obtained under different scenarios.

[0197] Taking a millimeter-wave radar sensor as an example, let's assume the scenarios include bustling market scenes, suburban scenes, highway scenes, and special weather scenes.

[0198] The parameters of the sensor corresponding to the bustling market scene can include: a millimeter-wave radar sensor operating in SRR mode. Thus, when the sensor operates in SRR mode, the corresponding target object's distance r, angle θ, velocity, and energy characteristic information such as SNR and RCS relative to the sensor are obtained.

[0199] The parameters of the sensors corresponding to the highway scenario can include: a millimeter-wave radar sensor operating in LRR mode. Thus, when the sensor operates in LRR mode, the corresponding target distance r, angle θ, velocity, and sensor characteristic information such as SNR and RCS relative to the sensor can be obtained.

[0200] In special weather scenarios, such as rainy weather, the sensor parameters can include: a millimeter-wave radar sensor operating in SRR mode. When the sensor operates in SRR mode under special weather conditions, it obtains the corresponding energy characteristic information such as the target object's distance r, angle θ, velocity, and SNR, RCS, and polarization information relative to the sensor.

[0201] Correspondingly, the period for collecting measurement information can also be set as needed to obtain better modeling results.

[0202] In other embodiments, considering that the vehicle may use multiple types of sensors to make decisions, the measurement information from multiple types of sensors can be collected during the acquisition of sensor measurement information. This results in more accurate environmental information, which helps the model to better simulate different scenarios.

[0203] Different scene names represent different categories of measurement information. For example, the scenes include bustling market scenes, suburban scenes, and highway scenes.

[0204] The parameters corresponding to the bustling market scene can include GPS operating in high-precision positioning mode, IMU and camera sensors reporting measurement information at fixed intervals according to a set period, and lidar and millimeter-wave radar sensors operating in SRR mode. Therefore, the determined measurement information includes: the sensor positioning information, the measurement information reported by the IMU and camera sensors, and the measurement information reported by the radar sensors.

[0205] Of course, in this scenario, measurement data collected by MRR or LRR type sensor models can also be collected to provide more training samples and improve the accuracy and robustness of the model.

[0206] Parameters for suburban scenarios may include GPS operating in low-precision positioning mode, IMU reporting measurement information at fixed intervals according to a set period, camera sensors reporting measurement information when pedestrians are detected within a set range, and LiDAR and millimeter-wave radar sensors operating in MRR mode. Therefore, the determined measurement information includes: sensor positioning information, measurement information reported by the IMU and camera sensors, and measurement information reported by the radar sensors.

[0207] Of course, in this scenario, measurement data collected by SRR or LRR type sensor models can also be collected to provide more training samples and improve the accuracy and robustness of the model.

[0208] The parameters corresponding to the highway scenario may include GPS operating in low-precision positioning mode, IMU and camera sensors reporting measurement information when pedestrians or vehicles are detected within a set range, and LiDAR and millimeter-wave radar sensors operating in LRR mode. Therefore, the determined measurement information includes: sensor positioning information, measurement information reported by the IMU and camera sensors, and measurement information reported by the radar sensors.

[0209] Of course, in this scenario, measurement data collected by SRR or MRR type sensor models can also be collected to provide more training samples and improve the accuracy and robustness of the model.

[0210] Based on the correspondence between sensor categories and sensor parameters, the testing device can model accordingly based on different sensor types and obtain more scene-related parameters through other sensors. This allows the subsequent decision-making module to use more information to make decisions and improves the simulation effect of the verification decision-making module.

[0211] In S402, the measurement information collected by the sensor during use is used as training samples for the sensor model to obtain the sensor model when the target vehicle is at different positions (e.g., relative distance, relative angle, yaw angle), speed information, and different environmental information (e.g., different weather, different road conditions, different road types) with respect to the sensor.

[0212] The output of this sensor model is the predicted value of the sensor's feature information (e.g., SNR, RCS, polarization information, etc.), while other measurement information (e.g., measurement data other than the sensor's feature information, location information, and environmental information, etc.) serves as the input to the sensor model for supervised learning training. Therefore, during training, a training sample can include training data and validation data. The training data consists of the sensor model input data, i.e., measurement data other than the sensor's feature information, location information, and environmental information. The validation data consists of the measured values ​​of the sensor's feature information from the training samples.

[0213] Taking the training of a millimeter-wave radar sensor model as an example, the output parameters of the millimeter-wave radar sensor model can be predicted values ​​of the feature information of the millimeter-wave radar sensor, such as predicted values ​​of SNR, RCS, and polarization information. The input parameters of the millimeter-wave radar sensor model can include: target position information relative to the sensor (range r, angle θ, yaw angle), velocity information, environmental information, positioning information, and other measurement information other than feature information.

[0214] Environmental information may include weather type, road type, etc. It may also include parameters obtained from other sensors, such as whether there are fallen leaves, rain, or snow obstructing the view within the sensor's detection range.

[0215] like Figure 4d As shown, different sensor models can be trained for different types of sensors.

[0216] In some embodiments, a supervised learning algorithm using a support vector regression (SVR) model can be employed to train the measurement information acquired by this type of sensor. The input data for the SVR model may include measurement information other than the sensor's feature information.

[0217] The output data of the SVR model may include: predicted values ​​of sensor feature information, such as predicted values ​​of SNR and RCS.

[0218] During training, the model can be trained on the feature information of each sensor. For example, it can be trained on the predicted SNR value. After the SNR feature information training reaches the required accuracy, the model can then be trained on the predicted RCS value. Alternatively, it can be trained on the predicted RCS value, and after the RCS feature information training reaches the required accuracy, the model can then be trained on the predicted SNR value. Of course, it is also possible to train on all feature information together; this is not a limitation.

[0219] Optionally, measurement information from the same sensor type can be used to train different measurement information collected in different scenarios, such as bustling market scenes, suburban scenes, and highway scenes.

[0220] The parameters corresponding to the bustling market scene can include GPS 126 operating in high-precision positioning mode, IMU 125 and camera sensor 123 reporting measurement information at fixed intervals according to a set period, and LiDAR sensor and millimeter-wave radar sensor operating in SRR mode. Therefore, for the sensor model of the LiDAR sensor or millimeter-wave radar sensor, the measured information can be stored in the SRR type and bustling market scene, so that subsequent sensor models can call the corresponding measurement information as training data. Of course, measurement data collected by other types of sensor models can also be collected in this scene to provide more training samples and improve the model's accuracy and robustness.

[0221] The configuration parameters for suburban scenarios can include GPS 126 operating in low-precision positioning mode, IMU 125 reporting measurement information at fixed intervals according to a set period, camera sensor 123 reporting measurement information when a pedestrian is detected within a set range, and LiDAR sensor and millimeter-wave radar sensor operating in MRR mode. Thus, for the sensor model of LiDAR sensor or millimeter-wave radar sensor, the measured measurement information can be stored in MRR type and suburban scenario, so that the sensor model can call the corresponding measurement information as training data for subsequent training.

[0222] The configuration parameters for the highway scenario can include GPS 126 operating in low-precision positioning mode, IMU 125 and camera sensor 123 reporting measurement information when pedestrians or vehicles are detected within a set range, and LiDAR and millimeter-wave radar sensors operating in LRR mode. Therefore, for the sensor models of LiDAR or millimeter-wave radar sensors, the measured information can be stored in LRR type and highway scenario, so that subsequent sensor models can call the corresponding measurement information as training data.

[0223] In some embodiments, when training an SRR-type sensor model, multiple scenarios can be used for training. For example, when training a bustling market scene, training samples of measurement information collected by an SRR-type sensor in a bustling market scene can be selected for training. Similarly, when training a suburban scene, training samples of measurement information collected by an SRR-type sensor in a suburban scene can be selected for training. Likewise, when training a suburban scene, training samples of measurement information collected by an MRR-type sensor in a suburban scene can be selected for training. Finally, when training a suburban scene, training samples of measurement information collected by an LRR-type sensor in a suburban scene can be selected for training. Thus, the trained sensor model can be used in different scenarios. Correspondingly, when training an MRR-type sensor model, MRR-type sensor models can be trained for scenarios such as bustling market scenes, suburban scenes, and highway scenes. Similarly, when training an LRR-type sensor model, LRR-type sensor models can be trained for scenarios such as bustling market scenes, suburban scenes, and highway scenes.

[0224] It should be noted that the above sensor model is an example of the SVR model. The sensor model can also be determined by other models or algorithms. For example, the sensor model includes, but is not limited to, regression models, NN models, random forests, deep neural networks, autoregressive moving average models (ARMA), gradient boosting decision tree (GBDT) models, or XGBoost models, etc.

[0225] Figure 5a This is an exemplary functional block diagram of a sensor testing system according to an embodiment of this application. Figure 5a As shown, this system can be applied in testing equipment or other application platforms. The following explanation uses a cloud server as an example. The system includes at least one sensor model, a decision-making module, and a scene module. The sensor model can be a... Figure 1a The sensor model, decision module, and scene module of the sensor system 120 shown can be integrated into a single test device. Alternatively, the sensor model, decision module, and scene module can be independent modules that share the memory of the test environment. It should be noted that the sensor model, decision module, and scene module can be implemented in any feasible combination, and this application does not impose any specific limitations. To better understand the embodiments of this application, the following will use... Figure 5a The embodiments of this application will be described using systems that are the same as or similar to the system shown. Figure 5a The application scenarios of the illustrated testing system can include testing devices, which can be testing devices with sensor models. The network elements of the testing device include hardware devices that support running simulation software, such as personal computers, servers, vehicle-mounted mobile terminals, industrial control computers, embedded devices, etc. For example, the testing device can be implemented by a cloud server or virtual machine. The testing device can also be a chip that supports running simulation software. Figure 5b The illustration shows a vehicle simulation method provided in an embodiment of this application, specifically including:

[0226] S501: Input the position and speed information of the first target vehicle relative to the simulated vehicle, as well as the road environment information of the simulated vehicle, into the sensor model to obtain the sensor feature prediction value of the first target vehicle.

[0227] The sensor feature prediction values ​​include at least one of the following: RCS prediction value and SNR prediction value; the sensor model is used to simulate the sensors in the simulated vehicle, the first target vehicle is a vehicle in the test environment where the simulated vehicle is located, the position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle are determined according to the test environment, and the sensor model is trained based on the sensor measurement data and the labeled road environment information.

[0228] S502: Input the sensor feature prediction value of the first target vehicle into the decision module of the simulation vehicle to obtain the simulation decision result of the simulation vehicle.

[0229] The decision module is used to output vehicle driving decisions based on sensor feature predictions.

[0230] In this embodiment, by considering the physical characteristics of the sensor, the output results of the sensor model are optimized, thereby effectively improving the simulation effect. Compared to using only relative velocity, phase distance, and angle data from the test environment as output parameters of the radar sensor, this application combines the physical characteristics of the radar sensor when measuring the target to establish the corresponding output parameters of the radar sensor model. This results in output parameters that are closer to those of a real millimeter-wave radar sensor.

[0231] Prior to S501, vehicle simulation devices could determine the parameters of sensors and target objects in the test environment.

[0232] The sensor is the sensor to be tested. The following description uses the example of the sensor being located on a simulated vehicle. If the sensor is located on other devices to be tested, this embodiment can be referred to.

[0233] In some embodiments, the testing apparatus can acquire test information about the target object in the test environment relative to the sensor. It should be noted that the target object is not limited to objects near the sensor. It can also be a target object within a preset area near the sensor. The preset area can be determined based on the sensor's detectable range, or it can be determined in other ways, without limitation here. The target object is not limited to vehicles, but can also be various objects in the test environment, such as roadside buildings, pedestrians, lanes, bridges, tunnels, etc.

[0234] The test information may include test data such as the pose state of the target object relative to the sensor and environmental information.

[0235] The pose information may include location and velocity information. Environmental information may include weather, road, traffic sign, and traffic light data.

[0236] For example, structural information such as the target object's relative angle to the sensor, the target object's relative distance to the sensor, the target object's relative velocity to the sensor, the target object's relative angular velocity to the sensor, the target object's relative acceleration to the sensor, the target object's relative angular acceleration to the sensor, and the target object's dimensions.

[0237] In some embodiments, the first target vehicle can be determined as the vehicle in the test environment where the simulated vehicle is located, based on the test environment. Furthermore, test information for the first target vehicle can be determined based on the test environment. For example, the test information for the first target vehicle may include: the pose state of the first target vehicle relative to the simulated vehicle and environmental information of the simulated vehicle, etc.

[0238] It should be noted that the test information can be determined based on the collected measurement information, or it can be determined through other means. For example, in some embodiments, the test environment can be provided by intelligent vehicle simulation testing software to simulate real-world traffic scene data. Test information of the simulated traffic object can be extracted from the test environment. For example, the simulation software can be vehicle testing software (e.g., VTD software), and the test environment is provided by the vehicle testing software.

[0239] For example, different types of sensor test information can correspond to different scenarios, such as bustling market scenes, suburban scenes, and highway scenes.

[0240] The parameters corresponding to the bustling market scene can include GPS 126 operating in high-precision positioning mode, IMU 125 and camera sensor 123 reporting measurement information at fixed intervals according to a set period, and lidar sensor and millimeter-wave radar sensor operating in SRR mode. Thus, in this scenario, the corresponding test information of the SRR type radar sensor can be called so that the subsequent sensor model can call the corresponding test information for prediction.

[0241] The configuration parameters for suburban scenarios can include GPS 126 operating in low-precision positioning mode, IMU 125 reporting measurement information at fixed intervals according to a set period, camera sensor 123 reporting measurement information when a pedestrian is detected within a set range, and lidar sensor and millimeter-wave radar sensor operating in MRR mode. Thus, in this scenario, the corresponding test information of the MRR type radar sensor can be called so that the subsequent sensor model can call the corresponding test information for prediction.

[0242] The configuration parameters for the highway scenario can include GPS 126 operating in low-precision positioning mode, IMU 125 and camera sensor 123 reporting measurement information when pedestrians or vehicles are detected within a set range, and LiDAR and millimeter-wave radar sensors operating in LRR mode. Therefore, in this scenario, the corresponding test information from the LRR type radar sensors can be retrieved so that subsequent sensor models can use this test information for prediction.

[0243] In some embodiments of S501, the sensor feature prediction value of the first target vehicle can be obtained by inputting the pose state information of the first target vehicle relative to the sensor and the environmental information of the simulated vehicle into the sensor model; the sensor feature prediction value includes at least one of the following: RCS prediction value and SNR prediction value; the first target vehicle is a vehicle in the test environment where the simulated vehicle is located, and the pose state information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle are determined according to the test environment.

[0244] The sensor model is obtained by training the sensor with supervised learning based on the measurement information collected by the sensor.

[0245] In using the sensor model, test information from the testing environment can be used as input. This test information can include other test information besides the sensor's feature information. For example, environmental information determined in the testing environment, the pose state information of the target vehicle relative to the sensor, etc. Thus, the predicted values ​​of the sensor's feature information corresponding to the target are predicted and output. For example, the predicted values ​​of the target object's SNR, RCS, and polarization information output by the sensor model.

[0246] The pose and environmental information of the target object within the sensor model's detection range are obtained through the communication interface provided by the testing environment and used as input to the sensor model. Therefore, the predicted value of the target object's feature information can be obtained from the prediction data output by the sensor model. In other words, the predicted value of the target object's feature information can be obtained through the sensor model under different pose and environmental conditions.

[0247] Therefore, based on the predicted values ​​of the target object's feature information and the test information of the target object, the predicted information of the target object can be determined.

[0248] The prediction information of the target object includes: the test information of the target object (e.g., test data such as the pose state information and environmental information of the target object), and the predicted value of the feature information of the target object (e.g., the predicted value of RCS and the predicted value of SNR).

[0249] In S502, the vehicle simulation device can input the predicted information of the target object into the decision module.

[0250] In some embodiments, the predicted information of the target object can be used as input to the decision control (or fusion perception) algorithm to verify the decision control (or fusion perception) algorithm, and the target object information output by the radar sensor can be received as input for calculation in order to obtain the decision result.

[0251] By using sensor models, we can obtain predicted values ​​of the feature information of target objects in the test environment. This makes the predicted information of target objects closer to the output of real millimeter-wave radar sensors, reflects the physical characteristics of the sensors, and is more conducive to simulating the performance of decision-making algorithms in actual use, thus improving the simulation effect.

[0252] In this embodiment of the application, the output results of the sensor model can also be optimized by taking into account the effects of the physical characteristics of the sensor, thereby effectively improving the simulation effect.

[0253] The following example illustrates the physical characteristics of a target when a radar sensor measures it.

[0254] (1) Considering the sensor's ability to distinguish target objects, radar may be unable to differentiate between two objects that are close together and at the same distance. In this case, the sensor model should also be able to output two objects that are close together and at the same distance as a single target object. This is beneficial for subsequent verification of whether the autonomous driving decision control module can handle scenarios where the sensor makes a mistake in identification. In one possible scenario, due to multipath propagation, the sensor may sometimes detect occluded objects. Therefore, the target object output by the sensor model should also include potentially occluded objects.

[0255] In map scenarios, the pose of targets in space is usually relatively stable, and the RCS (Radar Cross Section) measurements of spatial targets are stable. Therefore, the mean RCS of a target can be used as a feature to identify its structure, thereby classifying the reflection intensity of different targets and thus classifying their structures. For example, targets can be distinguished as lane boundaries, lane lines or curbs, road obstacles, tunnels, bridges, etc.

[0256] (2) The transmitted signal may include polarization information. Polarization reflects the time-varying pattern of the endpoints of the electric field vector of the wave. It can be classified into linear, circular, elliptical polarization and left-handed and right-handed polarization according to the shape and direction of its spatial trajectory. The polarization state of the electromagnetic wave reflects the time-varying characteristics of the electric field orientation of the electromagnetic wave received by the radar. The polarization parameters of the received signal can be estimated by using a polarized antenna or a polarization-sensitive array at the receiving end. Depending on the polarization mode of transmission and reception, the transmitted signal interacts with the target, and the echo scattering also varies. Wavelength and polarization mode both affect the acquired received signal. Therefore, the polarization information in the received signal may include: the polarization scattering matrix of the target and the polarization state of the electromagnetic wave. Among them, the polarization scattering matrix of the target is the polarization scattering effect of the target on the electromagnetic wave under a certain attitude and observation frequency. The polarization scattering matrix of the target characterizes the change in the polarization state of the radar target on the electromagnetic wave signal. That is, when the target is illuminated by the radar electromagnetic wave, the polarization state of the scattered electromagnetic wave may be different from the polarization state of the incident electromagnetic wave. The ability of a target to alter the polarization state of its electromagnetic waves is known as its depolarization characteristic. In this case, the radar target changes the polarization state of its electromagnetic waves. This change in polarization is determined by the target's shape, structure, and material. Therefore, the polarization information in the target's echo signal can be used to identify the target. In other words, polarization information can reveal the scattering characteristics of different targets and can be used to determine the target's surface features, shape, roughness, and other surface characteristics. Furthermore, by combining different polarization modes and wavelengths, different and complementary polarization information of the target can be determined, which is beneficial for obtaining more accurate information about the target's structure, material, and other surface characteristics.

[0257] (3) When the radar sensor measures the target, the influence of noise must also be considered. The sources of noise may be noise generated by the transmitter, noise received by the receiver, or interference from other radars. If the power of the interference signal is greater than the receiver sensitivity, the interference signal will interfere with the current radar. If the power of the interference signal is not greater than the receiver sensitivity, the interference signal will not interfere with the current radar and will be treated as noise.

[0258] Therefore, radar sensors also need to pass appropriate thresholds to determine whether the received signal is noise or a target object.

[0259] Given the differences in properties and parameters of different radar sensors—for example, variations in radar signal transmission power and receiver sensitivity—the corresponding thresholds also differ. This can lead to false negatives and false positives in the measurement of target objects.

[0260] False negatives occur when, during radar detection using threshold detection methods, the prevalence and fluctuations of noise mean that a target, though present, may not be detected due to its signal energy being below a certain threshold, and the sensor incorrectly identifies it as non-existent. False positives occur when, during radar detection, the target's signal energy is not higher than or even lower than the noise energy, but the threshold is set too low due to the prevalence and fluctuations of noise, allowing the millimeter-wave radar to detect it, thus mistaking a non-existent target for a present one.

[0261] When using threshold detection methods, due to the threshold mechanism and the prevalence of noise, four different scenarios arise when determining the presence of an echo signal in radar target detection. These four scenarios can be described by four probabilities. When a target is present, it is correctly identified as a target, a situation called "detection," and its probability is called the "detection probability." When a target is present, it is incorrectly identified as a target, a situation called "false negative," and its probability is called the "false alarm probability." When a target is not present, it is correctly identified as a target, a situation called "correctly not detected," and its probability is called the "correctly not detected probability." When a target is not present, it is incorrectly identified as a target, a situation called "false positive," and its probability is called the "false alarm probability."

[0262] Therefore, considering the physical characteristics of the target object when the radar sensor measures it, it can be determined that the sensor model can possess at least one of the following physical characteristics:

[0263] Based on the physical characteristics (1), the resolution capability of the measured target object is considered. Considering the resolution problem of the sensor, the radar may be unable to distinguish between two objects that are at the same distance and are close to each other. In this case, the sensor model should also be able to output two objects that are at the same distance and are close to each other as a single target object. This will help to verify whether the decision control module of autonomous driving can handle the scenario where the sensor identifies the wrong object.

[0264] Based on the physical characteristics (2), due to the multipath propagation phenomenon, the sensor can sometimes detect occluded objects. Therefore, the target object output by the sensor model should also include objects that may be occluded.

[0265] Combined with physical characteristics (3), the results of measuring the target object may result in false negatives or false positives.

[0266] Compared to using only relative velocity, phase distance, and angle data from the test environment as output parameters for radar sensors, this application combines the physical characteristics of the radar sensor when measuring the target to establish the output parameters of the corresponding radar sensor model. This results in output parameters that more closely approximate those of a real millimeter-wave radar sensor.

[0267] In one possible implementation, target objects can be filtered based on the sensor's detectable range.

[0268] In some embodiments, prior to S502, the vehicle simulation device determines the first target vehicle as a vehicle within the detection range of the sensor among the candidate vehicles, determined based on the position and speed information of the candidate vehicles relative to the simulation vehicle; the position and speed information of the candidate vehicles relative to the simulation vehicle are determined based on the test environment, and the candidate vehicles are vehicles in the test environment where the simulation vehicle is located.

[0269] Each target object can be a target object within the sensor's detectable range.

[0270] The sensor's detectable range can be determined based on the sensor parameters obtained during the modeling of the radar sensor model. Considering that the sensor's detectable range may vary depending on the environment, it can also be determined based on the measurement information collected by the sensor and the environmental information in the current test environment. No limitation is made here.

[0271] like Figure 6a As shown, this embodiment provides a sensor detection range, which is a conical region. This conical region can be determined by several parameters. For example, the detectable angle β on the left side of the sensor, the detectable angle γ on the right side of the sensor, the near-end detectable distance of the sensor can be a first distance, and the far-end detectable distance of the sensor can be a second distance.

[0272] By defining the sensor's detection range, target objects outside the detection range can be removed, reducing the computational load of the simulation. The specific process may include: determining the detection range based on the radar's detectable distance and angle range; eliminating target objects outside the detectable range; and retaining target objects that intersect with the boundary of the detectable range.

[0273] For example, such as Figure 6b As shown, target objects 4 and 5, which are completely outside the radar detection range of vehicle 1, are eliminated, while target object 3, which intersects with the boundary of the detection range area, is retained. Similarly, target objects 1 and 2, which are completely within the detection range, are also retained, while target object 2, which is completely obscured, is not eliminated.

[0274] It should be noted that, as can be seen, Figure 6bThe target object 2 shown is completely obscured by target object 1. However, in this embodiment, target object 2 is not determined to be an undetectable target object. For target objects obscured by other objects, this reflects the physical characteristics that the radar sensor may be able to detect due to multipath propagation, thus providing a basis for the sensor model to detect objects outside the line of sight. That is, in step 502, target objects that are completely outside the detection range are deleted according to the detectable range parameters of the millimeter-wave radar. Target objects that are obscured but within the detection range are also considered as target objects of the sensor.

[0275] In one possible implementation, physical properties can be used to filter target objects and their predicted information.

[0276] Based on the test information of the target object in the test environment and the feature information of the target object that the test environment cannot provide, as predicted by the sensor model, the target object is further filtered to better obtain the predicted information of the target output by the proximity radar sensor and the measurement information of the sensor approaching the target.

[0277] For each target object, the predicted SNR value is used to determine whether the target object is visible relative to the sensor.

[0278] In some embodiments, prior to S502, the vehicle simulation device determines that the predicted SNR value of the first target vehicle is greater than a visible threshold.

[0279] Optionally, other characteristic information representing the sensor can also be used to determine whether the target object is visible relative to the sensor. This application does not limit this. The following uses SNR as an example to illustrate an example of determining whether the target object is visible relative to the sensor.

[0280] In some embodiments, the predicted SNR of the target object is compared with a corresponding visibility threshold. For example, in one possible approach, the visibility threshold is 1, meaning that a target object is considered to exist when the RCS signal strength in the echo signal is greater than the noise signal strength. That is, a target object is considered to exist when the predicted SNR is greater than or equal to 1. Accordingly, target objects with a predicted SNR less than 1 can be deleted.

[0281] In scenarios where the SNR is too low relative to the noise level and therefore undetectable, the sensor might mistakenly identify the target as nonexistent, resulting in a false negative. Another possible scenario is where the noise level is too high, causing the sensor to identify the target as a positive, resulting in a false positive. This demonstrates the characteristic of millimeter-wave radar capable of producing false negatives and false positives, and further effectively ensures the detection of objects outside the line-of-sight range.

[0282] In other embodiments, considering the possibility of false positives and false negatives, they can also be described by four probabilities. That is, when a target exists, it is judged as having a target, and the judgment is correct. This situation is called "detection", and its probability is called "detection probability". When a target exists, it is judged as not having a target, and the judgment is incorrect. This situation is called "false negative", and its probability is called "false alarm probability". When a target does not exist, it is judged as not having a target, and the judgment is correct. This situation is called "correct but not detected", and its probability is called "correct but not detected probability". When a target does not exist, it is judged as having a target, and the judgment is incorrect. This situation is called "false alarm probability".

[0283] Therefore, by setting appropriate probability thresholds, such as a detection probability threshold (outputting the probability of target object detection via sensor model when SNR is greater than the detection probability threshold), the model can output the probability of target object detection via sensor model when SNR is greater than the false negative probability threshold, a correct non-detection probability threshold (outputting the probability of target object detection via sensor model when SNR is greater than the correct non-detection probability threshold), and a false alarm probability threshold (outputting the probability of target object detection via sensor model when SNR is greater than the false alarm probability threshold), the decision model can also obtain the probability of sensor misjudgment based on the corresponding probabilities, thereby improving the accuracy of decision-making.

[0284] Another possible implementation is to update the target object and its predicted information based on the physical characteristics of the sensor and the pose state information of the target object.

[0285] The testing device can also determine whether there are multiple indistinguishable target objects based on at least one or a combination of the target object's pose state information: the target object's relative angle to the sensor, the target object's relative distance to the sensor, the target object's relative velocity to the sensor, the target object's relative angular velocity to the sensor, the target object's relative acceleration to the sensor, and the target object's relative angular acceleration to the sensor.

[0286] For example, the target object identified above can be used as a candidate object. Then, based on the pose state information of the first candidate object and the second candidate object, it can be determined whether the first and second candidate objects are indistinguishable to the sensor. That is, whether to treat the first and second candidate objects as one target object or as two separate target objects.

[0287] In some embodiments, prior to S502, the vehicle simulation device determines that the first target vehicle includes a first candidate vehicle and a second candidate vehicle; the sensor feature prediction value of the first target vehicle is determined based on the sensor feature prediction value of the first candidate vehicle and the sensor feature prediction value of the second candidate vehicle.

[0288] The vehicle simulation device determines that the first candidate vehicle and the second candidate vehicle satisfy the following conditions: the relative position of the first position to the second position is less than the first position threshold; the first position is the position of the first candidate target vehicle relative to the simulated vehicle, and the second position is the position of the second candidate target vehicle relative to the simulated vehicle.

[0289] Optionally, the first candidate vehicle and the second candidate vehicle further satisfy the following conditions: the relative speed of the first speed to the second speed is less than a first speed threshold; the first speed is the speed of the first candidate target vehicle relative to the simulated vehicle, and the second speed is the speed of the second candidate target vehicle relative to the simulated vehicle.

[0290] For example, if a sensor cannot distinguish between two target objects, such as a first candidate object and a second candidate object.

[0291] like Figure 7a As shown, in some embodiments (e.g., condition 1), the first position information of the first candidate object relative to the simulated vehicle and the second position information of the second candidate object relative to the simulated vehicle are less than a first position threshold. In this case, it can be considered that the sensor cannot distinguish between the two candidate objects, and when the sensor measures the target object, the measurement result should be the measurement result of one target object. The first position information can be the position information of the center position of the first candidate object, and the second position information can be the position information of the center position of the second candidate object. Of course, other position information can also be used. For example, the first position information can be the position information of the closest position of the first candidate object relative to vehicle 1, and the second position information can be the position information of the closest position of the second candidate object relative to vehicle 1. The position information can also be determined based on the characteristics of the candidate objects to better simulate the situation where a real radar sensor identifies different candidate objects as the same target object; this application does not limit this.

[0292] Therefore, the first candidate object and the second candidate object can be output as a single target object.

[0293] One possible implementation is to output the prediction information obtained from the first candidate object and the prediction information of the second candidate object as the prediction information of a single target object. For example, if the target object is the first target object, then the sensor feature prediction value of the first target object is determined based on the sensor feature prediction values ​​of the first candidate object and the second candidate object.

[0294] In some embodiments, the sensor feature prediction value of the first target object can be the average or a weighted average of the sensor feature prediction values ​​of the first candidate object and the second candidate object. The weighting method can be determined based on the characteristics of the first and second candidate objects, or based on the relationship between the first candidate object and the sensor, or other factors, which are not limited here.

[0295] like Figure 7b As shown, in another possible embodiment (e.g., condition 2), after determining that the first position information of the first candidate object relative to the simulated vehicle and the second position information of the second candidate object relative to the simulated vehicle are less than a first position threshold, angle information can also be used to determine whether the first candidate object and the second candidate object meet the condition of being close and will be considered by the sensor as the same target object. In some embodiments, the first candidate object and the second candidate object further satisfy the condition that the first angle information of the first candidate object relative to the sensor and the second angle information of the second candidate object relative to the sensor are less than a first angle threshold.

[0296] like Figure 7c As shown, in another possible embodiment (e.g., condition 3), after determining that the first position information of the first candidate object relative to the simulated vehicle and the second position information of the second candidate object relative to the simulated vehicle are less than a first position threshold, speed information can also be used to determine whether the first candidate object and the second candidate object meet the condition of being close and will be considered by the sensor as the same target object. In some embodiments, the first candidate object and the second candidate object further satisfy the condition that the first speed information of the first candidate object relative to the simulated vehicle and the second speed information of the second candidate object relative to the simulated vehicle are less than a first speed threshold.

[0297] In another possible embodiment (e.g., condition 4), after determining that the first position information of the first candidate object relative to the simulated vehicle and the second position information of the second candidate object relative to the simulated vehicle are less than a first position threshold, acceleration information can also be used to determine whether the first candidate object and the second candidate object meet the condition of being close and will be considered by the sensor as the same target object. In some embodiments, the first candidate object and the second candidate object further satisfy the condition that the first acceleration information of the first candidate object relative to the simulated vehicle and the second acceleration information of the second candidate object relative to the simulated vehicle are less than a first acceleration threshold.

[0298] The specific threshold can be set according to the sensor's resolution parameters, or it can be determined in other ways, such as by the measurement information collected from the sensor. No specific limit is set here.

[0299] Of course, other methods can be used to determine whether the first and second candidate objects will be mistakenly identified by the sensor as the same target object.

[0300] In some embodiments, in addition to the above conditions, an additional condition can be added to determine whether the first candidate object and the second candidate object will be mistakenly identified as the same target object by the sensor under different weather conditions.

[0301] Taking condition 1 as an example, for instance, under the influence of snow accumulation, the feature values ​​output by the sensor model can be used to determine whether the snow accumulation effect needs to be considered. Therefore, a second position threshold considering the snow accumulation effect can be selected. This second position threshold might be larger than the first position threshold because snow accumulation makes it easier for the sensor to fail to distinguish between two candidate objects. Figure 7d As shown, the tails of the first and second candidate objects are predicted to have snow accumulation. Therefore, a second position threshold can be selected to determine whether the first and second candidate objects are two candidate objects that the sensor cannot distinguish. For example, if the first position information of the first candidate object relative to the simulated vehicle and the second position information of the second candidate object relative to the simulated vehicle are less than the second position threshold, then it can be considered that the sensor cannot distinguish between the two candidate objects. When the sensor measures the target object, the measurement result should be the measurement result of one target object.

[0302] Correspondingly, conditions 2, 3 and 4 can be set according to different weather conditions. You can refer to the corresponding settings for condition 1, which will not be repeated here.

[0303] In some embodiments, the sensor may be configured to recognize objects as the same target only when at least a few of the above conditions are met. For example, objects may be recognized as the same target only when all conditions are met. Alternatively, the sensor may configure the sensor to recognize objects as the same target only when at least three conditions are met. The number of conditions that can be met can be set according to the sensor's accuracy and is not limited here. Conversely, the sensor may configure objects as different target objects if none of the above conditions are met.

[0304] Optionally, priorities can be set for the above conditions. For example, condition 1 has the highest priority, and condition 4 has the lowest priority. This allows for a better simulation of scenarios where different target objects are misidentified when the sensor outputs the target object.

[0305] This is used to reflect the characteristic that a sensor may be unable to distinguish between two objects that are close together.

[0306] Optionally, noise simulation can be added to the predicted information output by the sensor model to simulate the errors caused by the influence of external environmental noise on the real sensor.

[0307] For example, Gaussian white noise can be added to both the output target object pose information and the feature information output by the sensor model. The noise power is selected based on the actual sensor parameters and is not limited here.

[0308] Considering that the pose state information of the target object is an ideal value extracted from the test environment, error simulation can simulate the characteristics of real sensor data being affected by environmental noise.

[0309] Figure 8a This is an exemplary functional block diagram of a sensor testing system according to an embodiment of this application. Figure 5a As shown, this system can be applied in testing devices or other application platforms. The following description uses a cloud server as an example. The system includes at least one sensor module (which can be the sensor model trained above), a sensor detection range filtering module, a physical characteristic filtering module, a noise simulation module, a decision-making module, and a scene module. The sensor module can be as follows: Figure 4d or Figure 4e The pair shown Figure 1aThe sensor model simulating one or more sensors in the sensor system 120 shown, the decision module, and the scene module can be integrated as a whole into a test device computer system 160. The sensor model, decision module, and scene module can also be independent modules, sharing the memory of the test environment. It should be noted that the sensor module, decision module, and scene module of this application can be implemented in any feasible combination, and this application does not impose specific limitations. To better understand the embodiments of this application, the following will use... Figure 8a The embodiments of this application will be described using systems that are the same as or similar to the system shown. Figure 8a The application scenarios of the illustrated testing system may include testing devices, which can be testing devices with sensor models. The network elements of the testing device include hardware devices that support running simulation software, such as personal computers, servers, in-vehicle mobile terminals, industrial control computers, embedded devices, etc. For example, the testing device can be implemented by a cloud server or virtual machine. The testing device can also be a chip that supports running simulation software. The following is a specific example illustrating a vehicle simulation method provided in this application, such as... Figure 8b As shown, it includes:

[0310] Step 801: Determine the parameters of the sensors and target objects in the test environment.

[0311] For details, please refer to step 601.

[0312] Step 802: Determine whether the target object is visible to the sensor based on its detection range. If yes, proceed to step 803; otherwise, proceed to step 808.

[0313] Step 803: Based on the test information of the sensors and the test information of the target object in the test environment, determine the predicted data of the target object output by the radar sensor model.

[0314] For details, please refer to step 602.

[0315] Step 804: Determine whether the target object is visible relative to the sensor based on the predicted SNR value. If yes, proceed to step 805; otherwise, proceed to step 808.

[0316] Step 805: Based on the physical characteristics of the sensor and the pose state information of the target object, determine whether there are at least two indistinguishable target objects. If yes, proceed to step 808; otherwise, proceed to step 806.

[0317] Step 806: Update at least two indistinguishable target objects to a single target object and the prediction information of the updated target object.

[0318] Step 807: Output the predicted information of the target object to the decision module.

[0319] Step 808: Delete the prediction information for the target object.

[0320] like Figure 9 The diagram shows a structural schematic of a vehicle simulation device provided in this application. The device may include a sensor feature prediction module 901 and an output module 902. This device can be applied to a testing device, which may be a testing device with a sensor model. The network elements of the testing device may include hardware devices that support running simulation software, such as personal computers, servers, in-vehicle mobile terminals, industrial control computers, embedded devices, etc. For example, the testing device may be implemented by a cloud server or virtual machine. The testing device may also be a chip that supports running simulation software.

[0321] Optionally, the device may further include: a first determining module, a second determining module, and a third determining module. Optionally, the device may further include a sensor model training module for training a sensor model; the sensor model training module may include: an acquisition module and a training module.

[0322] The sensor feature prediction module 901 can be used to input the position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle into the sensor model to obtain the sensor feature prediction value of the first target vehicle. The sensor feature prediction value includes at least one of the following: radar cross section (RCS) prediction value and signal-to-noise ratio (SNR) prediction value. The sensor model is used to simulate the sensors in the simulated vehicle. The first target vehicle is a vehicle in the test environment where the simulated vehicle is located. The position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle are determined according to the test environment. The sensor model is trained based on the measurement data of the sensors and the labeled road environment information.

[0323] Output module 902 is used to input the sensor feature prediction values ​​of the first target vehicle to the decision module of the simulated vehicle to obtain the simulation decision result of the simulated vehicle; wherein, the decision module is used to output the vehicle driving decision determined based on the sensor feature prediction values. This decision module can be a module in the vehicle simulation device or a separately set module, which is not limited here.

[0324] In one possible implementation, the device may further include:

[0325] The first determining module is used to determine the vehicle within the detection range of the sensor as the first target vehicle from among the candidate vehicles based on the position information and speed information of the candidate vehicle relative to the simulated vehicle; the position information and speed information of the candidate vehicle relative to the simulated vehicle are determined based on the test environment; the candidate vehicle is a vehicle in the test environment where the simulated vehicle is located.

[0326] In one possible implementation, the device may further include: a second determining module, configured to determine that the predicted SNR value of the first target vehicle is greater than a visible threshold.

[0327] One possible implementation further includes: a third determining module, configured to determine the sensor feature prediction value of the first target vehicle based on the sensor feature prediction value of the first candidate vehicle and the sensor feature prediction value of the second candidate vehicle; the first target vehicle includes the first candidate vehicle and the second candidate vehicle; the first candidate vehicle and the second candidate vehicle satisfy the following: the relative position of the first position to the second position is less than the first position threshold; the first position is the position of the first candidate target vehicle relative to the simulated vehicle, and the second position is the position of the second candidate target vehicle relative to the simulated vehicle.

[0328] In one possible implementation, the first candidate vehicle and the second candidate vehicle further satisfy the following: the relative speed of the first speed to the second speed is less than a first speed threshold; the first speed is the speed of the first candidate target vehicle relative to the simulated vehicle, and the second speed is the speed of the second candidate target vehicle relative to the simulated vehicle.

[0329] In one possible implementation, the device further includes: a sensor model training module, the sensor model training module comprising:

[0330] An acquisition module is used to acquire measurement data from a sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature values ​​of the second target vehicle collected by the sensor; the sensor feature values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measurement vehicle, and the second target vehicle is a vehicle near the measurement vehicle;

[0331] The training module is used to train a sensor model based on the measurement data and the obtained annotation information of the sensor. The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information of the measuring vehicle. The input of the sensor model is the position information and speed information of the first target vehicle relative to the sensor and the annotation information. The output of the sensor model is the sensor feature prediction value of the first target vehicle.

[0332] It should be noted that the module division in the above embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional modules in each embodiment of this application can be integrated into one processing module, exist as separate physical entities, or have two or more modules integrated into one module. One or more of the above modules can be implemented using software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as central processing units (CPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs).

[0333] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0334] like Figure 10 The diagram shown is a structural schematic of a vehicle simulation device provided in this application. The vehicle simulation device 1000 includes a communication interface 1010, a processor 1020, and a memory 1030.

[0335] The communication interface 1010 and memory 1030 are interconnected with the processor 1020. Optionally, the communication interface 1010 and memory 1030 can be interconnected with the processor 1020 via a bus; the bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0336] The communication interface 1010 is used to implement communication within the vehicle simulation device. For example, the position and speed information of a first target vehicle relative to the simulated vehicle, as well as the road environment information of the simulated vehicle, are input into a sensor model to obtain the sensor feature prediction values ​​of the first target vehicle. The sensor feature prediction values ​​include at least one of the following: radar cross section (RCS) prediction value and signal-to-noise ratio (SNR) prediction value. The sensor model is used to simulate the sensors in the simulated vehicle. The first target vehicle is a vehicle in the test environment where the simulated vehicle is located. The position and speed information of the first target vehicle relative to the simulated vehicle, as well as the road environment information of the simulated vehicle, are determined based on the test environment. The sensor model is trained based on the sensor measurement data and labeled road environment information. The sensor feature prediction values ​​of the first target vehicle are input into the decision module of the simulated vehicle to obtain the simulation decision result of the simulated vehicle. The decision module is used to output a vehicle driving decision determined based on the sensor feature prediction values.

[0337] The communication interface 1010 can also be used to enable communication between the vehicle simulation device and other devices.

[0338] Processor 1020 is used to implement the above-mentioned Figures 4b to 8b For details on the vehicle simulation method shown above, please refer to the above. Figures 4b to 8bThe descriptions in the illustrated embodiments will not be repeated here. Optionally, the processor 1020 may be a central processing unit (CPU) or other hardware chip. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. When implementing the above functions, the processor 1020 may be implemented in hardware, or it may be implemented by hardware executing corresponding software.

[0339] The memory 1030 is used to store program instructions and data. Specifically, the program instructions may include program code, which includes instructions for computer operation. The memory 1030 may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. The processor 1020 executes the program stored in the memory 1030 and, through the aforementioned components, implements the above-described functions, thereby ultimately realizing the method provided in the above embodiments.

[0340] like Figure 11 The diagram shown is a structural schematic of a sensor simulation device provided in this application. The device may include an acquisition module and a training module. This device can be applied to a testing apparatus.

[0341] The acquisition module 1101 is used to acquire measurement data from the sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature measurement values ​​of the second target vehicle collected by the sensor; the sensor feature measurement values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measurement vehicle; the second target vehicle is a vehicle near the measurement vehicle;

[0342] The training module 1102 is used to train a sensor model based on the measurement data and the obtained annotation information of the sensor. The sample input of the sensor model is the position information, speed information and annotation information of the second target vehicle relative to the sensor. The output of the sensor model is the sensor feature prediction value of the second target vehicle. The sensor feature prediction value of the second target vehicle includes at least one of the following: RCS prediction value and SNR prediction value.

[0343] The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located.

[0344] It should be noted that the module division in the above embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional modules in each embodiment of this application can be integrated into one processing module, exist as separate physical entities, or have two or more modules integrated into one module. One or more of the above modules can be implemented using software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as central processing units (CPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs).

[0345] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0346] like Figure 12 The diagram shown is a schematic diagram of a sensor simulation device provided in this application. The sensor simulation device 1200 may include a communication interface 1210, a processor 1220, and a memory 1230.

[0347] The communication interface 1210 and memory 1230 are interconnected with the processor 1220. Optionally, the communication interface 1210 and memory 1230 can be interconnected with the processor 1220 via a bus; the bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0348] The communication interface 1210 can be used to enable communication between the sensor simulation device and other devices (e.g., the vehicle simulation device 1000). For example, it enables the vehicle simulation device to obtain a sensor model.

[0349] Processor 1220 is used to implement the above-mentioned Figure 4b For details on the simulation method of the sensor shown, please refer to the above. Figure 4b The descriptions in the illustrated embodiments will not be repeated here. Optionally, the processor 1220 may be a central processing unit (CPU) or other hardware chip. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. When implementing the above functions, the processor 1220 may be implemented in hardware, or it may be implemented by hardware executing corresponding software.

[0350] The memory 1230 is used to store program instructions and data. Specifically, the program instructions may include program code, which includes instructions for computer operation. The memory 1230 may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. The processor 1220 executes the program stored in the memory 1230 and, through the aforementioned components, implements the above-described functions, thereby ultimately realizing the method provided in the above embodiments.

[0351] This application provides a computer-readable storage medium including computer instructions that, when executed by a processor, cause the vehicle simulation device to perform any of the possible methods described in the above embodiments.

[0352] This application provides a computer-readable storage medium including computer instructions that, when executed by a processor, cause the sensor simulation device to perform any of the possible methods described in the above embodiments.

[0353] This application provides a computer program product that, when run on a processor, causes the vehicle simulation device to perform any of the possible methods described in the above embodiments.

[0354] This application provides a computer program product that, when run on a processor, causes the sensor simulation device to perform any of the possible methods described in the above embodiments.

[0355] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0356] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0357] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0358] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0359] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A simulation method of a vehicle, characterized by, include: The position and speed information of the first target vehicle relative to the simulated vehicle, as well as the road environment information of the simulated vehicle, are input into the sensor model to obtain the sensor feature prediction value of the first target vehicle; the sensor feature prediction value includes at least one of the following: radar cross section (RCS) prediction value and signal-to-noise ratio (SNR) prediction value; The sensor model is used to simulate the sensors in the simulated vehicle. The first target vehicle is a vehicle in the test environment where the simulated vehicle is located. The position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle are determined according to the test environment. The sensor model is trained based on the measurement data of the sensors and the labeled road environment information. The sensor feature prediction values ​​of the first target vehicle are input into the decision module of the simulated vehicle to obtain the simulation decision result of the simulated vehicle; wherein, the decision module is used to output the vehicle driving decision determined based on the sensor feature prediction values.

2. The method as described in claim 1, characterized in that, The first target vehicle is a vehicle within the detection range of the sensor, determined from among the candidate vehicles based on the position and speed information of the candidate vehicles relative to the simulated vehicle; the position and speed information of the candidate vehicles relative to the simulated vehicle are determined based on the test environment, and the candidate vehicles are vehicles in the test environment where the simulated vehicle is located.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: The predicted SNR value of the first target vehicle is determined to be greater than the visibility threshold.

4. The method as described in claim 1 or 2, characterized in that, The first target vehicle includes a first candidate vehicle and a second candidate vehicle; the sensor feature prediction value of the first target vehicle is determined based on the sensor feature prediction values ​​of the first candidate vehicle and the second candidate vehicle. The first candidate vehicle and the second candidate vehicle satisfy the condition that the relative position of the first position to the second position is less than a first position threshold. The first position is the position of the first candidate vehicle relative to the simulated vehicle, and the second position is the position of the second candidate vehicle relative to the simulated vehicle.

5. The method as described in claim 4, characterized in that, The first candidate vehicle and the second candidate vehicle also satisfy the following: The relative speed of the first speed to the second speed is less than the first speed threshold; the first speed is the speed of the first candidate vehicle relative to the simulated vehicle, and the second speed is the speed of the second candidate vehicle relative to the simulated vehicle.

6. The method as described in claim 1 or 2, characterized in that, The sensor model is trained based on the sensor's measurement data and labeled road environment information, and includes: Acquire measurement data from the sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature values ​​of the second target vehicle collected by the sensor; the sensor feature values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measuring vehicle, and the second target vehicle is a vehicle near the measuring vehicle; A sensor model is obtained by training based on the measurement data and the obtained annotation information of the sensor; the annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located; the input of the sensor model is the position information and speed information of the first target vehicle relative to the sensor and the annotation information, and the output of the sensor model is the sensor feature prediction value of the first target vehicle.

7. A sensor simulation method, characterized in that, include: Acquire sensor measurement data; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature measurement values ​​of the second target vehicle collected by the sensor; the sensor feature measurement values ​​include: radar cross section (RCS) measurement value and signal-to-noise ratio (SNR) measurement value; the sensor is located in the measuring vehicle; the second target vehicle is a vehicle near the measuring vehicle; The sensor model is trained based on the measurement data and the obtained annotation information of the sensor; the sample input of the sensor model is the position information, speed information and annotation information of the second target vehicle relative to the sensor, and the output of the sensor model is the sensor feature prediction value of the second target vehicle; the sensor feature prediction value of the second target vehicle includes at least one of the following: RCS prediction value and SNR prediction value; The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located.

8. A vehicle simulation device, characterized in that, include: The sensor feature prediction module is used to input the position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle into the sensor model to obtain the sensor feature prediction value of the first target vehicle; the sensor feature prediction value includes at least one of the following: radar cross section (RCS) prediction value and signal-to-noise ratio (SNR) prediction value; The sensor model is used to simulate the sensors in the simulated vehicle. The first target vehicle is a vehicle in the test environment where the simulated vehicle is located. The position and speed information of the first target vehicle relative to the simulated vehicle and the road environment information of the simulated vehicle are determined according to the test environment. The sensor model is trained based on the measurement data of the sensors and the labeled road environment information. An output module is used to input the sensor feature prediction values ​​of the first target vehicle into the decision module of the simulated vehicle to obtain the simulation decision result of the simulated vehicle; wherein, the decision module is used to output the vehicle driving decision determined based on the sensor feature prediction values.

9. The apparatus as claimed in claim 8, characterized in that, Also includes: The first determining module is used to determine the vehicle within the detection range of the sensor as the first target vehicle from among the candidate vehicles based on the position information and speed information of the candidate vehicle relative to the simulated vehicle; the position information and speed information of the candidate vehicle relative to the simulated vehicle are determined based on the test environment; the candidate vehicle is a vehicle in the test environment where the simulated vehicle is located.

10. The apparatus as claimed in claim 8 or 9, characterized in that, Also includes: The second determining module is used to determine that the predicted SNR value of the first target vehicle is greater than the visibility threshold.

11. The apparatus as claimed in claim 8 or 9, characterized in that, Also includes: The third determining module is used to determine the sensor feature prediction value of the first target vehicle based on the sensor feature prediction value of the first candidate vehicle and the sensor feature prediction value of the second candidate vehicle. The first target vehicle includes a first candidate vehicle and a second candidate vehicle; the first candidate vehicle and the second candidate vehicle satisfy the following condition: the relative position of the first position to the second position is less than a first position threshold; The first position is the position of the first candidate vehicle relative to the simulated vehicle, and the second position is the position of the second candidate vehicle relative to the simulated vehicle.

12. The apparatus as claimed in claim 11, characterized in that, The first candidate vehicle and the second candidate vehicle also satisfy the following: The relative speed of the first speed to the second speed is less than the first speed threshold; the first speed is the speed of the first candidate vehicle relative to the simulated vehicle, and the second speed is the speed of the second candidate vehicle relative to the simulated vehicle.

13. The apparatus as claimed in claim 8 or 9, characterized in that, Also includes: Sensor model training module, the sensor model training module includes: An acquisition module is used to acquire measurement data from a sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature values ​​of the second target vehicle collected by the sensor; the sensor feature values ​​include: RCS measurement value and SNR measurement value; the sensor is located in the measurement vehicle, and the second target vehicle is a vehicle near the measurement vehicle; The training module is used to train a sensor model based on the measurement data and the obtained annotation information of the sensor. The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information of the measuring vehicle. The input of the sensor model is the position information and speed information of the first target vehicle relative to the sensor and the annotation information. The output of the sensor model is the sensor feature prediction value of the first target vehicle.

14. A sensor simulation device, characterized in that, include: An acquisition module is used to acquire measurement data from a sensor; the measurement data includes: position information and speed information of the second target vehicle relative to the sensor, and sensor feature measurement values ​​of the second target vehicle collected by the sensor; the sensor feature measurement values ​​include: radar cross section (RCS) measurement value and signal-to-noise ratio (SNR) measurement value; the sensor is located in the measurement vehicle; the second target vehicle is a vehicle near the measurement vehicle; The training module is used to train a sensor model based on the measurement data and the obtained annotation information of the sensor. The sample input of the sensor model is the position information, speed information and annotation information of the second target vehicle relative to the sensor. The output of the sensor model is the sensor feature prediction value of the second target vehicle. The sensor feature prediction value of the second target vehicle includes at least one of the following: RCS prediction value and SNR prediction value. The annotation information includes at least one of the following: the yaw angle of the second target vehicle relative to the sensor, the road environment information annotated when the sensor collects data, and the vehicle information where the sensor is located.

15. A vehicle simulation device, characterized in that, include: Processor and interface circuitry; The processor is coupled to the memory via the interface circuit, and the processor is used to execute program code in the memory to implement the method as described in any one of claims 1-6.

16. A sensor simulation device, characterized in that, include: Processor and interface circuitry; The processor is coupled to the memory via the interface circuit, and the processor is used to execute program code in the memory to implement the method as described in claim 7.

17. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed by a processor, cause a vehicle simulation device to perform the method as described in any one of claims 1-6 or cause a sensor simulation device to perform the method as described in claim 7.

18. A vehicle-to-everything (V2X) communication system, characterized in that, A vehicle simulation device comprising an in-vehicle system and a vehicle as described in any one of claims 8-13 and 15, wherein the in-vehicle system is communicatively connected to the vehicle simulation device; or, A simulation device comprising an in-vehicle system and a sensor as described in claim 14 or 16, wherein the in-vehicle system is communicatively connected to the simulation device for the sensor.

19. A chip system, characterized in that, include: A processor is configured to invoke a computer program or computer instructions stored in memory, such that the processor executes the program code in the memory to implement the method as described in any one of claims 1-6 or the method as described in claim 7.

Images