An in-the-loop test system for edge scene detection of vehicle-mounted cameras

Abstract

The present invention relates to a vehicle-mounted camera test platform and method, and in particular to an in-loop test system for vehicle-mounted camera edge scene detection, including a test platform, a six-degree-of-freedom motion platform, a display, and a meteorological simulation system, wherein the meteorological simulation system includes a graded rainfall simulation system, a lighting simulation system, and a fog simulation system; the test experimental steps are divided into three parts, namely, test platform calibration, edge scene test, and test data acquisition. The present invention can simulate multiple meteorological conditions at the same time, provide more abundant test scenes for vehicle-mounted cameras, perform edge scene tests on vehicle-mounted cameras through 5 different test levels, and improve the accuracy and reliability of camera tests. The platform of the present invention provides more accurate and abundant data information for vehicle-mounted camera function developers through the obtained functional parameters, thereby updating the functions of smart car vehicle-mounted cameras and the edge scenes corresponding to the functions to meet the actual smart car perception requirements.

Description

Technical Field

[0001] The present invention relates to a vehicle-mounted camera testing platform and method, and in particular to an in-loop testing system for vehicle-mounted camera edge scene detection. Background Art

[0002] At present, the testing of the image perception part of smart cars is still in the research stage. Major suppliers and research institutions mainly conduct real car road testing and simulation testing for the image perception part of smart cars. Edge scenes refer to scene libraries used to test and verify specific smart car functions. Edge scenes for safety, functionality, and comfort are of great significance to the development of smart cars. For traditional road tests, the frequency of edge scenes is extremely low, resulting in low test efficiency. At the same time, a large amount of test economic cost and time are required for test experiments, and the experimental repeatability is low. For simulation tests, the low degree of reality leads to low confidence in the test results. Therefore, there is a need for an edge scene test platform based on a combination of high-confidence virtual simulation scenes and actual physical environments, and a method for testing the functional boundaries of smart car on-board cameras in combination with the test platform is needed to meet actual needs. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides an in-loop test system for edge scene detection of a vehicle-mounted camera, comprising a test platform, a longitudinal motion slide rail is provided at one end of the test platform, a six-degree-of-freedom motion platform is provided on the longitudinal motion slide rail, the six-degree-of-freedom motion platform is slidably connected to the longitudinal motion slide rail, and the vehicle-mounted camera is arranged on the six-degree-of-freedom motion platform; a display is provided at the other end of the test platform; a weather simulation system is provided between the vehicle-mounted camera and the display, and automotive-grade silicon glass is provided between the vehicle-mounted camera and the weather simulation system; high-transmittance glass is provided between the weather simulation system and the display, and the automotive-grade silicon glass, the high-transmittance glass, and the side panels and top panels of the test platform are enclosed to form a sealed simulation cabin;

[0004] The meteorological simulation system includes a hierarchical rainfall simulation system, a light simulation system and a fog simulation system;

[0005] The hierarchical rainfall simulation system comprises a water spray pipeline, a water tank, a water pump, a rainfall controller and an optical rainfall sensor. Several water spray pipelines are arranged horizontally on the top of the simulation cabin along the platform, and the water spray pipelines are parallel to each other to simulate several levels of rainfall; a plurality of nozzles arranged equidistantly are arranged at the bottom of each water spray pipeline; the water tank is arranged outside the simulation cabin, and a drain is arranged at the bottom of the simulation cabin, which is connected to the water tank through a pipeline; the water outlet of the water tank is connected to the water pump, and the water pump is connected to the rainfall controller through a water pipe, and the rainfall controller is connected to the water spray pipeline through a water pipe; the optical rainfall sensor is arranged at the bottom of the simulation cabin on the same side of the vehicle-mounted camera, and is connected to the rainfall controller through a line; the optical rainfall sensor is used to feedback whether the experimental conditions meet the specified requirements of the experiment, so as to improve the confidence of the experimental platform.

[0006] The illumination simulation system comprises a light emitting diode solar simulator, a light intensity tester and a light controller. The light emitting diode solar simulator is arranged on the top of the simulation cabin on the display side, and the light intensity tester is arranged on the bottom of the simulation cabin on the same side as the vehicle-mounted camera. The light emitting diode solar simulator and the light intensity tester are respectively connected to the light controller through lines; the light emitting diode solar simulator is arranged above the display to simulate the real illumination angle, and the light intensity tester is used to feedback whether the experimental conditions meet the specified requirements of the experiment, so as to improve the confidence of the experimental platform.

[0007] The fog simulation system includes a fog controller, a visibility tester, and a fog water tank. The fog water tank is arranged outside the simulation cabin. A fog inlet is arranged on the side wall of the simulation cabin. The fog water tank is connected to the fog controller through a pipeline. The fog controller is connected to the fog inlet on the side wall of the simulation cabin through a pipeline. The visibility tester is arranged at the bottom of the simulation cabin on the same side as the vehicle-mounted camera and is connected to the fog controller through a line. The fog controller forms artificial fog and changes the humidity in the simulation cabin. Different humidity will simulate different fog visibilities. The visibility tester is used to feedback whether the experimental conditions meet the specified requirements of the experiment and improve the confidence of the platform.

[0008] The six-degree-of-freedom motion platform includes an upper platform, a lower platform, six retractable electric cylinders and a motion controller. Both the upper platform and the lower platform are regular hexagonal platforms. The six electric cylinders are evenly spaced between the upper platform and the lower platform and connected by ball joints. The six electric cylinders are connected to the motion controller through lines respectively. Under the control of the motion controller, the six electric cylinders drive the six electric cylinders to perform lateral, longitudinal, vertical, pitch, yaw and roll motion in space. The posture information comes from the vehicle pitch angle, roll angle and yaw angle output by the vehicle dynamics software. The posture simulation of the six-degree-of-freedom motion platform obtains six spatial posture parameters [x, y, z, α, β, γ] through the vehicle dynamics software;

[0009] First, establish a position coordinate system, including a static coordinate system (lower platform) and a dynamic coordinate system (upper platform). The center point of the lower platform is taken as the origin of the coordinate system O. The OX and OY axes are respectively in the plane. The OX axis is perpendicular to the display plane and points forward. The OZ axis is perpendicular to the plane and points upward. The OY axis is parallel to the display plane and determines its direction according to the right-hand rule. The six hinge points of the lower platform are (D1, D2, ..., D6). According to the same principle, establish a dynamic coordinate system O′-X′Y′Z′ on the upper platform. The OX′ axis is perpendicular to the display plane and points forward. The OZ′ axis is perpendicular to the plane and points upward. The OY′ axis is parallel to the display plane and determines its direction according to the right-hand rule. The six hinge points of the upper platform are (U1, U2, ..., U6). At the same time, the Euler angles around each coordinate axis are specified when the rigid body rotates. The rotation angles around the X′, Y′ and Z′ axes are α, β and γ.

[0010] The matrix represents the position change in the moving coordinate system when the Z axis rotates clockwise by an angle of γ:

[0011]

[0012] Similarly, we get the rotation matrix expression around the y-axis and x-axis, and get the rotation change matrix by matrix multiplication:

[0013]

[0014] Substitute [T] into the following formula and obtain the coordinates of the six hinge points in the static coordinate system of the upper platform through the conversion formula.

[0015]

[0016] Where (x s ′,y s ′,z s ′) are the dynamic coordinates of the six hinge points on the upper platform, and (x, y, z) are the coordinates of the hinge points in the dynamic coordinate system;

[0017] set up is the original position of the six hinge points of the lower platform in the static coordinate system. In order to achieve the angle required by vehicle dynamics, the spatial length L of each electric cylinder is solved. i (i=1,2,…,6):

[0018] The position and posture simulation of the vehicle-mounted camera is completed by changing the length of the corresponding electric cylinder.

[0019] The motion controller controls the extension and retraction of the six electric cylinders of the six-degree-of-freedom motion platform according to the input motion information such as the pitch and roll of the vehicle, ensuring that the on-board camera simulates the real motion posture; the host computer is installed with scene simulation software, which provides a high-confidence driving perspective of the on-board camera and plays it on the display to provide an image input source for the on-board camera; at the same time, the meteorological simulation system quantitatively determines the meteorological conditions according to the test requirements to simulate the real meteorological weather information; the edge scenes of the on-board camera are mainly divided into 5 different test levels, namely pixel level, image domain level, object level, image level and scene level.

[0020] The test experiment steps are divided into three parts: test platform calibration, edge scenario testing and test data collection.

[0021] Part 1: Test platform calibration. Test platform calibration mainly includes platform layout calibration, vehicle-mounted camera calibration, and weather simulation system calibration.

[0022] Platform layout calibration: At the beginning of the experiment, adjust the center point of the image captured by the on-board camera to align with the center point of the display, adjust the position of the six-degree-of-freedom platform on the longitudinal motion rail, and change the distance between the on-board camera and the display to meet the image capture conditions of the on-board camera during actual vehicle driving; on-board camera calibration: image dedistortion; meteorological simulation system calibration includes: graded rainfall simulation system calibration, illumination simulation system calibration and fog simulation system calibration;

[0023] (1) Calibration of the graded rainfall simulation system: The simulated rainfall intensity can be adjusted in several levels from 15 to 105 mm / hour, and different numbers of parallel rain levels are used to quantitatively evaluate the edge scenes of the vehicle camera function;

[0024] (2) Lighting simulation system calibration: The simulated illumination is adjustable from 5 to 2500 lux, which is used to quantitatively determine the edge scenes under the lighting conditions of the vehicle-mounted camera;

[0025] (3) Fog simulation system calibration: The artificial fog visibility can be adjusted in several levels from 15 to 100 meters. The edge scene of the vehicle camera can be quantitatively evaluated by simulating fog with different visibility.

[0026] Part II: Edge scene test. The present invention divides the vehicle camera edge scene test into five different test levels according to the increasing complexity of the image scene data corresponding to the vehicle camera:

[0027] Level 1, pixel-level edge test: mainly tests the scenarios where the pixel values ​​of the vehicle camera exceed the normal value limit in a large range and in a small range at the same time. The pixel-level edge test is achieved through exposure simulation of the lighting simulation system, partial image dead point simulation in the display, and partial stains on the automotive-grade silicon glass.

[0028] Level 2, image domain-level edge test: The semantics of the content in the image has not changed, but the background domain of the object has changed. For example, the algorithm test of the same object in rainy days, foggy days and nights can obtain the edge scene of the vehicle camera. At the same time, based on the adjustable function of the platform's meteorological simulation system, the functional edge of the vehicle camera is quantitatively evaluated; for the image domain-level edge test, the vehicle camera scenes in various image domains are tested by simulating rainy weather, dense foggy weather and displaying night scenes, and the edge scenes for the vehicle camera are quantitatively analyzed.

[0029] Level 3, object-level edge testing: images that the image recognition algorithm has never been trained on or known objects appear in the image but are partially occluded.

[0030] Level 4, image-level edge testing: At this level, all objects are considered in the image processing algorithm training set, but appear in unconventional positions or in unconventional poses within the data acquisition range of the vehicle camera.

[0031] Level 5, scenario-level edge testing: (1) Scenarios that were not considered in their entirety during the design process and that are somewhat dangerous; (2) Scenarios that were not considered in their entirety during the design process but are not dangerous; (3) Scenarios that were considered during the design process but are somewhat dangerous. Tests whether the response and perception functions of the on-board camera can meet the functional requirements within the time range of each scenario.

[0032] Part III: Data collection and result analysis. In a simulated environment, the effects of different test levels on the functions of the vehicle camera are recorded. The recorded data include the categories of objects recognized by the vehicle camera, the number of objects recognized by the vehicle camera, the distance of objects recognized by the vehicle camera, the delay from the appearance of the object in the image capture range to accurate recognition by the vehicle camera, and the IOU value between the vehicle camera recognition object frame and the real manually calibrated recognition frame. At the same time, the values ​​of the optical rain sensor, visibility tester and light intensity tester are recorded to ensure the confidence of the experiment.

[0033] Beneficial effects of the present invention:

[0034] The present invention can simulate multiple meteorological conditions at the same time, providing a richer test scene for the vehicle-mounted camera. The present invention performs edge scene tests on the vehicle-mounted camera through 5 different test levels to improve the accuracy and reliability of the camera test. The platform of the present invention provides more accurate and rich data information for the vehicle-mounted camera function developers through the obtained functional parameters, and then updates the smart car vehicle-mounted camera function and the edge scene corresponding to the function to meet the actual smart car perception requirements. Experiments are designed for the updated vehicle-mounted camera function to iteratively test the smart car vehicle-mounted camera function. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The structure of the test platform of the present invention is shown in FIG. Figure 1 ;

[0036] Figure 2 The structure of the test platform of the present invention is shown in FIG. Figure 2 ;

[0037] Figure 3 The figure is a schematic diagram of the test experiment flow of the test platform of the present invention.

[0038] 1. Test platform 2. Longitudinal motion slide rail 3. Six-degree-of-freedom motion platform 4. On-board camera 5. Display 6. Automotive-grade silicon glass 7. High-transmittance glass 8. Simulation cabin 9. Graded rainfall simulation system 10. Light simulation system 11. Fog simulation system 12. Water spray pipeline 13. Water tank 14. Water pump 15. Rainfall controller 16. Optical rainfall sensor 17. Nozzle 18. Drain 19. Filter 20. LED solar simulator 21. Light intensity tester 22. Light controller 23. Fog water tank 24. Fog controller 25. Visibility tester 26. Fog inlet 27. Upper platform 28. Lower platform 29. Electric cylinder. DETAILED DESCRIPTION

[0039] See also Figure 1-3 As shown:

[0040] The present invention provides an in-loop test system for edge scene detection of a vehicle-mounted camera, comprising a test platform 1, wherein a longitudinal motion slide rail 2 is provided at one end of the test platform 1, a six-degree-of-freedom motion platform 3 is provided on the longitudinal motion slide rail 2, the six-degree-of-freedom motion platform 3 is slidably connected to the longitudinal motion slide rail 2, and a vehicle-mounted camera 4 is arranged on the six-degree-of-freedom motion platform 3; a display 5 is provided at the other end of the test platform 1, the display 5 adopts a high-definition 144HZ display 5, and ensures that the image displayed by the display 5 during the platform calibration process can meet the capture requirements of the vehicle-mounted camera 4; a meteorological simulation system is provided between the vehicle-mounted camera 4 and the display 5, and an automotive-grade silicon glass 6 is provided between the vehicle-mounted camera 4 and the meteorological simulation system to simulate the blurred state of the windshield of the real vehicle-mounted camera 4 during data collection, so as to improve the confidence of the test platform 1; a high-transmittance glass 7 is provided between the meteorological simulation system and the display 5, and the automotive-grade silicon glass 6, the high-transmittance glass 7 and the side panels and top panels of the test platform 1 form a sealed simulation cabin 8;

[0041] The meteorological simulation system includes a hierarchical rainfall simulation system 9, a light simulation system 10 and a fog simulation system 11;

[0042] The graded rainfall simulation system 9 includes a water spray pipeline 12, a water tank 13, a water pump 14, a rainfall controller 15 and an optical rainfall sensor 16. Nine water spray pipelines 12 are arranged horizontally on the top of the simulation cabin 8 along the platform, that is, perpendicular to the vehicle-mounted camera lens; and the nine water spray pipelines 12 are parallel to each other, which can simulate 9 levels of rainfall; the lower part of each water spray pipeline 12 is provided with a plurality of nozzles 17 arranged at equal distances; the water tank 13 is arranged outside the simulation cabin 8, and a drain port 18 is provided at the bottom of the simulation cabin 8, and the drain port 18 is connected to the simulation cabin 8 through a pipeline. It is connected to a water tank 13; the water outlet of the water tank 13 is connected to a water pump 14, the water pump 14 is connected to a rainfall controller 15 through a water pipe, and the rainfall controller 15 is connected to a water spray pipeline 12 through a water pipe; filters 19 are respectively provided on the water inlet pipeline and the water outlet pipeline of the water tank 13; the optical rainfall sensor 16 is arranged at the bottom of the simulation cabin 8 on the same side of the vehicle-mounted camera 4, and is connected to the rainfall controller 15 through a line; the optical rainfall sensor 16 is used to feedback whether the experimental conditions meet the specified requirements of the experiment, so as to improve the confidence of the experimental platform.

[0043] The light simulation system 10 includes a light emitting diode solar simulator 20, a light intensity tester 21 and a light controller 22. The light emitting diode solar simulator 20 is arranged on the top of the simulation cabin 8 on the side of the display 5, and the light intensity tester 21 is arranged on the bottom of the simulation cabin 8 on the same side of the vehicle-mounted camera 4. The light emitting diode solar simulator 20 and the light intensity tester 21 are respectively connected to the light controller 22 through lines; the main light-emitting component of the light-emitting diode is composed of a PN junction. When a forward voltage is applied to the PN junction, electrons move and recombine at the junction to release a large amount of energy, which is released in the form of light. The characteristics of the light emitting diode solar simulator 20 are that it forms a wide spectrum range, is safe, stable and controllable; the light emitting diode solar simulator 20 is arranged above the display 5 to simulate the actual light angle, and the light intensity tester 21 is used to feedback whether the experimental conditions meet the specified requirements of the experiment, thereby improving the confidence of the experimental platform.

[0044] The fog simulation system 11 includes a fog water tank 23, a fog controller 24, and a visibility tester 25. The fog water tank 23 is arranged outside the simulation cabin 8, and a fog inlet 26 is arranged on the side wall of the simulation cabin 8. The fog water tank 23 is connected to the fog controller 24 through a pipeline. A filter 19 is also arranged on the pipeline between the fog water tank 23 and the fog controller 24. The fog controller 24 is connected to the fog inlet 26 on the side wall of the simulation cabin 8 through a pipeline; the visibility tester 25 is arranged at the bottom of the simulation cabin 8 on the same side of the vehicle-mounted camera 4, and is connected to the fog controller 24 through a line; the fog controller 24 pressurizes the filtered water to 5 to 8 MPa, and the atomized water particles enter the simulation cabin 8 through the pipeline. At the same time, the liquid water undergoes a physical state change to a gaseous state, forming an artificial fog phenomenon, and the humidity in the simulation cabin 8 changes. Different humidity will simulate different fog visibility. The visibility tester 25 is used to feedback whether the experimental conditions meet the specified requirements of the experiment and improve the platform confidence.

[0045] The six-degree-of-freedom motion platform 3 includes an upper platform 27, a lower platform 28, six retractable electric cylinders 29 and a motion controller. The upper platform 27 and the lower platform 28 are both regular hexagonal platforms. The six electric cylinders 29 are evenly and equidistantly arranged between the upper platform 27 and the lower platform 28, and are connected to the six corners of the regular hexagonal platform through ball joints. The six electric cylinders 29 are respectively connected to the motion controller through lines, and are controlled by the motion controller. Driven by the six electric cylinders 29, the six electric cylinders 29 perform lateral, longitudinal, vertical, pitch, yaw and roll motion in space. The posture information comes from the vehicle pitch angle, roll angle and yaw angle output by the vehicle dynamics software. The posture simulation of the six-degree-of-freedom motion platform 3 used belongs to the mechanism position inverse solution process, that is, the input constructed position is inversely solved according to the position and posture obtained by vehicle dynamics, and the six spatial posture parameters [x, y, z, α, β, γ] are obtained through the vehicle dynamics software;

[0046] First, a position coordinate system is established, including a static coordinate system (lower platform 28) and a dynamic coordinate system (upper platform 27). The center point of the lower platform 28 is used as the origin O of the coordinate system. The OX and OY axes are respectively in the plane, the OX axis direction is perpendicular to the display plane and faces forward, the OZ axis direction is perpendicular to the plane and faces upward, and the OY axis direction is parallel to the display plane and determines the direction according to the right-hand rule; the six hinge points of the lower platform are (D1, D2, ..., D6); according to the same principle, a dynamic coordinate system O′-X′Y′Z′ is established on the upper platform 27; the OX′ axis direction is perpendicular to the display plane and faces forward, the OZ′ axis direction is perpendicular to the plane and faces upward, and the OY′ axis direction is parallel to the display plane and determines the direction according to the right-hand rule; the six hinge points of the upper platform are (U1, U2, ..., U6); at the same time, the Euler angles around each coordinate axis when the rigid body rotates in posture are specified, and the rotation angles around the X′, Y′ and Z′ axes are α, β and γ;

[0047] The matrix represents the position change in the moving coordinate system when the Z axis rotates clockwise by an angle of γ:

[0048]

[0049] Similarly, we get the rotation matrix expression around the y-axis and x-axis, and get the rotation change matrix by matrix multiplication:

[0050]

[0051] Substitute [T] into the following formula and obtain the coordinates of the six hinge points in the static coordinate system of the upper platform through the conversion formula.

[0052]

[0053] Where (x s ′,y s ′,z s ′) are the dynamic coordinates of the six hinge points on the upper platform, and (x, y, z) are the coordinates of the hinge points in the dynamic coordinate system;

[0054] set up is the original position of the six hinge points of the lower platform in the static coordinate system. In order to achieve the angle required by vehicle dynamics, the spatial length L of each electric cylinder 29 is solved. i (i=1,2,…,6):

[0055]

[0056] The position and posture simulation of the vehicle-mounted camera 4 is completed by changing the length of the corresponding electric cylinder 29 .

[0057] The lower computer is installed with vehicle dynamics software to calculate the vehicle dynamics model in real time and transmit the vehicle's lateral and longitudinal control information to the upper computer. At the same time, the vehicle dynamics software calculates the vehicle's pitch, roll and other motion information and transmits it to the motion controller of the six-degree-of-freedom motion platform 3 to control the extension and retraction of the six electric cylinders 29 to ensure that the vehicle-mounted camera 4 simulates the real motion posture; the upper computer is installed with scene simulation software, which provides a high-confidence driving perspective of the vehicle-mounted camera 4 and plays it on the display 5 to provide an image input source for the vehicle-mounted camera 4; at the same time, the meteorological simulation system quantitatively determines the meteorological conditions according to the test requirements to simulate the real meteorological weather information; the edge scene of the vehicle-mounted camera 4 is mainly divided into 5 different test levels, namely pixel level, image domain level, object level, image level and scene level.

[0058] The test experiment steps are divided into three parts, namely test platform 1 calibration, edge scene testing and test data collection.

[0059] Part 1: Calibration of test platform 1. The calibration of test platform 1 mainly includes platform layout calibration, vehicle-mounted camera 4 calibration, and weather simulation system calibration.

[0060] The purpose of platform layout calibration is to ensure that the positions of the tested on-board camera 4, display 5 and the spatial layout of the meteorological simulation system meet the test requirements, to ensure that the center point of the camera image captured is aligned with the center point of the display 5 at the beginning of the experiment, to adjust the position of the six-degree-of-freedom platform on the longitudinal motion slide rail 2, and to change the distance between the on-board camera 4 and the display 5 to meet the image capture conditions of the on-board camera 4 during the actual vehicle driving process; the on-board camera 4 calibration is to remove distortion and ensure that the imaging results are directly used for the development and testing of image processing algorithms; the meteorological simulation system calibration is a meteorological simulation system for quantitatively evaluating the edge scenes of the on-board camera 4, which includes three subsystems: a hierarchical rainfall simulation system 9, a lighting simulation system 10 and a fog simulation system 11;

[0061] The graded rainfall simulation system 9 can simulate 9 levels of rainfall intensity of 15-105 mm / hour and is adjustable. Different numbers of parallel rain levels are used to quantitatively evaluate the functional edge scenes of the vehicle-mounted camera 4. For example, the pedestrian information in the display 5 can be correctly identified and the pedestrian trajectory prediction function can be completed when there are 2 layers of rain levels. However, when the number of rain levels increases to 3 layers, the vehicle-mounted camera 4 cannot correctly identify pedestrians. This scene corresponds to the edge scene of the vehicle-mounted camera 4 under this experimental condition. The rainfall intensity is obtained through the optical rainfall sensor 16 to ensure the platform confidence and ensure that the experimental conditions meet the experimental requirements.

[0062] The lighting simulation system 10 can simulate illuminance in 6 levels, from 5 to 2500 lux, and is used to quantitatively determine the edge scene of the vehicle-mounted camera 4 under lighting conditions. For example, it can quantitatively determine the edge scene of the vehicle-mounted camera 4 under exposure conditions that simulate the situation when a vehicle enters a normal lighting environment from a dark tunnel, that is, when the exposure intensity is 1700 lux, a short exposure will make the smart car's vehicle-mounted camera 4 unable to complete the lane line recognition function; and the platform confidence is guaranteed by the light intensity tester 21, while ensuring that the experimental conditions meet the experimental requirements.

[0063] The fog simulation system 11 satisfies the artificial fog visibility of 15-100 meters and is adjustable in 5 levels. It quantitatively evaluates the edge scene of the vehicle-mounted camera 4 by simulating fog with different visibilities. For example, the object to be tested is an active pedestrian avoidance system based on an image recognition algorithm. When the fog condition of the simulated scene reaches 70 meters, the system failure rate in the virtual traffic environment is higher than the functional design requirement, that is, the vehicle-mounted camera 4 cannot provide correct perception information in this scene, and the avoidance system fails. The platform confidence is ensured by the visibility tester 25, and at the same time, the experimental conditions are ensured to meet the experimental requirements.

[0064] Part II: Edge scene test. The present invention divides the edge scene test of the vehicle-mounted camera 4 into five different test levels (as shown in Table 1) according to the order of increasing complexity of the image scene data corresponding to the vehicle-mounted camera 4:

[0065] Level 1, pixel-level edge test: mainly tests the scene when the pixel values ​​of the vehicle-mounted camera 4 exceed the normal value limit at the same time in a large range and in a small range. The pixel-level edge test is implemented by exposure simulation of the illumination simulation system 10, partial image dead point simulation in the display 5, and partial stains on the automotive-grade silicon glass 6. For example, the scene of exposure phenomenon caused by sudden sunlight irradiation on the vehicle-mounted camera 4 is simulated to test the function of the vehicle-mounted camera 4, and the edge scene, that is, the exposure intensity at which the camera function fails at a certain lux value is explored to obtain a quantitative analysis of the impact of exposure intensity on the function of the vehicle-mounted camera 4.

[0066] Level 2, image domain level edge test: the semantics of the content in the image has not changed, but the background domain where the object is located has changed. For example, the algorithm test of the same object in rainy days, foggy days and at night can obtain the edge scene of the vehicle camera 4. At the same time, based on the adjustable function of the platform's meteorological simulation system, the functional edge of the vehicle camera 4 can be quantitatively evaluated. For the image domain level edge test, this platform can test the vehicle camera 4 scenes in various image domains and quantitatively analyze the edge scenes of the vehicle camera 4 by simulating rainy weather, dense foggy weather and displaying night scenes on the display 5. For example, the platform can simulate real rainy weather and test the target recognition algorithm for the vehicle camera 4 under test. The experimental results show that when the rainfall reaches 70 mm / hour, the vehicle camera 4 completely loses its recognition and perception function.

[0067] Level 3, object-level edge test: images that the image recognition algorithm has never been trained on or known objects that are partially blocked appear in the image; since the image recognition algorithm cannot meet the information recognition of all objects during training, this test mainly targets the generalization ability of the perception algorithm unit of the vehicle camera 4. For example, pedestrians walking on the road are blocked by vehicles. The blocked scene was not considered when the image recognition algorithm was designed. The response of the vehicle camera 4 function in this scene is tested and suggestions are provided for future development and design.

[0068] Level 4, image-level edge test: In this level, all objects are considered in the image processing algorithm training set, but appear in unconventional positions or in unconventional postures within the data collection range of the vehicle-mounted camera 4. This occurs frequently in actual scenarios and is challenging for smart car functions. For example, a large number of pedestrians cross the road, a traffic accident occurs in the middle of the road, and the front of the car reverses. The vehicle-mounted camera 4 is tested to have corresponding functional responses such as distance information.

[0069] Level 5, scene-level edge testing: Compared with image-level edge testing, this level of testing also takes the time dimension into account. It can be divided into three categories: (1) The design process does not consider the complete scene and there is a certain risk, such as a pedestrian suddenly appearing in the image captured by the vehicle-mounted camera 4; (2) The design process does not consider the complete scene but there is no risk, such as normal traffic at an intersection and the vehicle in the lateral lane brakes and stops; (3) The scene is considered in the design process but there is a certain risk, such as overtaking on the highway. Although it is fully considered during the function development process, it still has a certain risk. Test whether the response and perception function of the vehicle-mounted camera 4 can meet the functional requirements within the time range of the scene.

[0070] Part III: Data collection and result analysis. In a simulated environment, the effects of different test levels on the functions of the vehicle camera 4 are recorded. The recorded data include but are not limited to the categories of objects recognized by the vehicle camera 4, the number of objects recognized by the vehicle camera 4, the distance of objects recognized by the vehicle camera 4, the delay from the appearance of the object in the image capture range to the accurate recognition by the vehicle camera 4, the IOU value between the object recognition frame of the vehicle camera 4 and the real artificial calibration recognition frame, and other functional parameters. Specific data collection information can be designed according to the corresponding test data collection items for the functions described by the developer. At the same time, the values ​​of the optical rain sensor 16, the visibility tester 25 and the light intensity tester 21 are recorded to ensure the confidence of the experiment.

[0071] Table 1 Five different test levels for edge scenes of vehicle-mounted cameras

[0072]

[0073]

Claims

1. An in-the-loop test system for edge scene detection of a vehicle-mounted camera, characterized in that: It comprises a test platform, one end of which is provided with a longitudinal motion slide rail, a six-degree-of-freedom motion platform is provided on the longitudinal motion slide rail, the six-degree-of-freedom motion platform is slidably connected with the longitudinal motion slide rail, and a vehicle-mounted camera is provided on the six-degree-of-freedom motion platform; a display is provided at the other end of the test platform; a weather simulation system is provided between the vehicle-mounted camera and the display, and automotive-grade silicon glass is provided between the vehicle-mounted camera and the weather simulation system; high-transmittance glass is provided between the weather simulation system and the display, and the automotive-grade silicon glass, the high-transmittance glass, the side panels and the top panel of the test platform form a sealed simulation cabin; The meteorological simulation system includes a hierarchical rainfall simulation system, a light simulation system and a fog simulation system; The motion controller controls the six-degree-of-freedom motion platform to change its position according to the input vehicle pitch and roll information to simulate the real motion position of the vehicle camera; the simulation scene is played on the display to provide an image input source for the vehicle camera; at the same time, the weather simulation system quantitatively determines the weather conditions according to the test requirements to simulate the real weather information; The test experiment steps are divided into three parts: test platform calibration, edge scenario testing and test data collection; Part 1: Test platform calibration: Test platform calibration mainly includes platform layout calibration, vehicle-mounted camera calibration, and weather simulation system calibration; Platform layout calibration: At the beginning of the experiment, adjust the center point of the image captured by the on-board camera to align with the center point of the display, adjust the position of the six-degree-of-freedom platform on the longitudinal motion rail, and change the distance between the on-board camera and the display to meet the image capture conditions of the on-board camera during actual vehicle driving; Vehicle camera calibration: image dedistortion; meteorological simulation system calibration includes: (1) graded rainfall simulation system calibration: meet the simulated rainfall intensity of 15-105 mm / hour, which can be adjusted in several levels, and quantitatively evaluate the edge scenes of the vehicle camera function through different numbers of parallel rain levels; (2) illumination simulation system calibration: meet the simulated illumination of 5-2500 lux, which can be adjusted in several levels, and is used to quantitatively determine the edge scenes under the illumination conditions of the vehicle camera; (3) fog simulation system calibration: meet the artificial fog visibility of 15-100 meters, which can be adjusted in several levels, and quantitatively evaluate the edge scenes of the vehicle camera by simulating fog with different visibility; Part 2: Edge scenario testing: divided into 5 different test levels: Level 1, pixel-level edge test: mainly tests scenarios where the vehicle camera's pixel values ​​exceed the normal value limit in a large range and in a small range at the same time. The implementation methods include: exposure simulation through the lighting simulation system, partial image dead spot simulation in the display, and partial stains on the automotive-grade silicon glass; Level 2, image domain-level edge test: The semantics of the content in the image has not changed, but the background domain of the object has changed, including the same object target algorithm test in rainy days, foggy days and at night, to obtain the edge scene of the vehicle camera; at the same time, based on the adjustable function of the weather simulation system, quantitatively evaluate the functional edge of the vehicle camera; for the image domain-level edge test, the vehicle camera scenes in various image domains are tested by simulating rainy weather, dense foggy weather and displaying night scenes, and the edge scenes for the vehicle camera are quantitatively analyzed; Level 3, object-level edge testing: images that the image recognition algorithm has never been trained on or known objects appear in the image but are partially occluded; Level 4, image-level edge testing: In this level, all objects are considered in the image processing algorithm training set, but appear in unconventional locations or in unconventional poses within the data collection range of the vehicle camera; Level 5, scenario-level edge testing: (1) Scenarios that were not considered in their entirety during the design process and that are somewhat dangerous; (2) Scenarios that were not considered in their entirety during the design process but are not dangerous; (3) Scenarios that were considered during the design process but are somewhat dangerous; Tests whether the response and perception functions of the on-board camera can meet the functional requirements within the time range of each scenario; Part III: Data collection and result analysis: In a simulated environment, the effects of different test levels on the functions of the vehicle camera are recorded. The recorded data include the category of objects recognized by the vehicle camera, the number of objects recognized by the vehicle camera, the distance at which the vehicle camera recognizes objects, the delay from the object appearing in the image capture range to accurate recognition, and the IOU value between the vehicle camera's object recognition frame and the real manually calibrated recognition frame; at the same time, the values ​​of the optical rain sensor, visibility tester, and light intensity tester are recorded.

2. The in-the-loop test system for edge scene detection of a vehicle-mounted camera according to claim 1, characterized in that: The hierarchical rainfall simulation system comprises a water spray pipeline, a water tank, a water pump, a rainfall controller and an optical rainfall sensor. Several water spray pipelines are arranged horizontally on the top of the simulation cabin along the platform, and the water spray pipelines are parallel to each other to simulate several levels of rainfall; the lower part of each water spray pipeline is provided with several nozzles arranged at equal intervals; the water tank is arranged outside the simulation cabin, and a drain is arranged at the bottom of the simulation cabin, and the drain is connected to the water tank through a pipeline; the water outlet of the water tank is connected to the water pump, and the water pump is connected to the rainfall controller through a water pipe, and the rainfall controller is connected to the water spray pipeline through a water pipe; the optical rainfall sensor is arranged at the bottom of the simulation cabin on the same side of the vehicle-mounted camera, and is connected to the rainfall controller through a line; The light simulation system includes a light emitting diode solar simulator, a light intensity tester and a light controller. The light emitting diode solar simulator is arranged on the top of the simulation cabin on the display side, and the light intensity tester is arranged on the bottom of the simulation cabin on the same side as the vehicle-mounted camera. The light emitting diode solar simulator and the light intensity tester are respectively connected to the light controller through lines; The light emitting diode solar simulator is arranged above the display to simulate the real lighting angle; The fog simulation system comprises a fog making controller, a visibility tester and a fog making water tank. The fog making water tank is arranged outside the simulation cabin. A fog inlet is arranged on the side wall of the simulation cabin. The fog making water tank is connected to the fog making controller through a pipeline. The fog making controller is connected to the fog inlet on the side wall of the simulation cabin through a pipeline. The visibility tester is arranged at the bottom of the simulation cabin on the same side as the vehicle-mounted camera and is connected to the fog making controller through a line. The fog making controller forms artificial fog.

3. The in-the-loop test system for edge scene detection of a vehicle-mounted camera according to claim 1, characterized in that: The six-degree-of-freedom motion platform comprises an upper platform, a lower platform, six retractable electric cylinders and a motion controller. Both the upper platform and the lower platform are regular hexagonal platforms. The six electric cylinders are evenly and equidistantly arranged between the upper platform and the lower platform and connected by ball joints. The six electric cylinders are respectively connected to the motion controller through lines. Under the control of the motion controller, the six electric cylinders drive the six electric cylinders to perform lateral, longitudinal, vertical, pitch, yaw and roll motion in space. The posture information comes from the pitch angle, roll angle and yaw angle of the vehicle output by the vehicle dynamics software. The posture simulation of the six-degree-of-freedom motion platform obtains six spatial posture parameters [x, y, z, α, β, γ] through the vehicle dynamics software; First, establish a position coordinate system, including a static coordinate system of the lower platform and a dynamic coordinate system of the upper platform. The center point of the lower platform is taken as the origin O of the coordinate system. The OX and OY axes are respectively in the plane. The OX axis is perpendicular to the display plane and forward, the OZ axis is perpendicular to the plane and upward, and the OY axis is parallel to the display plane and the direction is determined according to the right-hand rule. The six hinge points of the lower platform are (D1, D2, ..., D6). According to the same principle, a dynamic coordinate system O'-X'Y'Z' is established on the upper platform. The OX' axis is perpendicular to the display plane and forward, the OZ' axis is perpendicular to the plane and upward, and the OY' axis is parallel to the display plane and the direction is determined according to the right-hand rule. The six hinge points of the upper platform are (U1, U2..., U6). At the same time, the Euler angles around each coordinate axis when the rigid body rotates are specified, and the rotation angles around the X', Y' and Z' axes are α, β and γ. The matrix represents the position change in the moving coordinate system when the Z axis rotates clockwise by an angle of γ: ； Similarly, we get the rotation matrix expression around the y-axis and x-axis, and get the rotation change matrix by matrix multiplication: ； Substitute [T] into the following formula and obtain the coordinates of the six hinge points in the static coordinate system of the upper platform through the conversion formula. : ； Where (x s ',y s ',z s ') are the dynamic coordinates of the six hinge points on the upper platform, and (x, y, z) are the coordinates of the hinge points in the dynamic coordinate system; set up is the original position of the six hinge points of the lower platform in the static coordinate system. In order to achieve the angle required by vehicle dynamics, the spatial length L of each electric cylinder is solved. i (i=1,2,…,6): ； The position and posture simulation of the vehicle-mounted camera is completed by changing the length of the corresponding electric cylinder.

Images