A hardware-in-the-loop testing method, device and system of a controller

Abstract

The application relates to the technical field of automatic driving test, and discloses a hardware-in-the-loop test method, device and system of a controller, the method comprising the following steps: calibrating the perspective of a projection image based on a real image, the real image being a road image collected by a camera at a to-be-tested position on a real vehicle, and the projection image being a simulated road image used for display on an electronic display screen; controlling a darkroom camera in a video darkroom to shoot the projection image; and sending a collected video signal to a to-be-tested controller by the darkroom camera, so as to test the to-be-tested controller through the video signal. The application makes the test environment and test data more real, improves the accuracy of the video signal, and further improves the test reliability of the subsequent to-be-tested controller.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving testing technology, and more specifically to a hardware-in-the-loop testing method, apparatus, and system for a controller. Background Technology

[0002] Hardware-in-the-loop testing (HIL) in the field of intelligent driving refers to simulating vehicle signals and transmitting these simulated signals to the controller under test (DUT) to test its ability to make correct decisions upon receiving the simulated vehicle signals. Some DUT tests require road video signals captured by the vehicle. To simulate these signals, some testing systems use a video dark box. Specifically, an electronic display screen is mounted on one side of the dark box, and a camera is mounted on the other. The electronic display screen shows simulated road footage, while the camera on the other side captures images of the screen to simulate the vehicle's road video signals. This captured video signal is then transmitted to the DUT for testing. However, the video images captured by the camera in the dark box often differ from those captured by the camera on the vehicle in terms of angle, size, etc., leading to inaccurate test results from the DUT using inaccurate images. Summary of the Invention

[0003] In view of this, the present invention provides a hardware-in-the-loop testing method, apparatus and system for controllers to solve the problem of inaccurate video signals in hardware-in-the-loop testing.

[0004] In a first aspect, the present invention provides a hardware-in-the-loop testing method for a controller, the method comprising: calibrating the viewing angle of a projected image based on a real image, wherein the real image is a road image captured by a camera at the location under test on a real vehicle, and the projected image is a simulated road image for display on an electronic display screen; controlling a darkroom camera in a video darkroom to capture the projected image; and controlling the darkroom camera to send the captured video signal to the controller under test, so as to test the controller under test through the video signal.

[0005] Based on the above technical means, before the projected image is played on the electronic display screen, the simulated road image perspective is calibrated according to the shooting angle of the real camera on the real vehicle, so as to obtain the calibrated projected image. The projected image is then displayed on the electronic display screen. After that, the dark box camera is controlled to collect the calibrated projected image, so as to test the controller under test. By calibrating the projected image, the similarity between the projected image and the real vehicle shooting scene is further improved, thereby making the test environment and test data more realistic, improving the accuracy of the video signal, and thus improving the reliability of the subsequent test of the controller under test.

[0006] In some optional implementations, calibrating the viewing angle of the projected image based on the real image includes: creating a virtual space and setting a virtual vehicle model and a virtual calibration board in the virtual space. The virtual vehicle model includes a virtual camera, and the virtual calibration board is set at a position that the virtual camera can capture. Importing a real image, which includes a real calibration board, the real image and the virtual space image captured by the virtual camera are overlaid and compared. If the comparison result indicates that the virtual calibration board and the real calibration board do not overlap, the position of the virtual calibration board in the virtual space is adjusted, and the shooting parameters of the virtual camera are adjusted. When the comparison result indicates that the virtual calibration board and the real calibration board overlap, the shooting parameters of the virtual camera are calibrated. A virtual road environment is created. Based on the calibrated shooting parameters, the virtual camera is controlled to capture images of the virtual road environment to obtain the calibrated projected image.

[0007] Based on the aforementioned technical means, this invention sets up a virtual calibration board and a virtual vehicle model in a created virtual space, then imports real images taken by a real vehicle into it. Using the alignment of the real calibration board and the virtual calibration board in the real image as a standard, the position of the virtual calibration board and the shooting parameters of the virtual camera are adjusted. When the real calibration board and the virtual calibration board coincide, the viewpoint of the virtual camera can be considered consistent with the viewpoint of the real camera on the real vehicle, thus completing the calibration. The adjusted shooting parameters are then used to control the virtual camera to capture images of the virtual road environment, resulting in a more realistic road simulation image.

[0008] In some optional implementations, before testing the controller under test via video signal, the method further includes: acquiring preset lane line parameters; reading the captured lane line parameters from the video signal; calculating the parameter error between the captured lane line parameters and the preset lane line parameters; when the parameter error is greater than a preset error threshold, performing a secondary adjustment on the shooting parameters of the virtual camera; controlling the virtual camera to re-capture images of the virtual road environment based on the secondary adjusted shooting parameters to obtain a secondary adjusted projected image; and returning to the step of controlling the dark box camera in the video dark box to capture the projected image.

[0009] According to the above-mentioned technical means, after capturing the projected image using a darkroom camera, this invention does not directly perform testing. Instead, it measures the relevant lane line parameters in the video signal acquired by the darkroom camera based on image display software. Then, it compares the captured lane line parameters with the preset lane line parameters when creating the virtual scene, analyzes the magnitude of the error, and if the error is within the allowable range, subsequent testing can be performed. If the error is large, the shooting parameters of the virtual camera are adjusted a second time, and the video signal is re-acquired until the error between the captured lane line parameters and the preset lane line parameters is reduced to the allowable range. This secondary fine-tuning process further calibrates the projected image and improves the accuracy of the video signal acquired by the darkroom camera.

[0010] In some alternative implementations, the darkroom camera is mounted in the video darkroom via an adjustable base mechanism. Before testing the controller under test via video signal, the method further includes: adjusting the adjustable base mechanism according to the shooting effect of the video signal to change the position of the darkroom camera; and returning to the step of controlling the darkroom camera in the video darkroom to capture a projected image.

[0011] Based on the aforementioned technical means, the darkroom camera is installed in the video darkroom through an adjustable base mechanism. When the shooting effect is not clear enough or the angle is not good enough, the position of the darkroom camera in the darkroom can be adjusted through the adjustable base mechanism, thereby improving the image shooting clarity and further improving the display effect of the video signal.

[0012] In some alternative implementations, adjusting the adjustable base mechanism according to the shooting effect of the video signal includes: setting a virtual obstacle in the projected image and recording the set distance from the virtual obstacle to the virtual camera; measuring a first observation distance from the virtual obstacle to the darkroom camera in the video signal; and adjusting the adjustable base mechanism according to a first error between the set distance and the first observation distance.

[0013] According to the above-mentioned technical means, after the projected image is calibrated once by adjusting the shooting parameters of the virtual camera, the adjustment of the dark box camera by the adjustable base mechanism can be regarded as a second calibration. This invention sets virtual obstacles in the projected image, and the setting distance between the virtual obstacles and the virtual camera is known. Then, the video signal obtained by the dark box camera from the projected image is used to measure the first observation distance from the virtual obstacle to the dark box camera. By comparing the first observation distance and the setting distance, if the two are almost the same, it means that the front and rear position of the dark box camera is not much different from the position of the camera on the real vehicle. If the two distances differ greatly, it means that the front and rear position of the dark box camera is significantly different from that of the camera on the real vehicle. The front and rear position of the dark box camera can be further changed by the adjustable base mechanism, thereby further improving the accuracy of the video signal.

[0014] In some optional implementations, the method further includes: measuring a second observation distance from the virtual obstacle to the darkroom camera from the received video signal using a test program of the controller under test; comparing the set distance and the second observation distance; controlling the controller under test to continue performing the test when the second error between the set distance and the second observation distance is less than a preset error threshold; and returning to the step of calibrating the viewing angle of the projected image based on the real image when the second error between the set distance and the second observation distance is greater than or equal to the preset error threshold.

[0015] Based on the above technical means, during the test, the observation distance and set distance of the virtual obstacle are compared again using the controller under test. The accuracy of the calibration is further determined based on the output result of the controller under test. If it is accurate, the test continues; otherwise, the aforementioned calibration is repeated, which improves the reliability of the calibration.

[0016] Secondly, the present invention provides a hardware-in-the-loop testing device for a controller, the device comprising: a projection image calibration module for calibrating the viewing angle of a projection image based on a real image, wherein the real image is a road image captured by a camera at the test location on a real vehicle, and the projection image is a simulated road image for display on an electronic display screen; a shooting module for capturing the projection image using a darkroom camera in a video darkroom; and a data transmission module for controlling the darkroom camera to send the captured video signal to the controller under test, so as to test the controller under test through the video signal.

[0017] Thirdly, the present invention provides a hardware-in-the-loop testing system for a controller, comprising: a host computer, a real-time machine, a video dark box, an electronic display screen, a dark box camera, and a controller under test; the host computer is used to perform the step of calibrating the viewing angle of the projected image based on the real image, and to control the real-time machine; the real-time machine is used to output virtual vehicle signals; the video dark box is used to provide a simulated road image acquisition environment, the electronic display screen and the dark box camera are respectively installed at both ends of the video dark box, the electronic display screen is used to display the simulated road image, and the dark box camera is used to capture the simulated road image and send the captured video signal to the controller under test.

[0018] In some alternative implementations, the system also includes an adjustable base mechanism, through which the darkroom camera is mounted in a video darkroom, the adjustable base mechanism being used to adjust the position of the darkroom camera in the video darkroom.

[0019] In some alternative implementations, the real-time machine is equipped with CAN boards, analog boards, and digital boards.

[0020] The technical solution provided by this invention has the following advantages:

[0021] (1) According to the above technical means, before the projected image is played on the electronic display screen, the simulated road image perspective is calibrated according to the shooting angle of the real camera on the real vehicle to obtain the calibrated projected image. The projected image is then displayed on the electronic display screen. After that, the dark box camera is controlled to collect the calibrated projected image to test the controller under test. By calibrating the projected image, the similarity between the projected image and the real vehicle shooting scene is further improved, thereby making the test environment and test data more realistic, improving the accuracy of the video signal, and thus improving the reliability of the subsequent test of the controller under test.

[0022] (2) Based on the above technical means, this invention sets up a virtual calibration board and a virtual vehicle model in the created virtual space, and then imports real images taken by real vehicles into it. Taking the overlap of the real calibration board and the virtual calibration board in the real image as the standard, the position of the virtual calibration board and the shooting parameters of the virtual camera are adjusted. When the real calibration board and the virtual calibration board overlap, the viewpoint of the virtual camera can be considered to be consistent with the viewpoint of the real camera on the real vehicle, thereby completing the calibration. Then, by using the adjusted shooting parameters to control the virtual camera to capture images of the virtual road environment, a more realistic road simulation image can be obtained.

[0023] (3) According to the above technical means, after the present invention captures the projected image by the dark box camera, it does not directly conduct the test. Instead, it measures the relevant shooting lane line parameters in the acquired video signal based on the image display software. Then, it compares the shooting lane line parameters with the preset lane line parameters when creating the virtual scene and analyzes the magnitude of the error between the two. If the error is within the allowable range, subsequent tests can be carried out. If the error is large, the shooting parameters of the virtual camera are adjusted a second time, and the video signal is re-acquired until the error between the shooting lane line parameters and the preset lane line parameters is reduced to the allowable range. Thus, the projected image is further calibrated through the process of secondary fine-tuning, thereby improving the accuracy of the video signal acquired by the dark box camera.

[0024] (4) According to the above technical means, the dark box camera is installed in the video dark box through an adjustable base mechanism. When the shooting effect is not clear enough or the angle is not good enough, the position of the dark box camera in the dark box can be adjusted through the adjustable base mechanism, thereby improving the image shooting clarity and further improving the display effect of the video signal.

[0025] (5) According to the above technical means, after the projection image is calibrated once by adjusting the shooting parameters of the virtual camera, the adjustment of the dark box camera by the adjustable base mechanism can be regarded as a second calibration. In this invention, virtual obstacles are set in the projection image, the setting distance between the virtual obstacles and the virtual camera is known, and the first observation distance from the virtual obstacle to the dark box camera is measured by the video signal obtained by the dark box camera from the projection image. By comparing the first observation distance and the setting distance, if the two are almost the same, it means that the front and back positions of the dark box camera and the virtual camera are not much different. If the two distances are much different, it means that the front and back positions of the dark box camera and the virtual camera are much different. The front and back positions of the dark box camera can be further changed by the adjustable base mechanism, thereby further improving the accuracy of the video signal.

[0026] (6) Based on the above technical means, during the test, the observation distance and setting distance comparison process of the virtual obstacle are repeated again using the controller under test. The accuracy of the calibration is further determined based on the output result of the controller under test. If it is accurate, the test continues; otherwise, the aforementioned calibration is repeated, which improves the reliability of the calibration. Attached Figure Description

[0027] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the hardware-in-the-loop test system for a controller according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic flowchart of a hardware-in-the-loop testing method for a controller according to an embodiment of the present invention;

[0030] Figure 3 This is another schematic flowchart of a hardware-in-the-loop testing method for a controller according to an embodiment of the present invention;

[0031] Figure 4 This is another schematic flowchart of a hardware-in-the-loop testing method for a controller according to an embodiment of the present invention;

[0032] Figure 5 This is another schematic diagram of a hardware-in-the-loop test system for a controller according to an embodiment of the present invention;

[0033] Figure 6This is a schematic diagram of the hardware-in-the-loop testing device for a controller according to an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] According to embodiments of the present invention, a hardware-in-the-loop testing system for a controller is provided, such as... Figure 1 As shown, it includes: a host computer, a real-time machine, a video darkroom, an electronic display screen, a darkroom camera, and a controller under test.

[0036] The host computer, which can be a personal computer, is the central hub for generating and executing a series of control strategies. The strategies it executes include, in subsequent method embodiments, calibrating the viewing angle of the projected image based on real images, generating virtual projected images, and inputting these images to the electronic display screen. The real-time machine (RTM) contains simulation software that generates virtual signals to simulate the vehicle's dynamic response, such as throttle and brake inputs. These virtual signals are then input to the controller under test (DUT) for testing. The RTM and DUT are connected via hardwired connections. Because the RTM's operation is more complex, it is typically a more powerful computer, such as a server. The RTM and host computer communicate via a network cable. The video darkroom provides the environment for acquiring the projected images, blocking ambient light to prevent interference with the electronic display screen's display. The electronic display screen and the darkroom camera are installed at opposite ends of the video darkroom, with their center lines aligned. The electronic display screen is connected to the host computer via an LVDS cable to display the projected images generated by the host computer. The dark box camera is a real camera that is connected to the controller under test via a connection cable to capture projected images and send the captured video signals to the controller under test to provide test data.

[0037] In some alternative implementations, the real-time machine is equipped with a CAN board, an analog board, and a digital board. The CAN board is used to simulate the CAN signals required by the system, the analog board is used to simulate some analog signals required by the controller under test, and the digital board simulates the switching and control signals required by the controller under test.

[0038] In some optional embodiments, an adjustable base mechanism is also installed in the video dark box. The dark box camera is installed in the video dark box through the adjustable base mechanism. In some specific embodiments, the adjustable base mechanism includes, but is not limited to, a six-degree-of-freedom adjustment platform, a three-degree-of-freedom gimbal bracket, etc. The adjustable base mechanism is mainly used to adjust the position of the dark box camera in the video dark box, including forward, backward, left, right, up, down and in-place rotation angle. Any mechanism that can realize the adjustment of the position of the dark box camera can be used. The embodiments of the present invention do not impose special limitations on the specific structure of the mechanism. For example, the six-degree-of-freedom adjustment platform and the three-degree-of-freedom gimbal bracket used are existing technologies and will not be described in detail in the present invention.

[0039] According to an embodiment of the present invention, a hardware-in-the-loop testing method embodiment for a controller is also provided, which is applied to the above-described hardware-in-the-loop testing system. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] This embodiment provides a hardware-in-the-loop testing method for a controller. Figure 2 This is a flowchart of a hardware-in-the-loop testing method for a controller according to an embodiment of the present invention. The process includes the following steps:

[0041] Step S201: The viewing angle of the projected image is calibrated based on the real image. The real image is a road image captured by a camera at the location to be tested on a real vehicle, and the projected image is a simulated road image used to be displayed on an electronic display screen.

[0042] Step S202: Control the camera in the video dark box to capture the projected image;

[0043] Step S203: Control the dark box camera to send the collected video signal to the controller under test, so as to test the controller under test through the video signal.

[0044] Specifically, during hardware-in-the-loop testing, the controller under test (DUT) needs to control a dark box camera to capture a projected image on an electronic display screen. The projected image is a simulated road image generated by a host computer. After the dark box camera captures the projected image on the electronic display screen, the captured video signal is transmitted to the DUT to "fool" the DUT into thinking that the received video signal is the real road scene captured by the vehicle's camera, thus enabling subsequent testing.

[0045] Therefore, to make the projected image on the electronic display screen more realistic, this invention calibrates the simulated road image perspective using real images captured by a real camera on a real vehicle before displaying the projected image. By adjusting the distance, angle, and field of view of the projected image, the visual effect of the projected image is made almost identical to the real image, thus obtaining the calibrated projected image. This embodiment then displays the calibrated projected image on the electronic display screen, and subsequently controls a darkroom camera to capture the calibrated projected image, thereby testing the controller under test. This invention further improves the similarity between the projected image and the real vehicle shooting scene through projection image calibration, thereby making the test environment and test data more realistic, improving the accuracy of the video signal, and ultimately improving the reliability of subsequent testing of the controller under test.

[0046] In some alternative implementations, such as Figure 3 As shown, step S201 above includes:

[0047] Step a1: Create a virtual space and set up a virtual vehicle model and a virtual calibration board in the virtual space. The virtual vehicle model includes a virtual camera, and the virtual calibration board is set in a position that the virtual camera can capture.

[0048] Step a2: Import the real image, which includes the real calibration board;

[0049] Step a3: Overlay and compare the real image and the virtual space image captured by the virtual camera;

[0050] Step a4: If the comparison result indicates that the virtual calibration board and the real calibration board do not overlap, then adjust the position of the virtual calibration board in the virtual space and adjust the shooting parameters of the virtual camera.

[0051] Step a5: When the comparison result indicates that the virtual calibration board and the real calibration board coincide, the shooting parameter calibration of the virtual camera is completed;

[0052] Step a6: Create a virtual road environment;

[0053] Step a7: Based on the calibrated shooting parameters, control the virtual camera to capture images of the virtual road environment and obtain the calibrated projected image.

[0054] Specifically, the image calibration method provided by this invention aims to improve the realism of projected images, making the display angle of the projected image closer to the shooting angle of an external camera on a real vehicle. First, this invention creates a virtual space on a host computer. This virtual space can be understood as a 3D animation space. Then, a virtual vehicle model and a virtual calibration board are set in the virtual space. The virtual vehicle model is a 3D model created based on the structural parameters of a real vehicle. The virtual calibration board is a virtual board with some special markings drawn on it, such as six square markings in different positions. This is just an example and not a limitation.

[0055] The virtual vehicle model includes a virtual camera, which is a 3D model created by imitating the structural parameters of a camera on a real vehicle. It is used to simulate the shooting function of a real vehicle camera, and the virtual calibration board is placed in a position that the virtual camera can capture.

[0056] Before calibration, real images are required. Real images refer to images of a real vehicle taken by its onboard camera, including a real calibration board. The real calibration board is a real-life board with special markings drawn on its surface, such as six square markings in different positions, and the same size as the virtual calibration board.

[0057] During calibration, the distance between the virtual calibration board and the virtual vehicle model is first roughly set, ensuring that the virtual camera can capture images of the virtual calibration board. The virtual camera's resolution needs to be adjusted to match that of the real camera. The virtual camera capturing images of the virtual calibration board will produce a virtual space image. Then, the real image is imported into the virtual space. Since both the virtual and real images contain images of the calibration board, they are overlaid for comparison. Because the placement of the virtual calibration board and the shooting parameters of the virtual camera differ from those in the real world, the calibration board in the real and virtual space images cannot perfectly overlap. Therefore, this embodiment of the invention fine-tunes the placement of the virtual calibration board and the shooting parameters of the virtual camera.

[0058] The shooting parameters of the virtual camera include, but are not limited to, intrinsic and extrinsic parameters such as the virtual camera's pitch angle, horizontal position, and focal length. It's important to note that the relative distance data between the real calibration board and the actual vehicle is based on the rear axle center of the actual vehicle as the origin of the coordinate axis, while the origin of the virtual vehicle model's coordinate axis is at the intersection of the rear center axis and the ground. Therefore, after coordinate data conversion, the position of the virtual calibration board is adjusted according to the coordinate error. By repeatedly adjusting the position of the virtual calibration board and the shooting parameters of the virtual camera, and continuously acquiring virtual space images through the virtual camera until the virtual calibration board in the virtual space image and the real calibration board in the real image completely overlap, it can be considered that the shooting parameters of the virtual camera, such as angle, position, and focal length, are consistent with those of the real camera on the actual vehicle. This results in more accurate virtual space images and completes the viewpoint calibration of the projected image.

[0059] After calibration, the virtual space image captured at this time still contains a virtual calibration board and cannot be directly displayed on the electronic display screen. This image is not directly used as a simulated road image for testing. Therefore, it is necessary to create a virtual road environment and then use the aforementioned calibrated shooting parameters to control the virtual camera to capture images of the virtual road environment. The captured images are calibrated projection images, which are the images that need to be displayed on the electronic display screen.

[0060] The solution provided in this invention involves setting up a virtual calibration board and a virtual vehicle model in a created virtual space, then importing real images taken by a real vehicle into the space. Using the alignment of the real calibration board and the virtual calibration board in the real image as a standard, the position of the virtual calibration board and the shooting parameters of the virtual camera are adjusted. When the real and virtual calibration boards align, the viewpoint of the virtual camera is considered to be consistent with the viewpoint of the real camera on the real vehicle, thus completing the calibration. The adjusted shooting parameters are then used to control the virtual camera to capture images of the virtual road environment, resulting in a more realistic road simulation image and improving the accuracy of the images displayed on the electronic display screen.

[0061] Specifically, such as Figure 4 As shown, in some optional embodiments, before performing tests on the controller under test, the hardware-in-the-loop testing method for a controller provided by the present invention further includes the following steps:

[0062] Step b1: Obtain the preset lane line parameters;

[0063] Step b2: Read the lane line parameters from the video signal;

[0064] Step b3: Calculate the parameter error between the captured lane line parameters and the preset lane line parameters;

[0065] Step b4: When the parameter error exceeds the preset error threshold, the shooting parameters of the virtual camera are adjusted a second time.

[0066] Step b5: Based on the second-adjusted shooting parameters, control the virtual camera to re-capture images of the virtual road environment to obtain the second-adjusted projected image;

[0067] Step b6 returns to the step of controlling the darkroom camera in the video darkroom to capture the projected image.

[0068] Specifically, after capturing the projected image using a darkroom camera, this invention does not directly perform testing. Instead, it provides a method for calibrating and fine-tuning the projected image. First, the acquired video signal is displayed on another device using image display software, such as... Figure 5 As shown, image display software is software installed on external smart devices and used to display and edit video signals.

[0069] The image display software measures relevant lane line parameters in the acquired video signal using a lane line extraction algorithm (the lane line extraction algorithm is existing technology, and its implementation principle will not be elaborated here). The lane line parameters measured in this invention include, but are not limited to, lane line width and the distance between lane lines. Since the lane line parameters in the virtual space have preset values ​​when creating a virtual scene, this embodiment compares the captured lane line parameters with the preset lane line parameters when creating the virtual scene, and analyzes the magnitude of the error. If the error is within the preset allowable range, it indicates that the projected image is displayed well on the electronic display screen, and subsequent testing can be performed. If the error between the captured lane line parameters and the preset lane line parameters is large, it indicates that the projected image is not displayed well on the electronic display screen. This embodiment will perform a secondary adjustment on the shooting parameters of the virtual camera, which is a small-amplitude fine-tuning, and then re-acquire the video signal until the error between the captured lane line parameters and the preset lane line parameters is reduced to the allowable range. Through the technical solution provided by this embodiment, the projected image is further calibrated based on the secondary fine-tuning process, improving the accuracy of the video signal acquired by the darkroom camera.

[0070] In some alternative implementations, the darkroom camera is mounted in a video darkroom via an adjustable base mechanism, and the system further includes the following features before testing the controller under test via video signals:

[0071] Step c1: Adjust the adjustable base mechanism according to the shooting effect of the video signal to change the position of the darkroom camera;

[0072] Step c2 returns to the step of controlling the darkroom camera in the video darkroom to capture the projected image.

[0073] Specifically, the darkroom camera is installed in the video darkroom through an adjustable base mechanism. When the shooting effect is not clear enough or the angle is not good enough, the position of the darkroom camera in the darkroom can be adjusted through the adjustable base mechanism, thereby improving the image shooting clarity and further improving the display effect of the video signal.

[0074] In some alternative implementations, step c1 above includes:

[0075] Step d1: Set up virtual obstacles in the projected image and record the set distance between the virtual obstacles and the virtual camera;

[0076] Step d2: Measure the first observation distance from the virtual obstacle in the video signal to the dark box camera;

[0077] Step d3: Adjust the adjustable base mechanism based on the first error between the set distance and the first observation distance.

[0078] Specifically, in this embodiment of the invention, after calibrating the projected image by adjusting the shooting parameters of the virtual camera, a second calibration can be performed by changing the position of the darkroom camera using an adjustable base mechanism, thereby obtaining a more accurate video signal. The specific steps are as follows: The invention sets virtual obstacles in the projected image, and the distance between the virtual obstacles and the virtual camera is known. The darkroom camera then captures the projected image to obtain a video signal. Image display software is used to measure the first observation distance from the virtual obstacle to the darkroom camera. Then, the first observation distance and the set distance are compared. If they are almost identical, it indicates that the front-to-back position of the darkroom camera is not significantly different from the set position of the virtual camera, and the darkroom camera does not need adjustment. If the first observation distance and the set distance differ significantly, it indicates that the front-to-back position of the darkroom camera and the virtual camera differs significantly. For example, the front-to-back position of the darkroom camera can be further changed by adjusting the six-degree-of-freedom platform. Then, the image is reshot, and the error between the first observation distance and the set distance is judged until the error is less than a preset error threshold, thereby further improving the accuracy of the video signal.

[0079] In some optional embodiments, the hardware-in-the-loop testing method for the controller provided by the present invention further includes:

[0080] Step e1: Measure the second observation distance from the virtual obstacle to the dark box camera from the received video signal using the test program of the controller under test;

[0081] Step e2: Compare the set distance and the second observation distance;

[0082] Step e3: When the second error between the set distance and the second observation distance is less than the preset error threshold, control the controller under test to continue to perform the test;

[0083] Step e4: When the second error between the set distance and the second observation distance is greater than or equal to the preset error threshold, return to the step of calibrating the viewing angle of the projected image based on the real image.

[0084] Specifically, in this embodiment of the invention, the observation distance and set distance comparison process of the virtual obstacle are repeated again using the controller under test during the testing process. The comparison principle is the same as that of the first observation distance comparison principle in the aforementioned embodiment, and will not be repeated here. Based on the CAN message output result of the controller under test, the accuracy of the secondary calibration is further determined. If it is accurate, the test continues; otherwise, the calibration is restarted. The solution provided by this embodiment of the invention further improves the reliability of the calibration.

[0085] This embodiment also provides a hardware-in-the-loop testing device for a controller, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0086] This embodiment provides a hardware-in-the-loop testing device for a controller, such as... Figure 6 As shown, it includes:

[0087] The projection image calibration module 601 is used to calibrate the viewing angle of the projection image based on the real image. The real image is a road image captured by a camera at the test position on a real vehicle, and the projection image is a simulated road image displayed on an electronic display screen.

[0088] The shooting module 602 is used to capture projected images using a darkroom camera in a video darkroom;

[0089] The data transmission module 603 is used to control the darkroom camera to send the acquired video signal to the controller under test, so as to test the controller under test through the video signal.

[0090] In some alternative implementations, the projection image calibration module 601 includes:

[0091] The virtual space creation unit is used to create a virtual space and set up a virtual vehicle model and a virtual calibration board in the virtual space. The virtual vehicle model includes a virtual camera, and the virtual calibration board is set at a position that the virtual camera can capture.

[0092] The real image import unit is used to import real images, including real calibration plates.

[0093] The image comparison unit is used to overlap and compare real images and virtual space images captured by a virtual camera;

[0094] The virtual parameter adjustment unit is used to adjust the position of the virtual calibration board in the virtual space and adjust the shooting parameters of the virtual camera if the comparison result shows that the virtual calibration board and the real calibration board do not overlap.

[0095] The virtual camera calibration completion unit is used to complete the shooting parameter calibration of the virtual camera when the comparison result indicates that the virtual calibration board and the real calibration board coincide.

[0096] The virtual road creation unit is used to create a virtual road environment;

[0097] The projection image generation unit is used to control the virtual camera to capture images of the virtual road environment based on the calibrated shooting parameters, and obtain the calibrated projection image.

[0098] In some alternative implementations, prior to the data transmission module 603, the following is also included:

[0099] A preset lane line parameter acquisition unit is used to acquire preset lane line parameters;

[0100] The lane line parameter acquisition unit reads the lane line parameters from the video signal.

[0101] The parameter error calculation unit is used to calculate the parameter error between the captured lane line parameters and the preset lane line parameters.

[0102] The virtual camera secondary adjustment unit is used to perform secondary adjustments on the shooting parameters of the virtual camera when the parameter error exceeds a preset error threshold.

[0103] The re-acquisition unit is used to control the virtual camera to re-acquire images of the virtual road environment based on the secondary adjusted shooting parameters, so as to obtain a secondary adjusted projected image.

[0104] The re-shooting unit is used to return to the step of capturing projected images by the darkroom camera in the control video darkroom.

[0105] In some alternative implementations, prior to the data transmission module 603, the following is also included:

[0106] The mechanism adjustment unit is used to adjust the adjustable base mechanism according to the shooting effect of the video signal to change the position of the darkroom camera;

[0107] The re-shooting unit is used to return to the step of capturing projected images by the darkroom camera in the control video darkroom.

[0108] In some alternative implementations, the mechanism adjustment unit includes:

[0109] The virtual obstacle unit is used to set virtual obstacles in the projected image and record the set distance between the virtual obstacle and the virtual camera;

[0110] The first observation unit is used to measure the first observation distance from the virtual obstacle in the video signal to the dark box camera.

[0111] The first comparison unit is used to adjust the adjustable base mechanism based on the first error between the set distance and the first observation distance.

[0112] In some alternative embodiments, the apparatus further includes:

[0113] The second observation unit is used to measure the second observation distance from the virtual obstacle to the dark box camera from the received video signal through the test program of the controller under test;

[0114] The second comparison unit is used to compare the set distance and the second observation distance;

[0115] The continued testing unit is used to control the controller under test to continue performing the test when the second error between the set distance and the second observation distance is less than a preset error threshold.

[0116] The recalibration unit is used to return to the step of calibrating the viewpoint of the projected image based on the real image when the second error between the set distance and the second observation distance is greater than or equal to a preset error threshold.

[0117] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0118] In this embodiment, the hardware-in-the-loop test device for the controller is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0119] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0120] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0121] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A hardware-in-the-loop testing method for a controller, characterized in that, The method includes: The viewing angle of the projected image is calibrated based on the real image, which is a road image captured by a camera at the test position on a real vehicle, and the projected image is a simulated road image to be displayed on an electronic display screen. The viewing angle calibration of the projected image based on the real image is used to adjust the shooting parameters of the virtual camera. Control the darkroom camera in the video darkroom to capture the projected image; The darkroom camera is controlled to send the acquired video signal to the controller under test, so as to test the controller under test through the video signal; Before testing the controller under test using the video signal, the method further includes: acquiring preset lane line parameters; reading the captured lane line parameters from the video signal; calculating the parameter error between the captured lane line parameters and the preset lane line parameters; when the parameter error is greater than a preset error threshold, performing a secondary adjustment on the shooting parameters of the virtual camera; controlling the virtual camera to re-capture images of the virtual road environment based on the secondary adjusted shooting parameters to obtain the secondary adjusted projected image; and returning to the step of controlling the darkroom camera in the video darkroom to capture the projected image. The dark box camera is mounted in the video dark box via an adjustable base mechanism. Before testing the controller under test using the video signal, the method further includes: adjusting the adjustable base mechanism according to the shooting effect of the video signal to change the position of the dark box camera; and returning to the step of controlling the dark box camera in the video dark box to capture the projected image. The step of adjusting the adjustable base mechanism according to the shooting effect of the video signal includes: setting a virtual obstacle in the projected image and recording the setting distance of the virtual obstacle to the virtual camera; measuring a first observation distance of the virtual obstacle to the darkroom camera in the video signal; and adjusting the adjustable base mechanism according to a first error between the setting distance and the first observation distance. The calibration of the viewing angle of the projected image based on the real image includes: Create a virtual space, and set up a virtual vehicle model and a virtual calibration board in the virtual space. The virtual vehicle model includes a virtual camera, and the virtual calibration board is set at a position that the virtual camera can capture. Import the real image, which includes a real calibration plate; The real image and the virtual space image captured by the virtual camera are overlaid and compared. If the comparison result indicates that the virtual calibration board and the real calibration board do not overlap, then the position of the virtual calibration board in the virtual space is adjusted, and the shooting parameters of the virtual camera are adjusted. When the comparison result indicates that the virtual calibration board and the real calibration board coincide, the shooting parameter calibration of the virtual camera is completed; Create a virtual road environment; Based on the calibrated shooting parameters, the virtual camera is controlled to capture images of the virtual road environment, thereby obtaining the calibrated projected image.

2. The method according to claim 1, characterized in that, The method further includes: The test program of the controller under test measures the second observation distance from the virtual obstacle to the dark box camera from the received video signal. Compare the set distance with the second observation distance; When the second error between the set distance and the second observation distance is less than a preset error threshold, the controller under test is controlled to continue performing the test; When the second error between the set distance and the second observation distance is greater than or equal to a preset error threshold, return to the step of calibrating the viewing angle of the projected image based on the real image.

3. A hardware-in-the-loop testing device for a controller, characterized in that, The device includes: The projection image calibration module is used to calibrate the viewing angle of the projection image based on a real image. The real image is a road image captured by a camera at the test location on a real vehicle. The projection image is a simulated road image displayed on an electronic screen. The calibration of the viewing angle of the projection image based on the real image is used to adjust the shooting parameters of the virtual camera. The calibration includes: creating a virtual space and setting a virtual vehicle model and a virtual calibration board in the virtual space. The virtual vehicle model includes a virtual camera, and the virtual calibration board is positioned where the virtual camera can capture images. The module also imports the... A real image, including a real calibration board, is used. The real image and a virtual space image captured by the virtual camera are overlaid and compared. If the comparison result indicates that the virtual calibration board and the real calibration board do not overlap, the position of the virtual calibration board in the virtual space is adjusted, and the shooting parameters of the virtual camera are adjusted. When the comparison result indicates that the virtual calibration board and the real calibration board overlap, the shooting parameters of the virtual camera are calibrated. A virtual road environment is created. Based on the calibrated shooting parameters, the virtual camera is controlled to capture images of the virtual road environment to obtain the calibrated projected image. The shooting module is used to capture the projected image using a darkroom camera in a video darkroom; A data transmission module is used to control the darkroom camera to send the acquired video signal to the controller under test (DUT) for testing the DUT via the video signal. Before testing the DUT via the video signal, the module further includes: acquiring preset lane line parameters; reading the captured lane line parameters from the video signal; calculating the parameter error between the captured lane line parameters and the preset lane line parameters; when the parameter error exceeds a preset error threshold, performing a secondary adjustment on the shooting parameters of the virtual camera; controlling the virtual camera to re-capture images of the virtual road environment based on the secondary adjusted shooting parameters to obtain the secondary adjusted projected image; and returning to control the darkroom camera in the video darkroom to capture the projected image. The steps include: the darkroom camera is mounted in the video darkroom via an adjustable base mechanism; before testing the controller under test using the video signal, the steps further include: adjusting the adjustable base mechanism according to the shooting effect of the video signal to change the position of the darkroom camera; returning to the step of controlling the darkroom camera in the video darkroom to capture the projected image; adjusting the adjustable base mechanism according to the shooting effect of the video signal includes: setting a virtual obstacle in the projected image and recording the set distance from the virtual obstacle to the virtual camera; measuring a first observation distance from the virtual obstacle to the darkroom camera in the video signal; and adjusting the adjustable base mechanism according to a first error between the set distance and the first observation distance.

4. A hardware-in-the-loop testing system for a controller, characterized in that, include: Host computer, real-time machine, video dark box, electronic display screen, dark box camera, controller under test; The host computer is used to execute the method as described in any one of claims 1-2, thereby calibrating the viewing angle of the projected image based on the real image, and controlling the real-time machine; The real-time machine is used to output virtual vehicle signals; The video dark box is used to provide an environment for acquiring projected images. The electronic display screen and the dark box camera are respectively installed at both ends of the video dark box. The electronic display screen is used to display the projected images, and the dark box camera is used to capture the projected images and send the captured video signals to the controller under test.

5. The system according to claim 4, characterized in that, It also includes an adjustable base mechanism, through which the darkroom camera is mounted in the video darkroom, and the adjustable base mechanism is used to adjust the position of the darkroom camera in the video darkroom.

6. The system according to claim 4, characterized in that, The real-time machine is equipped with a CAN board, an analog quantity board, and a digital quantity board.

Images