Vehicle-mounted camera simulation method and device, computer device and storage medium
By generating simulation images of vehicle-mounted cameras, the problem of accurate determination of camera placement was solved, achieving both accuracy and diversity in camera placement, and improving debugging efficiency and the accuracy of simulation results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FOSS (HANGZHOU) INTELLIGENT TECH CO LTD
- Filing Date
- 2022-09-15
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technology cannot accurately determine whether the placement of vehicle cameras meets imaging requirements, making it impossible to judge the imaging effect of camera placement in the early stages of vehicle design, thus affecting the accuracy and diversity of camera placement.
By acquiring simulation data from vehicle-mounted cameras, including environmental data and camera parameters, adjusting the relative position between the cameras and the vehicle, and performing distortion processing, an interference simulation diagram between the vehicle-mounted camera's field of view and the vehicle is generated. Combined with visible points, vehicle body contour points, and lane points, a detailed simulation diagram is formed to facilitate the assessment of the accuracy and diversity of camera placement.
It improves the accuracy and diversity of camera placement, reduces the workload of real vehicle testing, increases camera debugging efficiency, and verifies the accuracy of results through simulation image algorithms.
Smart Images
Figure CN115641378B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of camera simulation technology, and in particular to a method, apparatus, computer equipment, and storage medium for simulating vehicle-mounted cameras. Background Technology
[0002] With the development of technology and increasingly stringent laws and regulations, coupled with people's growing emphasis on automotive safety features, more and more car manufacturers are equipping their vehicles with in-vehicle cameras. As the number of in-vehicle cameras increases, their placement becomes particularly important.
[0003] The placement of cameras is determined based on data collection needs. However, each vehicle model has a unique shape, and a single specification cannot meet the needs of all vehicles. Therefore, for models that cannot meet the placement requirements, it is impossible to make accurate judgments based on the interference of the field of view in the early stages. Furthermore, at the initial stage of vehicle design, without a physical vehicle, it is impossible to see the imaging effect of the camera at that location, or to determine whether the imaging at that location meets the requirements of the algorithm. Usually, only when an actual car is available can the image presented by the camera be seen. However, the placement of the camera determines whether the shape can be frozen, and only after the shape is frozen can real-vehicle testing be conducted. Therefore, the initial placement of the camera is extremely important.
[0004] Therefore, an auxiliary system is needed to make accurate judgments by restoring the image quality of the camera. Summary of the Invention
[0005] Therefore, it is necessary to provide a vehicle-mounted camera simulation method, device, computer equipment, and storage medium that can improve the accuracy and diversity of camera placement to address the aforementioned technical problems.
[0006] Firstly, this application provides a method for simulating an in-vehicle camera, the method comprising:
[0007] Acquire simulation data from the vehicle-mounted camera. The simulation data includes at least environmental data and camera parameters. The environmental data includes at least the vehicle's body outline data and visible point data to assist in the simulation.
[0008] The vehicle body contour points are set based on the vehicle body contour data, and auxiliary visible points in the field of view of the vehicle camera are set based on the visible point data.
[0009] Adjust the relative position between the vehicle-mounted camera and the vehicle according to the camera parameters;
[0010] The coordinates of visible points and vehicle body contour points are normalized, and distortion processing is performed on visible points and vehicle body contour points according to camera parameters;
[0011] Based on the relative position between the vehicle-mounted camera and the vehicle, and combined with the visible points and vehicle body contour points after distortion processing, a first simulation image of the interference between the field of view of the vehicle-mounted camera and the vehicle is obtained. The vehicle body contour points are connected to form the vehicle body contour envelope in the first simulation image.
[0012] In one embodiment, the environmental data further includes lane data, and the method further includes:
[0013] The lane point is set based on the lane data;
[0014] Adjust the relative positions of the vehicle-mounted camera, lane, and vehicles according to the camera parameters;
[0015] The coordinates of the lane points are normalized, and distortion processing is performed on the lane points based on the camera parameters;
[0016] Based on the relative positions of the vehicle camera, lane, and vehicle, and combined with the visible points, lane points, and vehicle body contour points after distortion processing, a first simulation image of the interference between the vehicle camera's field of view and the vehicle is obtained. In this first simulation image, lane points are connected to form lane lines.
[0017] In one embodiment, the method further includes determining the accuracy of the vehicle camera simulation by comparing the differences in lane lines in the first simulation image and the actual test results.
[0018] In one embodiment, the camera parameters include external parameters and internal parameters, and adjusting the relative positions between the vehicle-mounted camera, the lane, and the vehicle based on the camera parameters includes:
[0019] The relative positions of the vehicle camera, lane, and vehicle are adjusted according to external parameters, which include at least the yaw angle, rotation angle, and pitch angle of the vehicle camera, as well as the position coordinates of the vehicle camera relative to the vehicle coordinate system and the height of the vehicle coordinate system from the ground.
[0020] In one embodiment, the camera parameters include multiple sets of internal parameters, and the method further includes:
[0021] By comparing the pixel changes at the same visible point in the first simulation image corresponding to multiple sets of internal parameters, the differences in results between internal parameters are determined.
[0022] In one embodiment, after obtaining a first simulation image of the interference between the vehicle camera's field of view and the vehicle based on the relative position between the vehicle-mounted camera and the vehicle, and combining the distorted visible points and vehicle body contour points, the method further includes:
[0023] The distortion of the first simulated image is removed to obtain the second simulated image.
[0024] In one embodiment, the first simulation image of the interference between the vehicle-mounted camera's field of view and the vehicle, based on the relative position between the vehicle-mounted camera and the vehicle, and combined with the visible points and vehicle body contour points after distortion processing, further includes:
[0025] Load the background image for the vehicle's camera;
[0026] The visible points and vehicle body contour points after distortion processing are loaded onto the background image to form the first simulation image of the interference between the field of view of the vehicle camera and the vehicle.
[0027] Secondly, this application also provides a vehicle-mounted camera simulation device, the device comprising:
[0028] The acquisition module is used to acquire simulation data from the vehicle-mounted camera. The simulation data includes at least environmental data and camera parameters. The environmental data includes at least vehicle body contour data and visible point data to assist in the simulation.
[0029] The initialization module is used to set the vehicle body contour points according to the vehicle body contour data, set the auxiliary visible points in the field of view of the vehicle camera according to the visible point data, and adjust the relative position between the vehicle camera and the vehicle according to the camera parameters.
[0030] The simulation processing module is used to normalize the coordinates of visible points and vehicle body contour points, and to perform distortion processing on visible points and vehicle body contour points according to camera parameters.
[0031] The image generation module is used to obtain a first simulation image of the interference between the field of view of the vehicle camera and the vehicle based on the relative position between the vehicle camera and the vehicle, and by combining the visible points after distortion processing and the vehicle body contour points. The vehicle body contour points are connected to form the vehicle body contour envelope in the first simulation image.
[0032] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the vehicle camera simulation method described in any of the above embodiments.
[0033] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle-mounted camera simulation method described in any of the above embodiments.
[0034] The aforementioned vehicle-mounted camera simulation method, device, computer equipment, and storage medium, on the one hand, can intuitively present the interference between the camera's field of view and the vehicle through simulation data, providing strong theoretical support for image algorithms to confirm the camera's placement position, and improving the accuracy and diversity of camera placement. At the same time, using lane lines as a reference object, the accuracy of simulation results can be verified in subsequent actual tests. On the other hand, the pixel changes at the same visible point can intuitively test the differences between two sets of internal parameters of the camera, providing more accurate simulation support for camera placement. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is an overall flowchart of the vehicle-mounted camera simulation method in one embodiment;
[0037] Figure 2 This is a schematic diagram of the visible points, lane points, and vehicle body contour points in a vehicle-mounted camera simulation method in one embodiment;
[0038] Figure 3 This is a schematic diagram of the first simulation of a vehicle-mounted camera simulation method in one embodiment.
[0039] Figure 4 This is a third-view schematic diagram of a vehicle-mounted camera simulation method in one embodiment;
[0040] Figure 5 This is a comparison chart of the corresponding results of two sets of internal parameters of the vehicle camera simulation method in one embodiment;
[0041] Figure 6 This is a flowchart illustrating the working process of a vehicle-mounted camera simulation method in one embodiment;
[0042] Figure 7 This is a structural block diagram of a vehicle-mounted camera simulation device in one embodiment. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
[0045] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe different objects corresponding to the same name, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the other object.
[0046] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” or “having,” etc., specify the presence of the stated feature, whole, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof.
[0047] In one embodiment, such as Figure 1 As shown, a vehicle-mounted camera simulation method is provided, the method including:
[0048] S100: Acquire simulation data from the vehicle-mounted camera. The simulation data includes at least environmental data and camera parameters. The environmental data includes at least vehicle body contour data and visible point data to assist in the simulation.
[0049] Specifically, the simulation data for the vehicle-mounted camera includes environmental data for setting the simulation environment and camera parameters for setting the simulation camera. The environmental data includes at least the vehicle's body contour data and visible point data to aid in the simulation. Connecting the visible points in the visible point data forms a grid, which serves as the camera's visible viewport, also known as the field of view (FOV). The vehicle body contour data is generated based on the vehicle's 3D data, and connecting the body contour points in the body contour data forms the envelope of the body contour, serving as the object carrier for camera placement. Camera parameters include internal and external parameters. External parameters include yaw, roll, pitch, the camera's position coordinates relative to the vehicle's coordinate system, and the vehicle's height above the ground. Internal parameters include the camera's focal length, pixel coordinates of the focal point, resolution, and distortion coefficient. These camera parameters are used to adjust the configuration of the simulation camera. Furthermore, the camera parameters can also include a camera model, such as a pinhole camera model or a fisheye model, to improve the flexibility and diversity of the camera simulation.
[0050] S200: Sets the vehicle body contour points based on the vehicle body contour data, and sets auxiliary visible points in the field of view of the vehicle camera based on the visible point data;
[0051] Specifically, based on the visible point data and vehicle body contour data in the aforementioned environmental data, visible points and vehicle body contour points are added in the simulation space. For example, see [link to relevant documentation]. Figure 2 Draw a 5mm×5mm grid within a 2m x 2m area 10cm in front of the camera. The points on the grid are the visible points. The distance, range, and spacing of the visible points can be adjusted according to actual needs.
[0052] S300: Adjusts the relative position between the vehicle camera and the vehicle according to the camera parameters;
[0053] Specifically, the camera angle and its position relative to the vehicle coordinates are adjusted according to external parameters. Specifically, the camera position is translated and / or rotated according to the external parameters in the camera parameters to adjust the relative position between the camera and the vehicle. Alternatively, the entire vehicle position can be translated and / or rotated to adjust the relative position between the camera and the vehicle.
[0054] S400: Normalizes the coordinates of visible points and vehicle body contour points, and performs distortion processing on visible points and vehicle body contour points according to camera parameters;
[0055] Specifically, the coordinates of the visible points and the vehicle body contour points are projected onto the normalized plane to obtain the coordinates in the normalized coordinate system. For example, the coordinates (X, Y, Z) are normalized to obtain (X / Z, Y / Z, 1) by projecting onto the Z-axis.
[0056] Specifically, the coordinates of visible points and vehicle body contour points are distorted according to the internal parameters in the camera parameters. The distortion processing includes radial distortion and / or tangential distortion. After the corresponding distortion processing, the image distortion present in the camera shooting can be simulated.
[0057] In some embodiments, the model of radial distortion can be represented as:
[0058] X=x(1+k1r 2 +k2r 4 +k3r 6 )
[0059] Y = y(1 + k1r) 2 +k2r 4 +k3r 6 )
[0060] r 2 =x 2 +y 2
[0061] Where (x, y) are the undistorted coordinates, (X, Y) are the distorted coordinates, and k1, k2, and k3 are the radial distortion coefficients. For cameras with pinhole imaging models, only k1 and k2 are generally used. For cameras with fisheye models that have large distortion, k3 is added.
[0062] In some embodiments, the model of tangential distortion can be represented as:
[0063] X = x + 2p1xy + p2(r) 2 +2x 2 )
[0064] Y = y + p1(r) 2 +2y 2 )+2p2xy
[0065] r 2 =x 2 +y 2
[0066] Where (x, y) are the undistorted coordinates, (X, Y) are the distorted coordinates, and p1 and p2 are the tangential distortion coefficients.
[0067] The model combining the two distortions described above can be represented as follows:
[0068] X=x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )
[0069] Y = y(1 + k1r) 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy
[0070] r 2 =x 2 +y 2
[0071] Correspondingly, the distorted pixel coordinates can be represented as:
[0072] u = f x X+C x
[0073] v = f y Y+C y
[0074] Among them, f x fy This represents the focal length of the camera, C. x C y This represents the horizontal and vertical offset of the camera's image origin relative to the optical center imaging point.
[0075] S500: Based on the relative position between the vehicle camera and the vehicle, and combined with the visible points and vehicle body contour points after distortion processing, a first simulation image of the interference between the field of view of the vehicle camera and the vehicle is obtained, wherein the vehicle body contour points are connected to form the vehicle body contour envelope in the first simulation image.
[0076] Specifically, see Figure 3 Using the range of visible points as the camera's visible window, and combining the camera's viewing angle with the relative position between the onboard camera and the vehicle, the camera's field of view (FOV) can be obtained. Connecting the vehicle's contour points yields the envelope of the vehicle's contour. Correlation between the camera's FOV and the vehicle's contour produces a first simulation image of their interference. This first simulation image can include images of interference with the vehicle captured by the camera at simulated angles. Further, see... Figure 4 The first simulation image may also include an image from a third-party perspective reflecting the interference between the camera's field of view and the vehicle.
[0077] The aforementioned vehicle camera simulation method can intuitively present the interference between the camera's field of view and the vehicle through simulation data, providing strong theoretical support for image algorithms to confirm the camera's placement position, and improving the accuracy and diversity of camera placement. In particular, by adjusting the simulation data, various cameras can be simulated. At the same time, if the vehicle data changes, the interference between each camera and the vehicle body contour can be quickly simulated, greatly reducing the camera debugging work in the subsequent real vehicle testing stage and improving the camera debugging efficiency.
[0078] In one embodiment, the environmental data further includes lane data, and the method further includes: setting lane points of the lanes according to the lane data; adjusting the relative positions between the vehicle camera, the lane, and the vehicle according to camera parameters; normalizing the coordinates of the lane points and distorting the lane points according to the camera parameters; and obtaining a first simulation image of the interference between the field of view of the vehicle camera and the vehicle based on the relative positions between the vehicle camera, the lane, and the vehicle, and combined with the visible points, lane points, and vehicle body contour points after distortion processing, wherein the lane points are connected to form lane lines in the first simulation image.
[0079] Specifically, see Figure 2In this embodiment, when simulating the camera, lane lines are added to the first simulation image. The environmental data also includes lane data. Lane points are set based on the lane data, and lane points are connected to form lane lines. Specifically, based on the camera's height from the ground and the distance between the camera and the lane lines, a lane point is added at regular intervals along the lane lines below the camera. These lane points are also subjected to the normalization and distortion processing described above, and the process will not be repeated here. On this basis, the first simulation image is generated by combining visible points, lane points, and vehicle body contours. The lane points are connected to form the lane lines in the first simulation image, thus completing the addition of lane lines to the first simulation image.
[0080] In one embodiment, the method further includes: determining the accuracy of the vehicle camera simulation by comparing the differences in lane lines in the first simulation image and the actual test results.
[0081] Specifically, in this embodiment, lane lines are added to the first simulation image. These lane lines can serve as a reference between the simulation results and the actual results. The reference based on the lane lines allows for a direct comparison of the deviation between the simulation results and the actual results, facilitating further adjustments to the camera in subsequent actual tests. This further reduces camera debugging work in the subsequent real vehicle testing phase and improves camera debugging efficiency.
[0082] In one embodiment, the camera parameters include external parameters and internal parameters. Adjusting the relative position between the vehicle camera, the lane, and the vehicle based on the camera parameters includes: adjusting the relative position between the vehicle camera, the lane, and the vehicle based on the external parameters. The external parameters include at least the yaw angle, rotation angle, and pitch angle of the vehicle camera, as well as the position coordinates of the vehicle camera relative to the vehicle coordinates and the height of the vehicle coordinates above the ground.
[0083] Specifically, the camera's angle is adjusted by using the camera's yaw, rotation, and pitch angles from the external parameters. The camera's position relative to the vehicle's coordinate system, the vehicle's height above the ground, and the vehicle's position relative to the ground are also adjusted using these external parameters. Controlling these parameters allows for easy adjustment of the camera's placement, significantly improving the simulation system's flexibility. The cameras can be flexibly adjusted to meet different needs, greatly enhancing the scalability of the camera simulation.
[0084] In one embodiment, the camera parameters include multiple sets of intrinsic parameters, and the method further includes: comparing the pixel changes of the first simulation image corresponding to the multiple sets of intrinsic parameters at the same visible point to determine the result differences between the intrinsic parameters.
[0085] Specifically, see Figure 5To better illustrate the differences between internal camera data, this embodiment can determine the differences in results between internal parameters by comparing first simulation images generated corresponding to multiple sets of internal camera parameters and combining the pixel changes at the same visible point. Furthermore, this embodiment can combine first simulation images corresponding to different internal parameters, simultaneously displaying the pixel position of the same visible point in the same simulation image. Based on the differences in pixel positions within the same simulation image, the pixel changes are determined, thus intuitively revealing the differences in results between internal parameters. For example, see [link to example]. Figure 5 By zooming in on the simulation image to the pixel level, the pixel position changes of the same visible point were compared between the two sets of internal parameters. In the simulation results of one set of internal parameters, the pixel position of the visible point was located in the area marked by the thick line at the top of the image, while in the simulation results of the other set of internal parameters, the pixel position of the same visible point was located in the area marked by the thick line at the bottom of the image. From the pixel position changes, it can be clearly seen that the simulation results of the two sets of internal parameters differ by 1 pixel horizontally and 6 pixels vertically. In this way, the difference between the two sets of internal parameters is clearly and intuitively demonstrated.
[0086] In one embodiment, after obtaining a first simulation image of the interference between the vehicle camera's field of view and the vehicle based on the relative position between the vehicle camera and the vehicle, and combining the visible points and vehicle body contour points after distortion processing, the method further includes: distorting the first simulation image to obtain a second simulation image.
[0087] Specifically, the first simulation image is the image after distortion processing. To further consider the requirement for a distortion-free image, this embodiment further performs distortion correction on the result of the first simulation image to obtain a distortion-calibrated image. In one implementation, the coordinates of each point before distortion processing can be used to generate the second simulation image; in another implementation, distortion correction can be performed based on the first simulation image to obtain the second simulation image. Distortion correction can derive the mapping relationship from the distorted image to the normal image using the radial distortion model and the tangential distortion model. Then, based on the mapping relationship, the coordinate position in the distorted image is calculated using the coordinate position of the normal image, and the corresponding pixel value is extracted to obtain the pixel value of the normal image. Since the pixel coordinates in the distorted image are often not integers (i.e., pixels in the distorted image do not have corresponding pixels in the normal image), interpolation can be used to process these pixels to obtain the second simulation image.
[0088] In one embodiment, obtaining the first simulation image of the interference between the vehicle camera's field of view and the vehicle, based on the relative position between the vehicle camera and the vehicle, and combined with the distorted visible points and vehicle body contour points, further includes: loading a background image of the vehicle camera; loading the distorted visible points and vehicle body contour points onto the background image to form the first simulation image of the interference between the vehicle camera's field of view and the vehicle.
[0089] Specifically, in order to improve the realistic display effect of the first simulation image, this embodiment displays the first simulation image by loading a background image. The background image is an image captured in advance by a real camera. These images include images of the actual test scene of the camera. By using the background image as the background of the first simulation image and adding all the points after distortion processing to the background image, a more realistic first simulation image is formed. This can simulate some actual test scenes and improve the accuracy of camera placement.
[0090] See Figure 6 The present embodiment will now be described in detail with reference to a practical application scenario, but it is not limited thereto.
[0091] The simulation data is obtained from user input or settings. This simulation data is the data corresponding to the camera of the pinhole imaging model. The simulation data includes environmental data corresponding to visible points, lane points, and vehicle body contour points, as well as camera parameters including internal and external parameters. Internal parameters include the camera's focal length, pixel coordinates of the focal point, resolution, and distortion coefficient. External parameters include yaw, roll, pitch, the camera's position coordinates relative to the vehicle coordinates, and the vehicle's height from the ground.
[0092] Based on the environmental data in the simulation data, add visible points, lane points, and vehicle outline points. Specifically, draw a 5mm x 5mm grid within a 2m x 2m area 10cm in front of the camera to form the visible point grid. Based on the input camera installation height and distance from the lane line, take lane points at regular intervals within a 3.75m width and 0-50m length range below the camera to form the lane line. Extract vehicle outline points from the vehicle's 3D data and connect them to form the vehicle outline.
[0093] Based on the external parameters of the camera, the position of the camera, lane lines, and vehicle body outline is translated and rotated to adjust the relative positions of the three. Specifically, the position of the camera is adjusted by yaw angle, rotation angle, and pitch angle, and the height of the vehicle body outline relative to the ground and the position of the camera relative to the vehicle body outline are adjusted by the position coordinates of the camera relative to the vehicle coordinates and the height of the vehicle coordinates above the ground.
[0094] The adjusted visible points, lane points, and vehicle body contour points are normalized and distorted. Projecting onto the Z-axis, the coordinates of all points (X0, Y0, Z0) are normalized to (X0 / Z0, Y0 / Z0, 1). Distortion is then performed in the normalized coordinate system using radial and tangential distortion models.
[0095] X=x(1+k1r 2 +k2r 4)+2p1xy+p2(r 2 +2x 2 )
[0096] Y = y(1 + k1r) 2 +k2r 4 )+p1(r 2 +2y 2 )+2p2xy
[0097] r 2 =x 2 +y 2
[0098] Where (x, y) are the undistorted coordinates, x = X0 / Z0, y = Y0 / Z0, (X, Y) are the distorted coordinates, p1 and p2 are the tangential distortion coefficients, and k1 and k2 are the radial distortion coefficients.
[0099] Correspondingly, the distorted pixel coordinates can be represented as:
[0100] u = f x X+C x
[0101] v = f y Y+C y
[0102] Among them, f x f y This represents the focal length of the camera, C. x C y This represents the horizontal and vertical offset of the camera's image origin relative to the optical center imaging point.
[0103] Based on the pixel coordinates of all points after distortion, the visible points are connected to form the visible window of the camera in the first simulation image, the lane points are connected to form the lane lines in the first simulation image, and the vehicle body contour points are connected to form the vehicle body contour in the first simulation image, thereby obtaining the first simulation image of the interference between the camera's field of view and the vehicle.
[0104] Through the above process, multiple sets of internal parameters are input to generate the corresponding first simulation image. The changes in pixel coordinates corresponding to the same visible point are compared to determine the differences in results between the multiple sets of internal parameters.
[0105] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0106] Based on the same inventive concept, this application also provides a camera simulation device for implementing the above-described vehicle camera simulation method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more camera simulation device embodiments provided below can be found in the limitations of the vehicle camera simulation method described above, and will not be repeated here.
[0107] In one embodiment, such as Figure 7 As shown, a camera simulation device is provided, the device comprising:
[0108] The acquisition module 10 is used to acquire simulation data from the vehicle-mounted camera. The simulation data includes at least environmental data and camera parameters. The environmental data includes at least vehicle body contour data and visible point data to assist in the simulation.
[0109] The initialization module 20 is used to set the vehicle body contour points according to the vehicle body contour data, set the auxiliary visible points in the field of view of the vehicle camera according to the visible point data, and adjust the relative position between the vehicle camera and the vehicle according to the camera parameters.
[0110] The simulation processing module 30 is used to normalize the coordinates of visible points and vehicle body contour points, and to perform distortion processing on visible points and vehicle body contour points according to camera parameters.
[0111] The image generation module 40 is used to obtain a first simulation image of the interference between the field of view of the vehicle camera and the vehicle based on the relative position between the vehicle camera and the vehicle, and by combining the visible points after distortion processing and the vehicle body contour points. The vehicle body contour points are connected to form the vehicle body contour envelope in the first simulation image.
[0112] In one embodiment, the environmental data also includes lane data. The initialization module sets the lane points of the lanes based on the lane data; adjusts the relative positions between the vehicle camera, the lane, and the vehicle based on the camera parameters; normalizes the coordinates of the lane points and performs distortion processing on the lane points based on the camera parameters; and obtains a first simulation image of the interference between the field of view of the vehicle camera and the vehicle based on the relative positions between the vehicle camera, the lane, and the vehicle, combined with the visible points, lane points, and vehicle body contour points after distortion processing. The lane points are connected to form the lane lines in the first simulation image.
[0113] In one embodiment, lane lines in the first simulation image are used to verify the difference between the first simulation image and the actual test results, wherein the accuracy of the vehicle camera simulation is determined by comparing the differences in lane lines in the first simulation image and the actual test results.
[0114] In one embodiment, the camera parameters include external parameters and internal parameters. The initialization module adjusts the relative positions between the vehicle camera, the lane, and the vehicle according to the camera parameters, including: adjusting the relative positions between the vehicle camera, the lane, and the vehicle according to the external parameters. The external parameters include at least the yaw angle, rotation angle, and pitch angle of the vehicle camera, as well as the position coordinates of the vehicle camera relative to the vehicle coordinates and the height of the vehicle coordinates above the ground.
[0115] In one embodiment, the camera parameters include multiple sets of internal parameters, and the device further includes a simulation comparison module for comparing the pixel changes of the first simulation image corresponding to the multiple sets of internal parameters at the same visible point to determine the result differences between the internal parameters.
[0116] In one embodiment, after the image generation module obtains a first simulation image of the interference between the vehicle camera's field of view and the vehicle based on the relative position between the vehicle camera and the vehicle, and combining the visible points and vehicle body contour points after distortion processing, the module further includes: distorting the first simulation image to obtain a second simulation image.
[0117] In one embodiment, the image generation module obtains a first simulation image of the interference between the vehicle camera's field of view and the vehicle based on the relative position between the vehicle camera and the vehicle, and by combining the distorted visible points and vehicle body contour points. The method further includes: loading a background image of the vehicle camera; and loading the distorted visible points and vehicle body contour points onto the background image to form the first simulation image of the interference between the vehicle camera's field of view and the vehicle.
[0118] Each module in the aforementioned vehicle-mounted camera simulation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0119] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the vehicle-mounted camera simulation methods described in the above embodiments. For detailed explanations, please refer to the corresponding descriptions of the methods; they will not be repeated here.
[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements any of the vehicle-mounted camera simulation methods described in the above embodiments. For detailed explanations, please refer to the corresponding descriptions of the methods, which will not be repeated here.
[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0123] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for simulating a vehicle-mounted camera, characterized in that, The method includes: Acquire simulation data from the vehicle-mounted camera. The simulation data includes at least environmental data and camera parameters. The environmental data includes at least vehicle body contour data and visible point data to assist in the simulation. The vehicle body contour points are set according to the vehicle body contour data, and auxiliary visible points in the field of view of the vehicle camera are set according to the visible point data. Adjust the relative position between the vehicle-mounted camera and the vehicle according to the camera parameters; The coordinates of the visible points and the vehicle body contour points are normalized, and distortion processing is performed on the visible points and the vehicle body contour points according to the camera parameters; Based on the relative position between the vehicle camera and the vehicle, and combined with the visible points after distortion processing and the vehicle body contour points, a first simulation diagram of the interference between the field of view of the vehicle camera and the vehicle is obtained, wherein the vehicle body contour points are connected to form the vehicle body contour envelope in the first simulation diagram. Based on the relative position between the vehicle-mounted camera and the vehicle, and combining the visible points after distortion processing with the vehicle body contour points, a first simulation image of the interference between the vehicle-mounted camera's field of view and the vehicle is obtained, including: The visible point range after distortion processing is used as the visible window of the vehicle camera. Combined with the viewing angle of the vehicle camera under the relative position between the vehicle camera and the vehicle, the field of view of the vehicle camera is obtained. The vehicle body contour points after distortion processing are connected to obtain the vehicle body contour envelope. The field of view of the vehicle camera and the vehicle body contour are then coherently correlated to obtain the first simulation image of their interference. The first simulation image may include the image of interference with the vehicle captured by the camera angle simulation.
2. The vehicle-mounted camera simulation method according to claim 1, characterized in that, The environmental data also includes lane data, and the method further includes: The lane point is set based on the lane data; Adjust the relative positions of the vehicle-mounted camera, lane, and vehicle according to the camera parameters; The coordinates of the lane points are normalized, and the lane points are distorted according to the camera parameters; Based on the relative positions of the vehicle-mounted camera, lane, and vehicle, and combined with the visible points, lane points, and vehicle body contour points after distortion processing, a first simulation image of the interference between the field of view of the vehicle-mounted camera and the vehicle is obtained, wherein the lane points are connected to form lane lines in the first simulation image.
3. The vehicle-mounted camera simulation method according to claim 2, characterized in that, The method further includes: determining the accuracy of the vehicle camera simulation by comparing the differences in lane lines between the first simulation image and the actual test results.
4. The vehicle-mounted camera simulation method according to claim 2, characterized in that, The camera parameters include external parameters and internal parameters, and adjusting the relative positions between the vehicle-mounted camera, the lane, and the vehicle based on the camera parameters includes: The relative positions of the vehicle-mounted camera, lane, and vehicle are adjusted according to the external parameters, which include at least the yaw angle, rotation angle, and pitch angle of the vehicle-mounted camera, as well as the position coordinates of the vehicle-mounted camera relative to the vehicle coordinate system and the height of the vehicle coordinate system from the ground.
5. The vehicle-mounted camera simulation method according to any one of claims 1 to 4, characterized in that, The camera parameters include multiple sets of internal parameters, and the method further includes: By comparing the pixel changes at the same visible point in the first simulation image corresponding to multiple sets of internal parameters, the differences in results between the internal parameters are determined.
6. The vehicle-mounted camera simulation method according to any one of claims 1 to 4, characterized in that, After obtaining the first simulation image of the interference between the vehicle camera's field of view and the vehicle based on the relative position between the vehicle-mounted camera and the vehicle, and combining the visible points after distortion processing with the vehicle body contour points, the method further includes: The first simulated image is distorted to obtain the second simulated image.
7. The vehicle-mounted camera simulation method according to any one of claims 1 to 4, characterized in that, The first simulation diagram of the interference between the vehicle-mounted camera's field of view and the vehicle, based on the relative position between the vehicle-mounted camera and the vehicle, and combined with the visible points after distortion processing and the vehicle body contour points, further includes: Load the background image for the vehicle's camera; The visible points and vehicle body contour points after distortion processing are loaded onto the background image to form the first simulation image of the interference between the field of view of the vehicle camera and the vehicle.
8. A vehicle-mounted camera simulation device, characterized in that, The device includes: The acquisition module is used to acquire simulation data from the vehicle-mounted camera. The simulation data includes at least environmental data and camera parameters. The environmental data includes at least vehicle body contour data and visible point data to assist in the simulation. An initialization module is used to set the vehicle body contour points according to the vehicle body contour data, set auxiliary visible points in the field of view of the vehicle camera according to the visible point data, and adjust the relative position between the vehicle camera and the vehicle according to the camera parameters. The simulation processing module is used to normalize the coordinates of the visible points and the vehicle body contour points, and to perform distortion processing on the visible points and the vehicle body contour points according to the camera parameters; The image generation module is used to take the range of visible points after distortion processing as the visible window of the vehicle camera, and combine the field of view of the vehicle camera with the relative position between the vehicle camera and the vehicle to obtain the field of view of the vehicle camera. The vehicle body contour points after distortion processing are connected to obtain the vehicle body contour envelope. The field of view of the vehicle camera is then coherently correlated with the vehicle body contour to obtain a first simulation image of the interference between the two. The first simulation image may include the image of interference between the camera and the vehicle captured by the camera angle simulation.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.