A calibration device for an AVM system
By introducing an image processor into the AVM system, the coupling problem between the calibration process and the camera image capture process is solved, enabling independent calibration and multi-environment simulation, improving calibration efficiency and accuracy, and supporting calibration of multiple vehicles and stitching algorithms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- SUZHOU ZHIHUA AUTOMOBILE TECHNOLOGY CO LTD
- Filing Date
- 2025-09-09
- Publication Date
- 2026-07-03
AI Technical Summary
In existing technologies, the calibration process of AVM systems is difficult to separate from the image capture process of cameras, resulting in low calibration efficiency and difficulty in simulating various environments.
An image processor is introduced into the AVM system, which connects to the storage device and AVM controller via an interface to convert and transmit images captured by the camera. This makes the calibration process independent of the camera capture process and supports the generation of images in various simulated environments and stress testing.
It achieves the independence of AVM system calibration, improves calibration efficiency and accuracy, supports multi-vehicle calibration and calibration with different stitching algorithms, and reduces the dependence on real vehicle photography.
Smart Images
Figure CN224457397U_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of AVM system calibration technology, and in particular to a calibration device for AVM systems. Background Technology
[0002] As vehicles become larger, the problem of blind spots around the vehicle is exacerbated, making it difficult for drivers to observe the surroundings and increasing driving risks. AVM (Around View Monitor) systems have emerged to address this issue.
[0003] The AVM system includes wide-angle cameras installed around the vehicle body and an AVM controller, which...
[0004] Wide-angle cameras around the vehicle capture raw images, which are then transmitted to the AVM controller. The AVM controller, based on pre-defined camera parameters, uses a built-in image stitching algorithm to correct and stitch the raw images together to create a panoramic image of the vehicle's surroundings. The driver can view this seamless 360° panoramic view in real-time on an in-vehicle screen. Its wide-angle and seamless stitching technology effectively solves the blind spot problem, significantly improving driver safety.
[0005] The accuracy of image stitching in an AVM (Autonomous View Monitor) system directly impacts driving safety. In real-world scenarios, camera parameters can be affected by various factors, leading to errors and consequently reducing the accuracy of AVM image stitching. For example, there may be errors in the camera's manufacturing process, assembly processes on the vehicle, and the vibrations caused by vehicle movement, all of which can cause parameter variations. These factors can result in discrepancies between the actual and intended camera parameters.
[0006] Therefore, the camera parameters of the AVM system need to be calibrated before or after the vehicle leaves the factory. Current calibration methods often employ the following approaches:
[0007] At a specially designed calibration site, multiple cameras on the actual vehicle capture multiple images of the target object and transmit them in real time to the AVM controller. The AVM controller then stitches the images together using predetermined camera parameters to generate a stitched image and calibrates the parameters of the surround-view cameras. In this method, the AVM controller calibration process and the camera image capture process can only be performed simultaneously.
[0008] However, in many scenarios, the camera image capture and AVM controller calibration processes need to be performed separately. For example, it might be necessary to schedule a vehicle to capture images at a calibration site during a certain time period, and then use these images for AVM calibration at a later time. Clearly, current calibration methods cannot separate AVM controller calibration from camera image capture. Therefore, there is an urgent need for a calibration method that can independently perform the calibration process from the camera image capture process. Utility Model Content
[0009] This application aims to at least address one of the technical problems existing in the prior art. To this end, this application provides a calibration apparatus for an AVM system to solve the problem that current AVM system calibration methods are difficult to implement using images captured in non-real-time.
[0010] This application provides a calibration device for an AVM system, the device comprising:
[0011] An image processor equipped with a first interface and a second interface is configured to acquire raw images captured by the surround-view camera of the AVM system from a storage device via the first interface, convert the raw images into a first image of a target format, and then output the first image via the second interface; the target format is a real-time video stream format.
[0012] An AVM controller is provided with a third interface, which is configured to connect to the second interface via the third interface to acquire the first image from the image processor and to perform calibration based on the first image.
[0013] Optionally, the device further includes:
[0014] The storage device is provided with a fourth interface, and the storage device is configured to connect to the first interface via the fourth interface to transfer the original image to the image processor.
[0015] Optionally, the device further includes:
[0016] A surround-view camera installed on a vehicle, the surround-view camera being configured to capture the raw images and connect to the storage device to transmit the captured raw images to the storage device for storage.
[0017] Optionally, the device may include multiple surround-view cameras on the same vehicle or multiple surround-view cameras on different vehicles.
[0018] Optionally, the storage device is an SD card and / or a USB flash drive.
[0019] Optionally, the first interface and the second interface are MIPI type interfaces; or, the first interface and the second interface are LVDS type interfaces.
[0020] Optional,
[0021] The image processor is also configured to generate a second image in a target format and output the second image through the second interface;
[0022] The AVM controller is configured to connect to the second interface via the third interface to acquire the second image from the image processor and to perform calibration based on the second image.
[0023] Optionally, the image processor is further configured to transmit more than a preset number of the first images to the AVM controller via the second interface within a preset time to stress test the AVM controller.
[0024] Optionally, the storage device can be configured to store images captured by surround-view cameras corresponding to multiple vehicles.
[0025] Optionally, the image processor is also configured to select a first image of the corresponding vehicle based on user instructions and transmit it to the AVM controller for calibration.
[0026] Optionally, the device further includes:
[0027] A display is configured to connect to the AVM controller to receive and display the calibration results from the AVM controller.
[0028] Optionally, the device further includes:
[0029] An algorithm configuration controller is configured to connect to the AVM controller for modifying the splicing algorithm in the AVM controller;
[0030] The AVM controller is configured to stitch the first image using the modified stitching algorithm configured by the controller according to the algorithm, so as to calibrate the accuracy of the modified stitching algorithm.
[0031] The above-described technical solutions of this application have at least one or more of the following effects:
[0032] In this application's technical solution, an image processor is added to the calibration device of the AVM system. This image processor has a first interface adapted to a storage device to retrieve raw images captured by the surround-view camera, and a second interface adapted to the AVM controller to transmit images in a target format. It can also convert the raw images into images in a target format recognizable by the AVM controller. Based on this, by using the added image processor as a transmission medium, the AVM controller can retrieve images stored in the storage device from the surround-view camera without any modifications, instead of necessarily acquiring real-time images directly from the surround-view camera. This allows the AVM calibration process to be independent of the camera's image capture process, solving the problem in existing technologies where AVM controller calibration can only be performed concurrently with the surround-view camera's image capture process.
[0033] Furthermore, in this application's technical solution, the image processor can also generate a second image in the target format and transmit it to the AVM controller for calibration via a second interface. Compared to existing technologies, this reduces reliance on real-vehicle image capture, increases image generation speed, reduces image generation difficulty, and improves calibration efficiency. Additionally, compared to the real-vehicle image capture process in existing technologies, software simulation can be used to simulate images under more complex environments (such as extreme lighting conditions), thereby enabling and improving calibration in more diverse environments.
[0034] Furthermore, in this application's technical solution, the image processor is also configured to transmit more than a preset number of first images to the AVM controller via a second interface within a preset time to perform stress testing on the AVM controller. Compared to existing technologies, this solution is not affected by the speed or number of images captured by the camera in real time, and can easily achieve the transmission of a set number of images at a set speed, and can be used to implement stress testing.
[0035] Furthermore, in this application's technical solution, the image processor is also configured to select a corresponding first image and transmit it to the AVM controller according to user instructions. Compared to existing technologies, this solution is not affected by the timing of images captured in real-time by the camera, and it can conveniently select the required image from multiple images for calibration.
[0036] Furthermore, in this application's technical solution, the storage device can be configured to store images captured by surround-view cameras corresponding to multiple vehicles. Correspondingly, the image processor is also configured to select the first image of the corresponding vehicle according to user instructions and transmit it to the AVM controller for calibration. Compared to existing technologies, this solution can achieve calibration of multiple vehicles in a single calibration process.
[0037] Additional aspects and advantages of this invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
[0038] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description
[0039] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:
[0040] Figure 1 This is a diagram illustrating the architecture of a calibration device according to an embodiment of this application;
[0041] Figure 2 This is a calibration device architecture diagram of another embodiment of this application. Detailed Implementation
[0042] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.
[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The terms “and / or” or “and / or” as used herein include any and all combinations of one or more of the associated listed items.
[0044] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that, unless otherwise specified, preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.
[0045] It should be understood that the term "vehicle" as used in this application includes general motor vehicles, such as passenger vehicles including sport utility vehicles (SUVs), buses, trucks, and various commercial vehicles, and includes, but is not limited to, hybrid vehicles, electric vehicles, and plug-in hybrid electric vehicles.
[0046] like Figure 1 The diagram shows the calibration device architecture of the AVM system, i.e., the panoramic monitoring system of this application. Specifically, the calibration device 10 includes:
[0047] The system includes a surround-view camera 11, a storage device 12, an image processor 13, an AVM controller 14, and a display 15.
[0048] AVM systems typically include surround-view cameras, an AVM controller, and a display. However, in the embodiments of the calibration device described in this application, Figure 1 The surround-view camera 11, AVM controller 14, and display 15 of the calibration device 10 shown can be configured for the same vehicle, together forming the vehicle's AVM system. In an alternative embodiment, Figure 1 The surround-view camera 11 of the calibration device 10 shown, and at least one of the AVM controller and display 15 can correspond to different vehicle settings, that is, belong to different AVM systems.
[0049] The surround-view camera 11 is a camera installed on the vehicle to capture images of the vehicle's surroundings. Typically, four surround-view cameras 11 can be installed on a single vehicle. As an example and not a limitation, the surround-view cameras 11 can be installed at the front left, rear left, front right, and rear right of the vehicle. The surround-view cameras 11 are wide-angle cameras, and the images acquired by multiple surround-view cameras are stitched together to obtain a panoramic image of the vehicle's surroundings. In some embodiments of this application, the multiple surround-view cameras 11 of the calibration device 10 can refer to multiple surround-view cameras on the same vehicle or multiple surround-view cameras on different vehicles. This application does not impose specific limitations in this regard.
[0050] The AVM controller 14 is a component installed in a vehicle for image stitching to generate a panoramic stitched image and for calibrating camera parameters. The AVM controller 14 is configured to receive and recognize real-time video data transmitted directly from the surround-view cameras, but cannot recognize static image data, such as image data stored on storage devices like SD memory cards. Therefore, it is compatible with... Figure 1Unlike typical AVM systems, where the AVM controller 14 is directly connected to the surround-view camera 11 and uses an interface of type MIPI (Mobile Industry Processor Interface) or LVDS (Low Voltage Differential Signaling) to acquire real-time video data captured by the camera, the calibration process and the camera image capture process can only be performed simultaneously. However, in this application, by adding a storage device 12 and an image processor 13, the images captured by the camera can be stored in the storage device 12 first. When the AVM controller 14 wants to perform calibration, it can retrieve the target format image from the storage device 12 at any time through the image processor 13. In other words, the calibration process and the camera image capture process can be performed separately.
[0051] Further reference Figure 1 As shown, the AVM controller 14 has a third interface 141 for communicating with the image processor 13.
[0052] The display 15 can be installed on the vehicle and communicate with the AVM controller 14 to acquire and display the stitched panoramic image or the calibrated result from the AVM controller 14.
[0053] Storage device 12 is used to communicate with surround view camera 11 to acquire real-time video data (also known as raw images) captured by surround view camera 11 and store it as static data. This application does not limit the type of storage device 12. As an example and not a limitation, in some embodiments, storage device 12 may be an SD memory card or a USB flash drive.
[0054] Further reference Figure 1 As shown, the storage device 12 is provided with a fourth interface 121 for communicating with the image processor 13.
[0055] These storage devices 12 cannot output real-time video streams like the surround-view camera 11. As mentioned earlier, the AVM controller 14 can only recognize real-time video streams and cannot recognize the data format of the storage devices. Therefore, the AVM controller 14 cannot directly obtain the raw images from the storage devices 12 for calibration.
[0056] Further reference Figure 1As shown, the image processor 13 is configured to communicate with both the storage device 12 and the AVM controller 14. The image processor 13 has a first interface 131 and a second interface 132. The image processor 13 is configured to acquire the raw image captured by the surround-view camera 11 from the fourth interface 121 of the storage device 12 via the first interface 131, convert the raw image into a first image of a target format, and then output the first image to the third interface 141 of the AVM controller 14 via the second interface 132. The target format in this application is a real-time video stream format recognizable by the AVM controller 14, i.e., the same as the data format output by the surround-view camera 11. Based on this, the AVM controller 14 can perform calibration based on the first image.
[0057] In this embodiment, the second interface 132 and the third interface 141 are interfaces capable of transmitting data in a format recognizable by the AVM controller 14. By way of example and not limitation, in some embodiments, the second interface 132 and the third interface 141 may be MIPI type interfaces, or they may be LVDS type interfaces. Since the data transmitted to the AVM controller 14 is in a format it can recognize, the AVM controller 14 does not need to modify its internal software to adapt to unrecognizable data formats.
[0058] In this embodiment, the third interface 141 is an interface that the AVM controller 14 already has based on its original functions, such as an interface for interfacing with the surround-view camera 11. Therefore, the AVM controller 14 does not need to add any additional interfaces.
[0059] Based on the above method, the AVM controller can indirectly obtain images captured by the surround-view camera stored in the storage device without making any changes (no additional interface or modification of AVM software), instead of having to directly obtain images from the surround-view camera. This allows the AVM calibration process to be independent of the surround-view camera image capture process, solving the problem in the prior art that the AVM controller calibration can only be performed together with the surround-view camera image capture process.
[0060] Since the calibration process described above is performed independently of the shooting process, the same AVM controller 14 can be used to calibrate the intrinsic and extrinsic parameters of the surround-view cameras 11 corresponding to multiple vehicles. Specifically, in some embodiments of this application, the storage device 12 can be configured to store the original images captured by the surround-view cameras 11 corresponding to multiple vehicles, and the image processor 13 is further configured to select the corresponding vehicle as the target vehicle according to user instructions, obtain the original image of the target vehicle from the storage device 12, convert it into a first image, and then send it to the AVM controller 14 for calibration of the surround-view cameras 11 on the target vehicle.
[0061] The accuracy of panoramic stitching in an AVM system is affected by a variety of factors, such as the accuracy of the internal and external parameters of the surround-view cameras, and the calibration of the stitching algorithm itself in the AVM controller (i.e., assuming that the camera parameters are accurate). Therefore, the calibration of the AVM system can be performed on the internal and external parameters of the surround-view cameras, or on the stitching algorithm itself in the AVM controller.
[0062] Calibration of the intrinsic and extrinsic parameters of a surround-view camera requires images captured by the camera as the basis for judgment. In other words, the images used for calibrating the camera's intrinsic and extrinsic parameters must be captured by the surround-view camera. However, in calibrating the stitching algorithm itself in the AVM controller, it can be assumed that the camera's intrinsic and extrinsic parameters are accurate; the focus is on whether the accuracy of the stitching algorithm model itself is suitable for various lighting conditions. In this case, the images used for calibration can be actually captured by the camera, such as the first image mentioned above, or simulated images. Considering that actual camera images require site calibration, involve human intervention, have low shooting efficiency, and are difficult to prepare for diverse and complex shooting environments, some embodiments of this application configure the generation of images through simulation. Specifically, the image processor 13 is also configured to generate a second image in the target format and output the second image to the third interface 141 of the AVM controller 14 through the second interface 132. Optionally, the second image is an image generated based on simulation. In this case, the AVM controller 14 is configured to perform calibration based on the second image. The calibration can be the above-mentioned calibration of the stitching algorithm in the AVM controller 14 itself, or it can be the calibration of other items that do not require actual images captured by the camera.
[0063] Furthermore, in some embodiments of this application, the calibration device 10 further includes an algorithm configuration controller (not shown in the figure), configured to be connected to the AVM controller 14, for modifying the stitching algorithm in the AVM controller 14; the AVM controller 14 is configured to stitch the first image according to the stitching algorithm modified by the algorithm configuration controller, so as to calibrate the accuracy of the modified stitching algorithm. Based on this, different stitching algorithms can be calibrated using the same calibration device 10.
[0064] If two stitching algorithm models, A and B, are preset in the algorithm configuration controller, stitching algorithm model A is first transmitted to the AVM controller 14. The AVM controller 14 stitches the first image according to stitching algorithm model A and calibrates the accuracy of stitching algorithm model A based on the stitched image. Then, the algorithm configuration controller can transmit stitching algorithm model B to the AVM controller 14 to replace stitching algorithm model A. The AVM controller 14 stitches the first image according to stitching algorithm model B and calibrates the accuracy of stitching algorithm model B based on the stitched image. Based on this, different stitching algorithms can be conveniently calibrated using the same calibration device 10.
[0065] In addition to the calibration described above, the calibration device in this application can also be used to perform stress tests on the AVM controller 14. Specifically, the image processor 13 is also configured to transmit more than a preset number of first images and / or second images to the AVM controller 14 via the second interface 132 within a preset time to perform stress tests on the AVM controller 14.
[0066] It should be noted that, Figure 1 The calibration device 10 shown is from a partial embodiment of this application. In alternative embodiments, such as... Figure 2 As shown, the calibration device 10 may only include an image processor 13, an AVM controller 14, and a display 15, as long as the image processor 13 is configured to obtain an image from any storage device and send it to the AVM controller 14 after conversion.
[0067] It should be noted that the focus of this application is on the various parts included in the device and their connection relationships. Specifically, by adding an image processor, the calibration process and the image capture process are made independent without changing the AVM controller, thereby improving the efficiency, accuracy, and convenience of calibration. That is, the focus of this application is not on the stitching algorithm or calibration algorithm inside the AVM controller 14, nor on how the graphics processor 13 specifically implements image format conversion. These can all be achieved using existing methods and software and are not the improvement points of this application.
[0068] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0069] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0070] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.
Claims
1. A calibration device for an AVM system, characterized in that The device includes: An image processor equipped with a first interface and a second interface is configured to acquire raw images captured by the surround-view camera of the AVM system from a storage device via the first interface, convert the raw images into a first image of a target format, and then output the first image via the second interface; the target format is a real-time video stream format. An AVM controller is provided with a third interface, which is configured to connect to the second interface via the third interface to acquire the first image from the image processor and to perform calibration based on the first image.
2. The calibration device of the AVM system according to claim 1, characterized in that, The device further includes: The storage device is provided with a fourth interface, and the storage device is configured to connect to the first interface via the fourth interface to transfer the original image to the image processor.
3. The calibration device of the AVM system according to claim 2, characterized in that, The device further includes: A surround-view camera installed on a vehicle, the surround-view camera being configured to capture the raw images and connect to the storage device to transmit the captured raw images to the storage device for storage.
4. The calibration device of the AVM system according to claim 1, characterized in that, The calibration device includes multiple surround-view cameras on the same vehicle or multiple surround-view cameras on different vehicles.
5. The calibration device for the AVM system according to claim 2, characterized in that, The storage device is an SD card and / or a USB flash drive.
6. The calibration device for the AVM system according to claim 1, characterized in that, The second and third interfaces are MIPI type interfaces; or, The second and third interfaces are LVDS type interfaces.
7. The calibration device for the AVM system according to claim 1, characterized in that, The image processor is also configured to generate a second image in a target format and output the second image through the second interface; The AVM controller is configured to connect to the second interface via the third interface to acquire the second image from the image processor and to perform calibration based on the second image.
8. The calibration apparatus for the AVM system according to any one of claims 1 to 7, characterized in that, The image processor is also configured to transmit more than a preset number of the first images to the AVM controller via the second interface within a preset time to stress test the AVM controller.
9. The calibration device of an AVM system according to any one of claims 1 to 7, characterized in that, The device further includes: A display is configured to connect to the AVM controller to receive and display the calibration results from the AVM controller.
10. The calibration device of an AVM system according to any one of claims 1 to 7, characterized in that The device further includes: An algorithm configuration controller is configured to connect to the AVM controller for modifying the splicing algorithm in the AVM controller; The AVM controller is configured to stitch the first image using the modified stitching algorithm configured by the controller according to the algorithm, so as to calibrate the accuracy of the modified stitching algorithm.