Accurate internal reference calibration for vehicle camera

Abstract

Aspects of the present disclosure include methods, apparatus, and vehicles for performing internal reference calibration of a camera in a vehicle. A processor may acquire a plurality of images of a target captured by a camera at different locations. For each image, the processor may tone map the image, binarize the tone-mapped image, define structural elements based on a digital form of the shape of the target, perform morphology on the image to align and calibrate the structural elements, filter the target and modify each of the other pixels to black, and compile the output image. Based on the calibration, the principal point value may be altered to account for the difference and store a calibration value in each camera.

Description

[0001] introduction

[0002] The present disclosure relates generally to interior and exterior vehicle-based and other cameras, and more specifically to intrinsic calibration of such cameras. Intrinsic calibration of cameras is critical for use in computer vision applications. These applications include, for example, calculating distances between objects and estimating the corresponding sizes of those objects based on images captured by the camera. Intrinsic calibration involves calibrating internal parameters of a camera, such as focal length, principal point, and distortion coefficients. Intrinsic calibration is important for computer vision applications because, as noted above, accurate focal length and principal point are required for accurate calculation of distance and size of objects. With respect to the simple example of a camera supplier providing a camera to an automobile manufacturer for use in a vehicle, the intrinsic calibration currently provided by the supplier lacks accuracy. These failures cause sensor alignment errors on a larger scale, meaning that the magnitude of errors associated with objects, distances, and sizes become more significant when the camera is not properly calibrated. Summary of the invention

[0003] Therefore, aspects of the present disclosure address these and other deficiencies in the art. Most notably, the addition of imaging morphology to calibrating target images is a unique addition to the process of intrinsic calibration. For example, incorporating a chronological morphological process into target image counts allows the camera to provide a more accurate and reliable intrinsic calibration while at the same time making it simpler to detect and monitor material changes in target images in greater detail. Higher-order resolutions can be achieved, and the overall distance between objects and their corresponding individual sizes can be relied upon more accurately than using existing methods.

[0004] In one aspect of the present disclosure, a device for intrinsic calibration of a camera includes a memory for storing images and executable codes (which can be defined as a plurality of different memory structures, such as cache memory, dynamic RAM (DRAM) and non-volatile memory) and an image processor coupled to the memory (which can include one or more general-purpose processors or one or more dedicated image processors, etc.). The image processor is configured to acquire a plurality of images of a target captured by a camera at different positions for storage in the memory. In addition, for each of the plurality of images, the image processor is configured to tone map the image to improve the black and white image level, binarize the tone mapped image, define a structural element based on a digital form of the target shape, perform morphology on the image to align and calibrate the structural element, filter the target and modify other pixels to black, compile an output image including the filtered target, determine an intrinsic value using the compiled output image, change a principal point when using the target for intrinsic calibration, and store the newly changed principal point in the memory.

[0005] In another aspect of the present disclosure, a method for intrinsic calibration of a camera includes: for a predetermined number of times, an image processor acquires multiple images of a target captured by a camera at different positions to store in a memory, and acquires a horizontal field of view (HFOV) and a vertical field of view (VFOV) of a camera lens based on the images. The method also includes, for each of the multiple images, tone mapping the image to improve the black and white image level, binarizing the tone mapped image, defining a structural element based on a digital form of the target shape, performing morphology on the image to align and calibrate the structural element, using the target at multiple positions to filter the target image, compiling an output image including the filtered target; using the compiled output image to determine an intrinsic parameter value, changing the principal point when using the target for intrinsic calibration, and storing the newly changed principal point in the memory.

[0006] In another aspect of the present disclosure, a vehicle includes a vehicle body having a camera mounted on its exterior, a cab housed inside the vehicle body, a dashboard near the front of the vehicle body, a memory for storing images and executable code in the vehicle body, and an image processor coupled to the memory and configured to acquire multiple images of a target captured by the camera at different positions for storage in the memory. For each of the multiple images, the image processor is further configured to tone map the image to improve the black and white image level, binarize the tone mapped image, define a structural element based on a digital form of the target shape, perform morphology on the image to align and calibrate the structural element, filter the target and modify other pixels to black, compile an output image including the filtered target, use the compiled output image to determine an intrinsic parameter value, change the principal point when using the target for intrinsic parameter calibration, and store the newly changed principal point in the memory.

[0007] In various embodiments for each of the above aspects, the apparatus, method, and vehicle are further configured to implement the following. The image processor may be configured to compute a kernel to define a structural element for use in performing morphology, which structural element is a digital form of the shape of an object used by the camera. The image processor may also be configured to identify a horizontal field of view (HFOV) and a vertical field of view (VFOV) based on an image from the camera, locate edges of the object for storage in a memory, perform an edge spread function (ESF) using the found edges, estimate the number of blurred pixels based on the ESF, and remove blurred pixels from the image. In yet other embodiments, the image processor may be configured to use calibration differences before and after a calibration process to include (1) a calibration method, (2) a morphological process, (3) output pixel size and height, (4) focal length, (5) principal point, and (6) distortion coefficients. In other embodiments, the calibration parameters of the calibration method may include pixel size, focal length, principal point, and distortion coefficients. The image processor may also be configured to add intrinsic calibration to incorporate the time-ordered morphological process into the target image counts, thereby achieving accurate intrinsic calibration and making it easy to detect and monitor changes in the target image with high detail.

[0008] In various embodiments, the image processor is configured to add intrinsic calibration to incorporate the morphological process in a time-ordered manner into the target image count, thereby achieving accurate intrinsic calibration and detecting and monitoring changes in the target image with high detail. In embodiments involving self-calibration of a vehicle, the image processor and memory are part of an electronic control system (ECU) within the vehicle body. When the calibration ECU is part of the vehicle body, the image processor can also be configured to incorporate the changed principal point and focus.

[0009] The above summary is not intended to represent every embodiment or every aspect of the present disclosure. Rather, the foregoing summary merely provides examples of some of the novel concepts and features set forth herein. When understood in conjunction with the accompanying drawings and the appended claims, the above-mentioned features and advantages of the present disclosure and other features and attendant advantages will be apparent from the following detailed description of the illustrated examples and representative modes for implementing the present disclosure. In addition, the present disclosure explicitly includes various combinations and sub-combinations of the elements and features presented above and below.

[0010] The present invention also includes the following solutions.

[0011] Solution 1. A device for calibrating the intrinsic parameters of a camera, comprising:

[0012] Memory for storing images and executable code; and

[0013] an image processor coupled to the memory and configured to:

[0014] acquiring a plurality of images of the target captured by the camera at different positions for storage in a memory;

[0015] For each of the multiple images:

[0016] Tone mapping of images to improve black and white image quality;

[0017] Binarize the tone-mapped image;

[0018] Defining structural elements based on the digital form of the target shape;

[0019] Perform morphology on images to align and demarcate structural elements;

[0020] Filter the target and modify all other pixels to black;

[0021] compiling an output image including the filtered target;

[0022] Use the compiled output image to determine the intrinsic parameter value;

[0023] Changing the principal point when using a target for internal calibration; and

[0024] The newly changed primary point is stored in memory.

[0025] Option 2. An apparatus according to Option 1, wherein the image processor is configured as a computing core to define a structural element for use in performing morphology, the structural element being a digital form of the shape of a target used by the camera.

[0026] Solution 3. The apparatus according to Solution 2, wherein the image processor is configured to:

[0027] Identify the horizontal field of view (HFOV) and vertical field of view (VFOV) based on the camera image;

[0028] locating target edges for storage in memory;

[0029] performing an edge spread function (ESF) using the target edge; and

[0030] The number of blurry pixels is estimated based on the ESF.

[0031] Option 4. An apparatus according to Option 1, wherein the image processor is configured to use calibration differences before and after the structuring process to include (1) calibration method, (2) morphological process, (3) output pixel size and height, (4) focal length, (5) principal point, and (6) distortion coefficients.

[0032] Option 5. An apparatus according to Option 4, wherein calibration parameters used for the calibration method include pixel size, focal length, principal point and distortion coefficient.

[0033] Option 6. An apparatus according to Option 1, wherein the image processor is configured to add internal parameter calibration to incorporate the morphological process arranged in time sequence into the target image count, thereby achieving accurate internal parameter calibration and detecting and monitoring changes in the target image in detail.

[0034] Solution 7. A method for calibrating an intrinsic parameter of a camera, comprising:

[0035] acquiring, by the image processor, a plurality of images of the target captured by the camera at different positions for a predetermined number of times for storage in a memory;

[0036] Obtain the horizontal field of view (HFOV) and vertical field of view (VFOV) of the camera lens based on the image;

[0037] For each of the multiple images:

[0038] Tone mapping of images to improve black and white image quality;

[0039] Binarize the tone-mapped image;

[0040] Defining structural elements based on the digital form of the shape of the target;

[0041] Perform morphology on images to align and demarcate structural elements;

[0042] Using targets at multiple locations to filter target images;

[0043] compiling an output image including the filtered target;

[0044] Use the compiled output image to determine the intrinsic parameter value;

[0045] Changing the principal point when using a target for internal calibration; and

[0046] The newly changed primary point is stored in memory.

[0047] Option 8. The method of Option 7 further includes a computing core to define a structural element for use in performing morphology, wherein the structural element is a digital form of the shape of a target used by the camera.

[0048] Scheme 9. The method according to Scheme 8 further comprises:

[0049] Identify the horizontal field of view (HFOV) and vertical field of view (VFOV) based on the camera image;

[0050] locating target edges for storage in memory;

[0051] performing an edge spread function (ESF) using the located target edge; and

[0052] The number of blurry pixels is estimated based on the ESF.

[0053] Solution 10. The method according to Solution 7 further comprises:

[0054] The calibration differences before and after using the calibration structure element are used to include (1) calibration method, (2) morphological process, (3) output pixel size and height, (4) focal length, (5) principal point, and (6) distortion coefficients.

[0055] Option 11. A method according to Option 10, wherein the calibration parameters of the calibration method include pixel size, focal length, principal point and distortion coefficient.

[0056] Option 12. The method according to Option 7 also includes adding internal reference calibration to incorporate the morphological process arranged in time sequence into the target image count, thereby achieving accurate internal reference calibration and easily detecting and monitoring changes in the target image in detail.

[0057] Solution 13. A vehicle, comprising:

[0058] a vehicle body having a camera mounted on an exterior thereof;

[0059] a cab housed within the vehicle body;

[0060] The instrument panel near the front of the vehicle interior;

[0061] Memory for storing images and executable code within the vehicle body;

[0062] an image processor coupled to the memory and configured to:

[0063] acquiring a plurality of images of the target captured by the camera at different positions for storage in a memory;

[0064] For each of the multiple images:

[0065] Tone mapping of images to improve black and white image quality;

[0066] Binarize the tone-mapped image;

[0067] Defining structural elements based on the digital form of the target shape;

[0068] Perform morphology on images to align and demarcate structural elements;

[0069] Filter the target and modify all other pixels to black;

[0070] compiling an output image including the filtered target;

[0071] Use the compiled output image to determine the intrinsic parameter value;

[0072] Changing the principal point when using a target for internal calibration; and

[0073] The newly changed primary point is stored in memory.

[0074] Option 14. A vehicle according to Option 13, wherein the image processor is configured as a computing core to define a structural element for use in performing morphology, the structural element being a digital form of the shape of a target used by the camera.

[0075] Embodiment 15. The vehicle according to embodiment 14, wherein the image processor is configured to:

[0076] Identify the horizontal field of view (HFOV) and vertical field of view (VFOV) based on the camera image;

[0077] locating target edges for storage in memory;

[0078] Using the found edges to perform an edge spread function (ESF); and

[0079] The number of blurry pixels is estimated based on the ESF.

[0080] Option 16. A vehicle according to Option 13, wherein the image processor is configured to use calibration differences before and after a calibration process to include (1) a calibration method, (2) a morphological process, (3) an output pixel size and height, (4) a focal length, (5) a principal point, and (6) a distortion coefficient.

[0081] Option 17. A vehicle according to Option 16, wherein the calibration parameters include pixel size, focal length, principal point and distortion coefficient.

[0082] Option 18. A vehicle according to Option 13, wherein the image processor is configured to add intrinsic calibration to incorporate morphological processes arranged in time sequence into target image counts, thereby achieving accurate intrinsic calibration and detecting and monitoring changes in target images with high detail.

[0083] Option 19. A vehicle according to Option 13, wherein the image processor and the memory are part of an in-vehicle electronic control system (ECU).

[0084] Option 20. A vehicle according to Option 19, wherein the image processor is further configured to incorporate a changed primary point and focal point when the ECU is part of the vehicle body. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0086] Figure 1 is a flow chart of the prior art for calibrating vehicle cameras.

[0087] Figure 2 is a flow chart of an improved technique for calibrating a vehicle camera according to one aspect of the present disclosure.

[0088] Figure 3 is a detailed block diagram of a technique for calibrating a vehicle camera according to one aspect of the present disclosure.

[0089] Figure 4 is an example image representation before morphology according to one aspect of the present disclosure.

[0090] Figure 5 is a representation of the image that is undergoing the representation to modify the objects into their correct alignment in the image.

[0091] Figure 6 is a representation of the image immediately before the target augmentation.

[0092] Figure 7 is a representation of the image after target enhancement has been applied, with aligned deformation alerts.

[0093] Figure 8 is a table describing factors associated with differences in internal reference calibration before and after the morphological process.

[0094] Fig. 9A plan view of a vehicle and the battery system coupled to an electronic control unit (ECU) to control various aspects of the battery.

[0095] Fig.10 is a plan view of the specified location where the intrinsic calibration of the vehicle camera is performed, including the hardware devices that can be used for calibration.

[0096] The drawings are not necessarily drawn to scale and may present simplified representations of various features of the present disclosure, including, for example, specific sizes, orientations, positions, and shapes. In some cases, well-known features may be omitted from certain drawings to avoid unduly obscuring the concepts of the present disclosure. Details associated with such features will be determined in part by the specific intended application and use case environment. DETAILED DESCRIPTION

[0097] The present disclosure allows for embodiments in many different forms. Representative examples of the present disclosure are shown in the accompanying drawings and described in detail herein as non-limiting examples of the disclosed principles. For this reason, elements and limitations described in the abstract, introduction, summary, and detailed description but not explicitly set forth in the claims should not be incorporated into the claims individually or collectively by implication, inference, or otherwise.

[0098] For the purposes of this specification, unless otherwise stated, the use of the singular includes the plural, and vice versa, the terms "and" and "or" shall be both conjunctions and disjunctions, and the words "including", "comprising", "having", and the like shall mean "including without limitation". In addition, approximate words such as "about", "almost", "substantially", "generally", "approximately", and the like may be used herein in the sense of "at, close to, or nearly at", or "within 0-5% of ...", or "within acceptable manufacturing tolerances", or logical combinations thereof. As used herein, a component that is "configured to" perform a specified function is capable of performing the specified function without change, not merely having the potential to perform the specified function after further modification. In other words, when explicitly configured to perform a specified function, the hardware described is specifically selected, created, implemented, utilized, programmed, and / or designed for the purpose of performing the specified function.

[0099] Currently, many manufacturers of vehicles, transport structures, and other heavy machinery and equipment that rely on precisely assembled cameras are entrusted to individually calibrate those vehicles or equipment at a dedicated location, given the current lack of accurate intrinsic calibration data from suppliers. Among other disadvantages, although necessary, this separate calibration is expensive, involves engineer workload, storage of large quantities of vehicles or other equipment, and the overhead associated with these logistics. Incorporating image morphology into this process is beneficial because it provides more accurate and reliable intrinsic calibration results. These superior results make it easier to detect and track changes in target images over time because, for example, the morphological process can provide detailed information about the structure of the target image.

[0100] As mentioned above, incorporating image morphology into the calibration target enables the user to modify the target image counts using a chronological morphological process to improve the intrinsic calibration. By incorporating a chronological morphological process into the target image counts, this novel approach can provide a more accurate and reliable intrinsic calibration, as well as making it easier to detect and monitor changes in the target image in more detail.

[0101] Figure 1 is a flow chart of the prior art for calibrating a vehicle camera. As in various figures in the present disclosure, the hardware element that performs the intrinsic calibration function may include a computing device disposed in a location where the calibration is to be performed. For example, Fig.10 An example of this configuration is shown. Fig. 9 In addition, Figure 1-10 The functions generally described in the can be performed by one or more vehicle external image processors, which can include general purpose processors, dedicated graphics or image processors, and other complex and reduced instruction set computers. Figure 2-10 The functions in may also be performed partly in software and partly in hardware by one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and other circuits that may incorporate software, hardware, middleware, and firmware to perform the functions described.

[0102] Return to reference Figure 1 , at one location by the processing system ( Fig.10 ) performs a series of intrinsic calibration steps. Starting with logic block 10, the camera to be calibrated takes a series of photos in which the target is placed at different positions, orientations and distances from the camera. It should be noted that the intrinsic calibration can be performed with the camera removed from the vehicle. In other configurations, the calibration can be performed using a camera mounted on the vehicle. The present disclosure is designed to cover both scenarios. In the illustrated embodiment, for ease of processing and simplicity of calibration, the cameras are considered to be calibrated separately.

[0103] Next, referring to logic block 20, an applicable processor performs intrinsic calibration. In intrinsic calibration, the calibration function can use intrinsic parameters to calculate the camera matrix. The intrinsic parameters may include, for example, the optical center and focal length of the camera, along with other parameters discussed herein. Based on the results of the intrinsic calibration process, control proceeds to logic block 30, where optimized intrinsic parameters (e.g., those mentioned above) are generated and temporarily cached. At that point, the optimized intrinsic parameters are flashed into the calibrated camera. That is, at logic block 40, the values ​​are stored in the camera's memory. In the final step, at logic block 50, the camera and any associated parts can then be shipped to the manufacturer's factory where the vehicle is assembled.

[0104] Figure 2 is a flow chart of an improved technique for calibrating a vehicle camera according to one aspect of the present disclosure. Figure 1 In some configurations, the functions of the improved techniques may be performed by an external computer component fixed at a certain location. Figure 2 The functions of may be performed using a camera mounted on the vehicle, although for the purposes of this illustration it is assumed that the camera is a standalone unit received from a supplier. Fig.10 In still other embodiments discussed, internal calibration may occur inside the vehicle so that the camera may be easily restarted in the event of an electrical problem or other error that inadvertently modifies the optimal values.

[0105] by Figure 2 Starting with logic block 10, a target is acquired by taking multiple images, where the selected target is located at different positions relative to the lens of the camera in question. Although the number of targets can vary depending on the size and resolution of the camera in question and its field of view, in one configuration, 10-16 targets with different positions and distances can be acquired. This number can vary depending on many factors.

[0106] Figure 1 One of the drawbacks of taking a photograph of the target at logic block 10 is that, eventually, the calibration value may be off due to variances discussed further below. Accordingly, to account for those variances, Figure 2 The process involves performing image morphology as described in the following figure. In practice, Figure 2 and Figure 1 Both show a high-level view of the general steps of internal reference calibration, but using Figure 2 The accuracy of the values ​​optimized by using morphology and related steps in its preparation, as described in logic block 25, is potentially significantly increased.

[0107] Figure 3300 is a detailed block diagram of a technique for calibrating a vehicle camera according to one aspect of the present disclosure. The camera to be calibrated in these embodiments is camera 104. From camera 104, at logic block 106 as in the previous figure, camera 104 is configured to acquire images with targets placed at multiple locations until a threshold number of images suitable for the camera's resolution and field of view are taken. As noted, in some embodiments, approximately 10-16 photos may be acquired, although other values ​​are possible. After logic block 106, we are left with multiple images with targets at corresponding locations. As the calibration process proceeds, this information may be temporarily cached in memory. The cached image information may be subjected to a calibration process (not shown) in which MATLAB or a similar program is used to calibrate the photos from logic block 106. After the image information is calibrated, it may enter logic block 120. At logic block 120, the black and white levels of the image may be improved by tone mapping. Tone mapping is similar to bit plane slicing, in which an image digitally represented in terms of pixels may be represented in terms of bits. For example, in the case where the exemplary image includes an 8-bit representation for each pixel, there are 8 bits for each pixel. This eight-bit image is then decomposed into eight one-bit planes. In this example, plane 1 includes the lowest order bits for each pixel in the image, and plane 8 includes the highest order bits. The image at logic block 120 is also binarized. Image binarization is the conversion of a document image into a binary document image, where pixels are divided into one of two values: black or white. In essence, image binarization creates a binary black and white image from a grayscale image, which can be used to define objects in an image by separating the foreground from the background.

[0108] Thereafter, once the image is tone mapped and binarized, the method can filter for the target, as shown at logic block 130. Part of the filtering procedure for the target includes finding the edge of the target (logic block 110) and executing the procedure of logic block 108. That is, in addition to logic block 106, the camera 104 proceeds to the operations at logic block 108. These operations may require obtaining the horizontal field of view (HFOV) and vertical field of view (VFOV) of the lens based on the image of the camera. These parameters are useful in defining the boundaries of the calibration because the image from the camera will have the maximum HFOV and VFOV, and these measurements should be considered in the final calibration to increase its accuracy and ensure that each measurement falls within the boundaries defined by these parameters. Therefore, the main purpose of this logic block 110 is to identify the edges of the target.

[0109] In an exemplary embodiment, after finding the target edge at logic block 110, the processor may use the information obtained from logic block 108 and logic block 110 to perform an edge spread function (ESF) at logic block 190. The ESF describes how an image reproduces an edge, such as a change from a low intensity portion of an image to a high intensity portion. Next, referring to logic block 180, the processor uses the information obtained from the ESF to estimate the number of blurred pixels. In a blurring configuration, pixels closest to the center of the core are given more weight than pixels farther from the center. More simply stated, blurring is the process of performing a weighted average operation on the entire image. In some embodiments, blur is removed before morphology is performed on the image. Deblurring excludes unnecessary target areas from the image and limits the image to only target coordinates before the morphological process.

[0110] Moving to logic block 170, the ESF can provide the point size of the target from the image being analyzed, and this is a calibratable value. The structural element can be used to calculate the kernel. In contrast, the structural element or constructed element is a digital form of the shape used in the target. The kernel will vary based on the size of the target. In various embodiments, the kernel is a calibratable value that can be transmitted to the camera as a calibration value. In one example, the kernel can form a circle. With the kernel at logic block 170, and having been filtered and deblurred for the target at logic block 130, the processor can perform morphology at logic block 140.

[0111] As from the following Figure 4 and Figure 5 Obviously, image morphology involves taking each object (here, shaped as a circle or a point) and equalizing their size using calibratable values. Thus, objects that are too small are increased in size, and objects that are too large are reduced. The center of the object helps determine the accuracy of the camera calibration. The objects can be other shapes (e.g., squares, rectangles, etc.), but the principle remains the same. There are different types of morphology, and they may involve opening, closing, erosion, and dilation. Pairs of these attributes can be used. For example, opening and dilation can be used in one technique. Closing and erosion can be used in another technique.

[0112] Once each target is equalized and identified, intrinsic calibration may occur at logic block 150, where pixels in the camera other than the target are converted to black. According to this procedure, the output image may be compiled, and intrinsic calibration may continue until the final intrinsic values ​​are determined. Then, at logic block 160, the calibration values ​​from the initial to the final principal point change may be sent to logic block 102, where they are "flashed" or read into the applicable camera. Control then returns to camera 104, where the next camera (if any) is ready for calibration. The process is repeated for each camera in a manner summarized according to an example embodiment, as follows:

[0113] Key steps of a process according to one embodiment

[0114] Step 1: Get camera images in different locations from camera vendors

[0115] Step 2: Improve black and white levels by enhancing tone mapping

[0116] Step 3: Binarize the image

[0117] Step 4: Define the structural element based on the point size of the object from the image (a calibratable value)

[0118] Step 5: Perform image morphology

[0119] Step 6: Filter the target and modify every other pixel to black in the camera.

[0120] Step 7: Compile the output image

[0121] Step 8: Continue the calibration process

[0122] Step 9: Determine the internal parameter value

[0123] Step 10: Change the main point (calibrable value)

[0124] Step 11: Flash Camera with New Intrinsic Parameters

[0125] Figure 4 4 is an example image representation before morphology according to one aspect of the present disclosure. As can be seen, there is a considerable amount of grayscale noise in the image, and it is unclear whether each roughly circular target is properly aligned with the camera's HFOV and VFOV. As an example, it is generally understood that targets in a camera's calibration image should be circular targets, because each target 30 appears like this.

[0126] Figure 5 is a representation 5 of the image that has been subjected to the representation to modify the objects to their correct alignment in the image. That is, Figure 5 Corresponding to the following morphological process Figure 4 , the morphological process aligns the circles according to the camera's VFOV and HFOV and shapes them appropriately into circles. In addition, according to ESF, Figure 4 The associated blur has been removed to provide a clear demarcation of the target 31 .

[0127] Figure 6 is the representation of the image that has undergone target enhancement6. That is, Figure 6 Image representation Figure 5 except that some objects are smaller than others and have uneven shapes. For example, the initial representation of object 30 appears more like a "C" than a circle. Figure 7 is a representation of Image 7 after object augmentation has been applied, with aligned and resized deformation alerts. Each object is circular, has a suitable center point, and is aligned with the camera's field of view. Figure 7 The information in can be used to change the initial values ​​to calibrated values, which can be flashed into the camera as mentioned above. It should also be noted that the interior of the filters have been converted to black to match them with the Figure 4 and Figure 5 Due to the almost black nature of the image, Figure 4 and Figure 5 The values ​​represented in are more difficult to visualize.

[0128] Figure 8 is a table describing factors associated with differences in internal reference calibration before and after a morphology process according to an embodiment of the present disclosure. Figure 8 The upper table above shows the calibration method (including the number of images), the morphological values, the identity of the chip, and a list of calibration parameters, including output image size, focal length, principal point, and distortion coefficients, and the magnitude of their change from initial to final values. The lower table, which is a horizontal extension of the table shown by the letter "A", includes a list of principal points and distortion coefficients for the three cameras.

[0129] Fig. 9 is a plan view of a vehicle and the battery system coupled to an electronic control unit (ECU) to control various aspects of the battery. Fig. 9 In an embodiment of the present invention, calibration is performed by an ECU inside the vehicle. Advantageously, whenever the calibration is deemed to have been degraded for whatever reason, or even at power-up, the ECU can apply the principles of the present disclosure to recalibrate each camera on the vehicle. That is, calibration can be performed on the vehicle using the described onboard electronics.

[0130] Although Fig. 9An electric vehicle is shown in FIG, but it will be appreciated that the present disclosure is not limited thereto and that the internal calibration procedure may be performed by any vehicle having appropriately programmed circuitry. Although the above hysteresis model may be applied to many different physical configurations, Fig. 9 One such example is shown. Fig. 9 Depicts a high voltage battery pack (B HV ) 912, for which an SOC estimate can be made. In a non-limiting example, the battery pack 912 may be embodied as a high capacity battery having a voltage capacity of approximately 400-800 volts or more, wherein the actual voltage capacity of the battery pack 912 is provided based on the desired operating / SOC range, gross weight, and power rating of the load connected to the battery pack 912. In a possible construction, the battery pack 912 may be a propulsion battery pack, which is typically composed of an array of lithium-ion or lithium-ion polymer rechargeable electrochemical cells, which may be cylindrical cells. The present teachings may also be applied to prismatic cells, and in possible configurations may be applied to pouch-type cells, and thus cylindrical cells are exemplary without limitation.

[0131] Although the internal details of the battery cells in the battery pack 912 are omitted for simplicity of illustration, it will be appreciated by those skilled in the art that the battery cells contain electrolyte materials, working electrodes in the form of cathodes and anodes, and permeable separators (not shown) within the cell cavities, which are packaged together in an electrically insulating can or housing. Grouped battery cells can be connected in series or in parallel using electrical interconnect boards and associated buses, sensing hardware, and power electronics (not shown, but well known in the art). A dedicated number of battery cells in the battery pack 912 can be arranged in columns and rows relative to the battery tray 913. In a nominal "xyz" Cartesian reference frame, for example, when viewed from above or below, the battery tray 913 can have a length (x dimension) and a width (y direction), with a height (z dimension) extending in an orthogonal direction away from the battery tray 913.

[0132] In a representative use case, the electrified power system 910 may be used as part of an EV 911 or another mobile system. As shown, the EV 911 may be embodied as a battery electric vehicle, wherein the present teachings may also be extended to plug-in hybrid electric vehicles. Alternatively, the electrified power system 910 may be used as part of another mobile system, such as, but not limited to, a rail vehicle, an aircraft, a ship, a robot, farm equipment, etc. Likewise, the electrified power system 910 may be stationary, such as in the case of a powerplant, a crane, a drive belt, or a conveyor system. Thus, Fig. 9 The electrified powertrain 910 in the representative vehicle embodiment is intended to illustrate the present teachings and not to limit the present teachings.

[0133] Fig. 9 EV 911 shown in FIG. 9 includes a body 922 and wheels 924F and 924R, where "F" and "R" indicate respective front and rear positions. Wheels 924F and 924R rotate about respective axes 925 and 950, where wheels 924F, wheels 924R, or both are driven by a rotating electric machine (M) from electrified powertrain 910. E ) Output torque of 926 (arrow T O ) to propel (power), as indicated by arrow

[24] . Thus, in this embodiment, wheels 924F and 924R represent mechanical loads, where other possible mechanical loads are possible in different host systems. To this end, the electrified powertrain 910 includes a power inverter module (PIM) 928 and a high voltage battery pack 912, such as a multi-cell lithium-ion propulsion battery or a battery having another chemistry suitable for the application, both of which are arranged on a high voltage DC bus 927. As is appreciated in the art, the PIM 928 includes a DC side (980) and an alternating current (AC) side 920, where the latter is connected to individual phase windings (not shown) of the rotating electrical machine 926 when the rotating electrical machine 926 is configured as a multi-phase rotating electrical machine in the form of a propulsion or traction motor as shown.

[0134] Fig. 9 The battery pack 912 is then connected to the DC side 980 of the PIM 928, so that the battery voltage from the battery pack 912 is provided to the power inverter module (PIM) 928 during the propulsion mode of the EV 911. The PIM 928, or more precisely a set of semiconductor switches (not shown) resident therein, is controlled via pulse width modulation, pulse density modulation, or other suitable switching control techniques to convert the DC input voltage on the DC bus 927 into an AC output voltage suitable for energizing the high-voltage AC bus 920. Therefore, the high-speed switching of the resident semiconductor switches of the PIM 928 ultimately energizes the rotating motor 926, thereby causing the rotating motor 926 to output torque (arrow T in other embodiments). O ) as motor drive torque delivered to one or more wheels 924F and / or 924R of another coupled mechanical load.

[0135] The electrical components of the electrified powertrain 910 may also include an accessory power module (APM) 929 and an auxiliary battery (B AUX) 930. As is appreciated in the art, the APM 929 is configured as a DC-DC converter connected to the DC bus 927. In operation, the APM 929 is capable of reducing the voltage level on the DC bus 927 via internal switching and voltage conversion to a lower level suitable for charging the auxiliary battery 930 and / or supplying low voltage power to one or more accessories (not shown) such as lights, displays, etc. Thus, "high voltage" refers to voltage levels that far exceed the typical 12-15V low / auxiliary voltage levels, with 400V or more being exemplary high voltage levels in some embodiments of the battery pack 912.

[0136] In some configurations, Fig. 9 The electrified powertrain 910 may include an onboard charger (OBC) 932 that is selectively connectable to an off-board charging station 933 via an input / output (I / O) block 932A during a charging mode in which the battery pack 912 is charged by an AC charging voltage (V CH ) recharge. The I / O block 932 can be connected to the charging port 917 on the vehicle body 922. For example, the charging cable 935 can be connected to the charging port 917, such as via an SAE J1772 connection. As appreciated in the art, in one or more embodiments, the electrified powertrain 910 can also be configured to selectively receive a DC charging voltage, in which case the OBC 932 will be selectively bypassed using a circuit (not shown), for example, which can be used to gradually charge and / or discharge the battery pack 912 to perform various functions, such as testing the SOC. The OBC 932 can also operate in different modes, including a charging mode and a charging mode indicated by arrow V X denoted as a discharge mode, during which the OBC 932 receives an AC charging voltage (V CH ) to recharge the battery pack 912 after the low battery indicator light is displayed on the instrument panel, during which the OBC 932 unloads power from the battery pack 912 to the external AC electrical load (L). In this way, the OBC 932 can embody a bidirectional charger.

[0137] Still reference Fig. 9 The electrified powertrain 910 may also include an electronic control unit (ECU) 934. The ECU 934 is operable to transmit electronic control signals (arrow CC) via O ) to regulate the ongoing operation of the electrified powertrain 910. The ECU 934 responds to electronic input signals (arrow CC I ) to do so. In various embodiments, such an input signal (arrow CC I) can be actively transmitted or passively detected, so that ECU 934 is operable to determine a specific operating mode. In response, ECU 934 controls the operation of electrified power system 910. Therefore, ECU and its auxiliary components can act as a BMS to perform functions including estimating SOC, etc.

[0138] To this end, the ECU 934 may be equipped with one or more processors (P), such as logic circuits, (multiple) combinational logic circuits, (multiple) application specific integrated circuits (ASICs), (multiple) electronic circuits, (multiple) central processing units, semiconductor IC devices, etc., as well as (multiple) input / output (I / O) circuits, appropriate signal conditioning and buffering circuits, and other components such as high-speed clocks to provide the SOC functions described in the previous figures, as well as different functions identified by CC input signals. The ECU 934 also includes relevant computer-readable storage media, that is, memory (M) including read-only, programmable read-only, random access, hard disk drives, etc., regardless of whether it is resident, remote, or a combination of the two. Control routines (including code for executing a SOC model with hysteresis) are executed by the processor to monitor relevant inputs from sensing devices and other networked control modules (not shown), and execute control and diagnostic routines to manage the operation of the electrified power system 910. The I / O circuits may be directly coupled to the ECU 934 along with the memory M and one or more processors P for executing code for estimating the SOC. In one aspect, the BMS system can be collectively implemented as ECU 934, OBC 932, and bus 927. OBC 932 and bus 927 can be devices within the BMS, or included as part of the BMS, which are enabled to connect to the external terminals of the battery pack 912 to perform the functions described herein. In some embodiments, the BMS can be directly coupled to the battery pack.

[0139] It is noted that the EV 911 may include a dashboard implanted within or otherwise connected to the body of the EV 911, like other vehicles. The body houses the cab, where the driver and any passengers reside. The apparatus discussed above may include control signals to the dashboard and conversion circuitry to enable the driver to estimate the remaining SOC based on the amount of charge remaining or the percentage of charge remaining, the estimated time until the vehicle will be scrapped or urgently needs to be recharged, and other data. At least some of these aspects may be calculated by the BMS, including the ECU 934 and its associated processor P running code from the memory M. Messages may be sent to other parts of the vehicle via the I / O circuitry, via CC0 or another connection not specifically shown.

[0140] In another embodiment, the ECU 934, along with its I / O, memory, and processor, may additionally or alternatively be used to calibrate and recalibrate each camera or selected cameras in the EV. In this case, the flowchart above may be run on the processor and the ECU 934 may be appropriately connected to perform calibration for each camera. This may occur during a suspected miscalibration caused by a force event, or it may simply be recalibrated every X times the driver turns on the EV 911. It should be noted again that another type of combustion-based or hybrid vehicle may be used in this embodiment. This embodiment also eliminates the expensive and time-consuming need to perform calibration independently for each vehicle at startup. The ECU 934 may be coupled to each camera via a hardwired or networked connection, or it may be connected to selected cameras.

[0141] Fig.10 is a plan view of a designated location 1080 where intrinsic calibration of a vehicle camera is performed, including hardware devices that may be used for calibration. It should be noted initially that in many embodiments, the vehicle itself is not used, and the location 1080 is limited to the calibration hardware and the camera(s) requiring intrinsic calibration. However, in other embodiments where serial calibration of the camera is preferred, similar to Fig.10 The configuration in may be preferred.

[0142] Here, vehicle 1060 includes a backup camera 1086, which happens to have been installed. Camera 1086 is connected via hardware connection 1051 (or wireless connection) to computer 1050, which can be any kind of suitable computing device as discussed in detail above. For example, computer 1050 may include a processor P, which is defined as one or more processors of the above type or other types. P is connected to a first memory M1, which can be DRAM or flash memory for storing code and quickly supplying code to P to perform calibration measures. M2 can be a non-volatile memory, such as a solid state drive (SSD), a magnetic disk drive (MDD), flash memory or any other sufficient memory for storing calibration data received as a result of calculations, and the data can be read into the applicable camera via the correct connection. The internal bus 1082 may be used to connect to the transceiver 1025, which in turn is coupled to one such connection 1051 or network path for receiving calibration data and for storing changed calibration data in the camera's memory, or in dedicated memory of the vehicle 1060 storing the memory of a specific camera.

[0143] The detailed description and the accompanying drawings or figures are support and description of the present teaching, but the scope of the present teaching is limited only by the claims. Although some best modes and other embodiments for implementing the present teaching have been described in detail, there are various alternative designs and embodiments for practicing the present teaching defined in the appended claims. In addition, the present disclosure explicitly includes combinations and sub-combinations of the elements and features presented above and below.

Claims

1. A device for calibrating the intrinsic parameters of a camera, comprising: Memory for storing images and executable code; and an image processor coupled to the memory and configured to: acquiring a plurality of images of the target captured by the camera at different positions for storage in a memory; For each of the multiple images: Tone mapping of images to improve black and white image quality; Binarize the tone-mapped image; Defining structural elements based on the digital form of the target shape; Perform morphology on images to align and demarcate structural elements; Filter the target and modify all other pixels to black; compiling an output image including the filtered target; Use the compiled output image to determine the intrinsic parameter value; Changing the principal point when using a target for internal calibration; and The newly changed primary point is stored in memory.

2. The apparatus of claim 1, wherein the image processor is configured as a computational core to define a structure element for use in performing morphology, the structure element being a digital form of a shape of an object used by the camera.

3. The apparatus according to claim 2, wherein the image processor is configured to: Identify the horizontal field of view (HFOV) and vertical field of view (VFOV) based on the camera image; locating target edges for storage in memory; performing an edge spread function (ESF) using the target edge; and The number of blurry pixels is estimated based on the ESF.

4. The apparatus of claim 1 , wherein the image processor is configured to use calibration differences before and after calibrating the structuring process to include (1) a calibration method, (2) a morphological process, (3) an output pixel size and height, (4) a focal length, (5) a principal point, and (6) a distortion coefficient.

5. The apparatus of claim 4, wherein calibration parameters used in the calibration method include pixel size, focal length, principal point and distortion coefficient.

6. The device according to claim 1, wherein: The image processor is configured to add an internal calibration to incorporate the time-sequential morphological process into the target image count, thereby achieving accurate internal calibration and detecting and monitoring changes in the target image in detail.

7. A vehicle comprising: a vehicle body having a camera mounted on an exterior thereof; a cab housed within the vehicle body; The instrument panel near the front of the vehicle interior; Memory for storing images and executable code within the vehicle body; an image processor coupled to the memory and configured to: acquiring a plurality of images of the target captured by the camera at different positions for storage in a memory; For each of the multiple images: Tone mapping of images to improve black and white images; Binarize the tone-mapped image; Defining structural elements based on the digital form of the target shape; Perform morphology on images to align and demarcate structural elements; Filter the target and modify all other pixels to black; compiling an output image including the filtered target; Use the compiled output image to determine the intrinsic parameter value; Changing the principal point when using a target for internal calibration; and The newly changed primary point is stored in memory.

8. The vehicle according to claim 7, wherein: The image processor is configured as a computational core to define a structure element for use in performing morphology, the structure element being a digital form of a shape for an object used by the camera.

9. The vehicle according to claim 8, wherein: The image processor is configured to: Identify the horizontal field of view (HFOV) and vertical field of view (VFOV) based on the camera image; locating target edges for storage in memory; Using the found edges to perform an edge spread function (ESF); and The number of blurry pixels is estimated based on the ESF.

10. The vehicle according to claim 7, wherein: The image processor is configured to use calibration differences before and after a calibration process to include (1) a calibration method, (2) a morphological process, (3) an output pixel size and height, (4) a focal length, (5) a principal point, and (6) distortion coefficients.

Images