Camera system safety calibration method and camera pod assembly

By creating and storing hash values ​​in the camera system and performing encrypted matching, the problems of non-compliant images and network security caused by component damage or removal in camera surveillance systems are solved, enabling secure calibration and simplifying the assembly process.

CN116209994BActive Publication Date: 2026-11-10MOTHERSON INNOVATIONS CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202180062794.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-09-15
Filing Date
2021-09-15
Publication Date
2026-11-10
Estimated Expiration
2041-09-15

AI Technical Summary

Technical Problem

Existing camera surveillance systems in vehicles are susceptible to damage, defects, or removal of system components, resulting in images that do not meet mandatory requirements and are vulnerable to cyberattacks. Furthermore, the calibration process consumes assembly space and time.

Method used

Secure calibration of the camera system is achieved by creating and storing hash values ​​of camera modules and camera pod components, using encryption strategies for matching and verification, and transmitting calibration data via wireless or wired connections using NFC, WLAN, 5G, USB, LIN bus, storage device readers, Bluetooth, or LAN.

Benefits of technology

It improves network security and image quality of the camera system, ensures the integrity of calibration data, prevents unauthorized replacement and tampering, simplifies the calibration process, and reduces assembly time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116209994B_ABST
    Figure CN116209994B_ABST
Patent Text Reader

Abstract

The present invention relates to a security setting method for a camera system (20) for a vehicle (18), comprising the steps of: creating a camera system (20) having a first camera module (12) with a first identifier, a second camera module (14) with a second identifier, and an electronic control unit (ECU); creating at least one first camera module hash value (H12) for the first identifier, at least one second camera module hash value (H14) based on the second identifier, and / or at least one camera pod assembly (10) based on the first and second identifiers; and storing at least one of the hash values. The invention also relates to a camera pod assembly (10) comprising at least two camera modules (12, 14) attached to a vehicle (18) via a mounting element (16) and suitable for use in such a security setting method; a camera system (20) comprising at least one such camera pod assembly (10); and a vehicle (18) comprising such a camera system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure generally relates to a security setting method for a camera surveillance system for a vehicle, a camera pod assembly suitable for using such a security setting method, a camera system having such a camera pod assembly, and a vehicle using such a camera system. More specifically, the invention relates to system calibration for various elements used to provide network security. Background Technology

[0002] The statements in this section provide only background information in connection with this disclosure and do not constitute prior art.

[0003] Vehicles such as cars, vans, and trucks may include camera surveillance systems (CMS). Camera surveillance systems can be used to detect objects or provide images of the area around the vehicle to support safety systems or replace rear-view devices. Due to component differences, CMS solutions typically use individual system calibrations to meet mandatory requirements and operate correctly. This usually involves several calibration steps at different stages of production to ensure the system is consistent with specifications and regulations. This approach also consumes assembly space and time during vehicle manufacturing.

[0004] Camera surveillance systems become an integral part of vehicle safety operation when used in place of mirrors to monitor the area around the vehicle. These systems can be vulnerable to changes in system integrity. If one of the system components is damaged, defective, or removed, the camera surveillance system may be compromised. This could result in images generated by the camera surveillance system failing to meet mandatory requirements or make the system susceptible to cyberattacks.

[0005] The objective of this disclosure is to provide a calibration strategy for CMS that overcomes at least some of the problems stated above. Summary of the Invention

[0006] This disclosure relates to a security setting method for a camera system for a vehicle, comprising the following steps: creating a camera system having at least a first camera module with a first identifier, a second camera module with a second identifier, and an electronic control power supply (ECU); creating at least one hash value for the first camera module based on the first identifier, at least one hash value for the second module based on second verification data, and / or at least one hash value for a camera pod assembly based on the first and second identifiers; and storing at least one of the hash values, including storing at least one first hash value in the second camera module and at least one second hash value in the first camera module and / or storing at least one camera pod assembly hash value in the camera pod assembly and / or storing at least one first hash value and at least one second hash value in pairs and / or storing at least one first hash value and at least one second hash value in the ECU as an ECU hash value.

[0007] According to one aspect, the method further includes the following (one or more) steps: encrypting at least one of the hash values ​​before storing and / or transmitting the at least one hash value; and / or monitoring and / or verifying at least one first hash value stored in the second camera module and at least one second hash value stored in the first camera module by matching it with an ECU hash value stored in the ECU; and / or transmitting calibration data to the camera pod assembly and / or the camera module and / or the ECU, wherein the transmission is via a wireless or wired connection.

[0008] It is recommended that the monitoring step occur upon vehicle startup; and / or the calibration data for the transmission step is defined by calibration datasets assigned to each camera module and / or camera pod assembly, wherein each calibration dataset is assigned a specific calibration data identifier; and / or the transmission step uses a data transmission interface, which preferably enables data transmission standards in the form of NFC, WLAN, 5G, USB, LIN bus, storage device reader, Bluetooth, or LAN.

[0009] Furthermore, the method may include the following steps(s): displaying a warning if the ECU hash value does not match at least one first or second hash value or a camera pod assembly hash value; and / or creating at least one calibration data hash value for at least one calibration data identifier and storing the at least one calibration data hash value in the ECU and / or camera module and / or camera pod assembly, preferably displaying a warning if at least one calibration data hash value stored in at least one of the camera module and / or camera pod assembly does not match at least one calibration data hash value stored in the ECU.

[0010] The method is also suggested to include one or more of the following steps: storing at least one calibration dataset in at least one storage device; and / or simultaneously or independently providing a camera pod assembly and at least one calibration dataset, wherein preferably the at least one calibration dataset and the camera pod assembly are delivered independently to the OEM. Storage devices may be selected to include cloud, flash drives, hard drives, CDs, DVDs, printable codes, chips, and / or mobile devices.

[0011] At least one of the hash values ​​may include several identifiers and / or several hash values ​​and / or portions of several identifiers and / or portions of several hash values; and / or the first and second hash values ​​may additionally include information about the calibration dataset; and / or the first and second hash values ​​may include at least two hash values, wherein at least one first hash value determines the hardware module of the camera pod assembly and at least one second hash value determines the calibration dataset; wherein the matching, preferably performed during verification and / or monitoring and / or displaying warnings(one or more) steps, is further based on at least one calibration data hash value, specifically, this at least one calibration data hash value is at least partially stored in at least one part of the ECU and / or the camera module and / or the camera pod assembly.

[0012] According to one aspect, it is recommended that the calibration data be transmitted wirelessly from the camera pod assembly to the ECU; and / or that calibration data updates be checked, wherein if calibration data updates are available, a warning is preferably displayed; and / or that the new calibration dataset is transmitted as an update to the camera pod assembly, at least one of the camera modules, and / or the ECU.

[0013] Furthermore, it is recommended that the camera system include more than two camera modules and / or calibration datasets for creating, storing, monitoring, and verifying corresponding hash values; and / or at least two hash values ​​are combined, specifically combined into a single hash value or a defined list of hash values; and / or at least one of the hash values ​​includes information about the stored calibration dataset; and / or at least one identifier is a serial number; and / or at least one calibration data hash value is stored in each camera module, such that calibration data hash values ​​are cross-stored in the camera modules so that each camera module stores a calibration data hash value related to another camera module.

[0014] The method disclosed herein can provide a system calibration method for a camera system, including steps for calibrating individual camera modules, steps for calibrating the camera system, and steps for calibrating a vehicle camera.

[0015] The steps for camera system calibration may include steps for storing the assigned serial numbers of each camera module, steps for installing the camera modules in the camera system, steps for cross-storing or cross-verifying camera module pairings, steps for checking the existence of stored pairing values, steps for matching hash values, and steps for assigning camera modules as calibrated.

[0016] In one aspect, the method can be applied to a camera surveillance system component for monitoring security parameters, including steps for checking the presence of stored paired hash values, steps for matching hash values, and steps for assigning a camera module as calibrated, or displaying a camera calibration warning and / or for authorizing recalibration, wherein preferably the process then returns to the step of checking the presence of stored paired hash values.

[0017] According to another approach, at least one camera module or camera pod component can be replaced by the following steps: replacing at least one camera module or camera pod component, creating a new hash value for the replacement camera module or camera pod component, creating a new calibration data hash value for the replacement camera module or camera pod component, and storing the new hash value for the replacement camera module or camera pod component to replace the old calibration data and hash value.

[0018] This method can also be applied to calibrating camera pod assemblies and ECUs using a database, including steps for storing calibration data, for pairing camera modules of the camera pod assembly, for creating a database snapshot, for delivering the paired camera pod assembly, display unit, and calibration database snapshot independently, for sending information about the pairing of the camera pod assembly and display unit with the database snapshot, and for retrieving calibration data from the database snapshot and providing it to the ECU.

[0019] This method allows for recalibration without transmitting calibration data from the camera pod assembly via a coaxial cable. The method includes steps for checking the camera system and for checking the presence of data on the ECU, steps for checking the safety of the camera system if data is present on the ECU, or steps for obtaining information data from the camera module if data is absent on the ECU but present on the camera module, and steps for providing calibration data to the camera module specifically via a wired or wireless connection. Preferably, this then returns to the steps for checking the camera system and for checking the presence of data on the ECU, steps for checking the serial number of the camera module and / or the camera pod assembly, and steps for updating the calibration data of the ECU by providing calibration data to the ECU if a new serial number for the camera module and / or the camera pod assembly exists. Preferably, this then returns to the steps for checking the camera system and for checking the presence of data on the ECU, or steps for using the camera module without updating if a new serial number for the camera module and / or the camera pod assembly does not exist.

[0020] A camera pod assembly is also provided, comprising at least two camera modules attached to a vehicle via mounting elements, and is suitable for use with the safety setup method of this disclosure. The storage device may be a flash drive, hard disk, CD, DVD, printable code, and / or chip.

[0021] Furthermore, a camera system is provided, comprising at least one camera pod component and an ECU as disclosed herein. Storage devices may be provided by the cloud and / or mobile devices.

[0022] Furthermore, a vehicle is provided that includes the camera system disclosed herein and at least one display unit. The ECU may be provided by the display unit.

[0023] It should be noted that features individually stated in the following description may be combined with each other in any technically advantageous manner and present other forms of this disclosure. The disclosure is further characterized and described in detail with reference to the accompanying drawings. Attached Figure Description

[0024] In order to enable this disclosure to be fully understood, its various forms will now be described by way of example with reference to the accompanying drawings, wherein: Figure 1 A front view of an exemplary camera pod assembly according to this disclosure is depicted; Figure 2 Examples of components of a camera system according to this disclosure are schematically illustrated; Figure 3 An exemplary method for performing a system calibration method for a camera system in a vehicle is illustrated. Figure 4 An exemplary method for calibrating security settings of a camera surveillance system component according to this disclosure is illustrated; and Figure 5 An exemplary method for a camera surveillance system component for monitoring security parameters, according to the present disclosure, is illustrated. Figure 6 An example of components of a camera system having a replacement camera module according to the present disclosure is schematically illustrated; Figure 7 An exemplary method for calibrating a camera pod assembly and an ECU using a database is illustrated. Figure 8 An exemplary method for recalibrating an ECU without transmitting calibration data from a camera pod assembly via a coaxial cable is illustrated. Detailed Implementation

[0025] The following description is exemplary in nature only and is not intended to limit this disclosure, application, or use. It should be understood that throughout the accompanying drawings, corresponding reference numerals indicate corresponding parts and features.

[0026] Figure 1 An exemplary camera pod assembly 10 on a vehicle 18 is illustrated. The camera pod assembly 10 includes mounting elements 16 that accommodate at least two camera modules 12, 14. In this form, the camera pod assembly has a first camera module 12 and a second camera module 14. In other forms, the camera pod assembly 10 may contain more than two camera modules. In this form, the first camera module 12 and the second camera module 14 have different camera parameters, such as field of view (FOV). Camera module parameters within the scope of this disclosure may be the same. Figure 1 The exemplary camera pod assembly 10 shown is mounted on the exterior of vehicle 18. The camera pod assembly 10 can be located anywhere outside the vehicle or inside the vehicle and utilizes the camera calibration methods described in this disclosure.

[0027] Figure 2 The components for camera system 20 are illustrated. Camera system 20 includes at least two camera modules 12, 14 and an electronic control unit (ECU) 26. Camera system 20 is not necessarily a single-structure component. At least two camera modules 12, 14 have camera flash memory 22, 24 respectively for storing camera-specific calibrations based on the characteristics of the mounted lens systems 30, 31. For example, the first camera module 12 includes a first camera module flash memory 22 for storing internal calibrations associated with the first lens system 30. The second camera module 14 includes a second camera flash memory 24 associated with the second camera system 31. This specific internal calibration is referred to below as part of step 200. Figure 3 Further discussion. The first and second camera modules 12 and 14 each also include a system calibration storage device 28. The system calibration storage device 28 may be part of the camera flash memory 22 and 24 or a separate storage area as part of the camera module. The system calibration storage device 28 may be a read / write storage device, for example, capable of storing information received from the camera flash memory 22 and 24 and / or the electronic control unit (ECU) 26 and sending information to the ECU 26 upon request. The ECU 26 receives information from the system calibration storage device 28 and returns the information to the camera modules 12 and 14 for subsequent storage in the system calibration storage device 28. The information may include data identifying calibration data. The ECU 26 includes a storage device for storing calibration data.

[0028] Figure 3The steps of a vehicle camera system calibration method 100 are illustrated. Method 100 calibrates at least two camera modules 12, 14 in a camera pod assembly 10. Step 200 is a camera calibration step for the individual camera calibration of camera modules 12, 14 in the camera pod assembly 10. Each lens system 30, 31 includes a lens surface with a given curvature that allows a wide area or field of view (FOV) to be captured in front of the respective camera, which can also be described in terms of the field of view cone. As the FOV increases in terms of the angular size of the field of view cone, the curvature of the lens surface increases accordingly, affecting the distortion of the captured FOV. The calibration in step 200 approximates the lens distortion via a mathematical model so as not to distort the image. One application of this is, for example, after a post-processing step of the captured image, for each straight line on the captured target to appear as a straight line.

[0029] The calibration of the first camera module 12 and the second camera module 14 can be completed during the manufacturing of the respective cameras and stored internally in the camera flash memory 22, 24 for use in step 300. Furthermore, the calibration data can be stored in an additional storage device that may need to transfer the calibration data to a storage device integrated into the ECU 26. The additional storage device can be a portable device such as a USB flash drive, or it can be the cloud. In some embodiments, the additional storage device will be delivered with the camera pod assembly 10 or with any components of the camera pod assembly 10 that must be modified. In another embodiment, the additional storage device is the cloud, and access to appropriate data for the calibration data of the camera pod assembly 10 or its components is thereby sent to an authorized service provider or vehicle user for access to the required calibration data via the cloud.

[0030] Step 300 is the camera system calibration step used in the vehicle camera system calibration method 100. In step 300, at least two camera modules 12, 14 have completed the stored individual camera module calibrations that began in step 200. Step 300 calibrates the camera pod assembly 10 to combine the individual images from the first and second camera modules 12, 14 and produce a unified camera system image. In this form, at least two camera modules 12, 14 with different view cones (i.e., different lenses) are combined in the camera pod assembly 10. For example, the first camera module 12 with a wide-angle FOV faces the ground, covering a large area next to the vehicle, while the second camera module 14 with a small-angle FOV faces the area behind the vehicle 18. The first and second camera modules 12, 14 are positioned so that the camera modules in the camera pod assembly 10 have a shared view area. If the position and orientation of the camera modules are known, it is possible to calibrate the vehicle's camera system 20, combining multiple camera views into a unified view. The obtained calibration data can be stored in an additional storage device, which may need to transfer the calibration data to the storage device integrated into the ECU26.

[0031] Step 300 includes what is termed an effective external calibration between the camera pod assembly 10 and the vehicle 18 and / or further vehicle components. According to this disclosure, the camera pod assembly 10 and the display unit, along with a calibration database that can be stored on an additional storage device, can be delivered independently to the OEM. The camera pod assembly 10 and the display unit (not shown) can be installed in the vehicle. The OEM then retrieves the calibration data corresponding to the delivered camera pod assembly 10 and can provide this data to a storage device integrated into the ECU 26. The corresponding calibration data is then permanently stored on a storage device available on the ECU 26. The calibration data can be provided to the display unit via air (air flash) or via a storage device and / or other data transmission means. This excludes the option of retrieving calibration data between the camera pod assembly 10 and the ECU 26 via, for example, a coaxial cable. This provides a further safety measure, as authorized service partners require separate calibration data for any repairs.

[0032] Step 400 includes the final vehicle CMS calibration process. This is a calibration of parameters that occurs when the camera pod assembly 10 has been assembled into the vehicle 10. Taking into account variations such as component tolerances, the vehicle calibration in step 400 allows for fine-tuning of the images displayed by the camera system 20. The obtained calibration data may be stored in an additional storage device, which may need to transfer the calibration data to a storage device integrated into the ECU 26.

[0033] Figure 4An exemplary method for calibrating a camera surveillance system to provide camera module verification checks is illustrated. Verification calibration process 301 is a subprocess performed in step 300 described above. Component verification calibration process 301 can also be implemented in the camera surveillance system at any time after at least two camera modules 12, 14 have been installed in the camera pod assembly 10, which includes authorized component replacement services. Verification calibration process 301 provides the camera system 20 with settings to individually verify that the camera modules currently installed in the camera pod assembly 10 are the same camera modules identified during the camera system calibration process 300. For compliance and cybersecurity reasons, this verification calibration process 301 creates and stores unique identifiers assigned to individual cameras.

[0034] Vehicle operators are concerned about the cybersecurity of the vehicle's camera system 20 because it is replacing a conventional mirror system in the vehicle. A mirror replacement due to failure or damage can be replaced if the correct mechanical accessories are functional. Replacing a camera module is more complex. When replacing a camera module, camera pod assembly calibration issues can arise, leading to image sharpness problems and potential system security issues. If a camera module is replaced and calibration fails verification, the system may produce a confused view or a view that does not comply with current mandatory safety regulations. Cybersecurity issues or misuse can also arise due to providing unauthorized access to the camera system 20 with the replacement camera module 512. To maintain the overall quality and security of the camera system, the component verification calibration process 301 for the camera module in the camera pod assembly 10 allows the camera system 20 to determine if at least one unauthorized replacement of a camera module has occurred and if a corresponding authorized recalibration of the camera system 20 has not yet been performed.

[0035] Verification calibration process 301 begins with the first camera module 12 having a first storage identifier in its associated first camera module flash memory 22 and the second camera module 14 having a second storage identifier in its associated second camera module flash memory 24 (step 302). In the example below, each identifier is a serial number, but could be a QR code, etc. In addition to the identifier of the camera module itself, each camera module may have both associated calibration data that may include one or more datasets and at least one calibration data identifier that may be associated with the calibration of each installed camera module or camera system. These calibration data identifiers may include information about the associated hardware and the creation date of the calibration data.

[0036] Each assigned serial number belongs to a specific camera module and is generated during the manufacture of the camera module. Therefore, this serial number is part of the camera module before the camera module is installed in the camera pod assembly 10 in step 304.

[0037] Step 306 is a pairing step for providing the security check element in the system calibration storage device. In step 302, ECU 26 receives serial number and calibration data information from at least two camera modules 12, 14 and camera pod assembly 10 in the camera system 20 as a whole. In this way, ECU 26 will receive... Figure 2 The first camera module 12 receives a first serial number and a second serial number from the second camera module 14. Additionally, the ECU 26 receives calibration data that can combine the calibration data of each camera module 12, 14 with that of the camera system 20 and camera pod assembly into a single unit. In some embodiments, the calibration data may be separate.

[0038] In one embodiment, the serial numbers of all at least two camera modules 12, 14 are stored in a single hash value H2 of the camera pod assembly 10. In this way, the camera pod assembly 10 is processed in the same manner as a single camera module as described in the various embodiments of this disclosure.

[0039] In one embodiment, the serial number of the camera module is converted into a new hash value for the camera module. In this way, the first serial number from the first camera module 12 is converted into a hash value called H2, and the second serial number for the second camera module 14 is converted into a hash value H14. The hash value can be based on... Figure 2 The smaller portion of the camera sequence number generated by the camera serial number received by the first camera module 12 is used to create H2, or the smaller portion of the camera sequence number generated by the camera serial number received from the second camera module 14 is used to create H14.

[0040] Furthermore, at least one identifier of the calibration data can be converted into a new hash value for the camera pod assembly 10. In the case of several calibration data identifiers, in some embodiments, several hash values ​​or a single combined hash value can be created. The hash value can be a smaller portion of the calibration data identification number generated from the calibration data identifiers received from the calibration data to create a calibration data hash value (H4).

[0041] The creation of hash values ​​H12 and H14, along with the calibration information used in the verification process, enhances the system's security. Compared to the camera module serial number and / or calibration data identification number, it is more difficult for an unauthorized user to determine the correct hash values ​​for the camera module and calibration data, thus providing a first level of security. The created hash values ​​are also encrypted using an encryption strategy. This encryption of hash values ​​H12 and H14, and the hash values ​​for the calibration information, creates a second level of security for the camera system 20. The memory can be encrypted and cannot be read from blank text.

[0042] Once the hash value (e.g., H12, H14, or calibration data information hash value) has been created and encrypted, the system provides a third layer of security by storing the hash value (e.g., H12, H14, or calibration data information hash value) in the system calibration storage 28 of the paired camera modules 12, 14 in the camera system 20. The system additionally stores a combination of H12 and H14 in the memory of the ECU 26 for reference. In some embodiments, the system additionally stores a combination of H12 and H14 and at least one calibration data information hash value in the memory of the ECU 26 for reference.

[0043] In some embodiments, the first camera module 12, the second camera module 14, and the system calibration storage device 28 for the hash value will operate as follows: First camera module 12 storage The serial number from the first camera module 12, and H14 and / or Second camera module 14 storage Serial number from second camera module 14 H12 and / or The ECU stores the value h3, which can be defined as follows: h3 = hash(hash(serial number first camera module 12) || hash(serial number second camera module 14)) or h3 = (H12) || (H14) The cross-storage of camera module hash values ​​in paired camera modules and ECU 26 provides for the inspection of camera modules 12, 14 in camera system 20 and for third-level security of camera system 20. Alternatively, the first camera module 12 and the second camera module 14 may also include the storage of hash values ​​for all camera modules in camera system 20. Alternatively, the camera modules may store hash values ​​separately from their serial numbers.

[0044] In another embodiment, for the first camera module 12 and the second camera module 14, the system calibration storage device for the hash values ​​will operate as follows: First camera module 12 storage Serial number from the first camera module 12 H14, and At least one calibration data information hash value and / or Second camera module storage Serial number from second camera module 14 H12, and At least one hash value of calibration data information, and / or The ECU stores the value h3, which can be defined as follows: h3 = hash(hash(serial number of first camera module 12, at least one calibration data) || hash(serial number of second camera module 14, at least one calibration data)) or h3 = (H12, at least one hash value of calibration data) || (H14, at least one hash value of calibration data).

[0045] The cross-storage of camera module hash values ​​in paired camera modules and ECU 26 provides for the inspection of camera modules 12, 14 in camera system 20 and for third-level security of camera system 20. Cross-storage of hash values ​​can also be applied in embodiments where there is more than one calibration data information hash value (e.g., one per camera module). In another embodiment, the first camera module 12 and the second camera module 14 may also include the storage of hash values ​​for all camera modules and calibration data information in camera system 20. In yet another embodiment, camera modules may store hash values ​​separately from serial numbers and calibration data identifiers.

[0046] In one embodiment, in step 308, ECU 26 may check the camera modules and calibration data to verify the hash values ​​stored in the first and second camera modules 12, 14. If the values ​​returned from the first camera module 12 and the second camera module 14 match the expected hash values ​​stored in ECU 26, the paired camera pod assembly 10 and its calibration data are verified. Verification of the calibration data can be achieved by storing a set of calibration data in both the camera pod assembly 10 and the ECU. Therefore, information about the calibration data sent from the camera pod assembly 10 must satisfy the expected information of the calibration data stored in the ECU. The camera calibration matches the expected settings, and it is confirmed that the functionality of the camera system 20 has been calibrated for the camera system 20 (step 312). The memory of ECU 26 and the hash values ​​of each camera module are encrypted and cannot be read as blank text. For verification, any camera in the camera pod assembly 10 can also be separately cross-verified for their pairing verification, not just from ECU 26 alone. Once the first camera module 12 and the second camera module 14 are paired and verified, the system has the means to verify that the first and second camera modules 12 and 14, as well as the calibration data, are the camera modules and calibration data during the calibration period to meet the system rules.

[0047] Figure 5 An exemplary component verification monitoring process 500 is illustrated for monitoring camera surveillance system components and calibration data for safety parameters. This process takes action to monitor the individual camera modules and calibration data within the camera pod assembly 10 and verifies that the calibrated first and second camera modules 12, 14 are the correct camera modules, including the correct calibration data paired by an authorized calibration process. The component verification monitoring process 500 can run at any time to verify the installed camera modules. An exemplary time to implement the component verification monitoring process 500 is when the vehicle is started, to detect any potential changes to any camera modules within the service model or system during any maintenance. In this form, the component verification process will check whether the first and / or second camera modules 12, 14 have been changed and / or whether updated calibration data is available.

[0048] In step 502, ECU 26 may send a query to the camera module to verify the stored hash values ​​from other camera modules in the calibrated system. In this form, the H14 value will be verified from the first camera module 12, and the H12 value will be verified from the second camera module 14. In step 504, ECU 26 compares the received H12 value (from the second camera module 14) and the H14 value (from the first camera module 12) with the value of H3 stored in ECU 26. If the hash values ​​match the hash values ​​stored in the ECU, no change to the camera module is detected in step 506, and the camera module is designated as properly calibrated.

[0049] In an alternative to step 502, ECU 26 may send a query to the camera module to verify the hash values ​​stored in other camera modules within the calibrated system and the hash values ​​of the calibration data. In this form, the H14 value from the second camera module 14 and at least one calibration data hash value (H4) will be verified from the first camera module 12, and the H12 value from the first camera module 12 and at least one calibration data hash value (H4) will be verified from the second camera module 14. In step 504, ECU 26 will compare the received value with the value of H3 stored in ECU 26. If the hash value matches the hash value stored in the ECU, then in step 506, no change in the camera module or calibration data is detected, and the camera module is designated as properly calibrated.

[0050] In another alternative to step 502, ECU 26 may send a query to the camera module to verify the stored hash value from the calibration data. In this form, at least one calibration data hash value (H4) from the second camera module 14 will be verified from the first camera module 12, and at least one calibration data hash value (H4) from the first camera module 12 will be verified from the second camera module 14. In step 504, ECU 26 will compare the received value with the value of H3 stored in ECU 26. If the hash value matches the hash value stored in the ECU, no change to the camera module and calibration data is detected in step 506, and the camera module is designated as properly calibrated.

[0051] Step 508 initiates a camera calibration warning to the user. If the hash value returned from the first camera module 12 or the second camera module 14 and ECU 26 during verification step 504 does not match the expected hash value, the system warns of potential tampering, camera malfunction, calibration data alteration, or camera module replacement. In some embodiments, the warning may be delivered to the vehicle user by providing advice such as “Please upgrade the system,” “Please download the update,” “Please insert a suitable storage device to update the calibration data,” “Please use the service app and follow the steps to update the calibration data installed on the camera system,” “Please use some kind of display connected to the vehicle and follow the steps to update the calibration data,” or “Please connect a suitable device to the vehicle via Bluetooth, NFC, or a similar standard and perform a calibration data update.” Detected changes can affect the overall calibration of the system and image display. In one embodiment, when a camera module is replaced, the information stored in the camera module is lost; therefore, the stored hash value will not be available in the system. The camera module is found in an authorized replacement or stolen device. After the system confirms the verification error, a warning will be initiated and displayed (step 508). This warning may be displayed to the vehicle operator at one or more locations within vehicle 18. Display locations may include the head-up display, center console, dashboard, any external or internal display surface, or mobile device warning. This display will notify the vehicle user that a camera calibration problem that could affect the functionality and safety of camera system 20 has been discovered. This warning may also guide the user to follow an authorized service technician. This warning may remain visible until the system has been properly recalibrated and the hash value matches during the calibration procedure (step 510).

[0052] According to the method described above, changes or updates to calibration data follow the same calibration data corresponding logic. In step 510, authorized service personnel will be able to determine whether the verification problem is a camera module malfunction or whether the camera module and / or calibration data have been replaced or updated. Assisted calibration of the camera system 20 can then be performed to correct the problem and create a new pairing between the camera module and ECU 26.

[0053] In one embodiment, after the camera pod assembly is replaced, the new calibration data is stored on the newly delivered camera pod assembly 10. Therefore, the new calibration data is transmitted to the ECU 26 to provide the same calibration data on both the camera pod assembly 10 and the ECU 26. The transmission can be performed via: an external storage device inserted into an interface integrated into the storage device of the ECU 26; wireless transmission via standards such as Wi-Fi, Bluetooth, NFC, or similar, wherein the corresponding transmission device is attached to the ECU 26 or any other component of the vehicle (with the ECU 26 connected to that component); via connecting the ECU to a mobile device, such as a smartphone, and transmitting the calibration data; via mobile internet access connecting the ECU to the vehicle to obtain the new calibration data; via a data transmission device or interface connected to the ECU 26; or via wirelessly transmitting data from the storage device of the camera pod assembly 10 (e.g., RFID, NFC) to the storage device integrated into the ECU 26. In another embodiment, the calibration data is stored in the cloud, and each time the vehicle is started, the ECU 26 and the camera pod assembly 10 connect to the cloud and download the appropriate calibration data.

[0054] In another embodiment, a calibration update may be required without replacing physical components. Information regarding the need for updated calibration data may be provided by an authorized service provider or technician, or by transmitting this information to the user, the vehicle, or a combination thereof. The calibration data update must be performed on both the ECU 26 and the storage device of the camera pod assembly 10. The transmission of calibration data can be performed via: an external storage device that interfaces with the storage device integrated into the ECU 26 and camera pod assembly 10; wireless transmission via standards such as Wi-Fi, Bluetooth, NFC, or similar, wherein the corresponding transmission device is attached to the ECU 26 or camera pod assembly 10 or any other component of the vehicle (where the storage device of the ECU 26 and camera pod assembly 10 is connected to such a component); transmission of calibration data via connecting the storage device of the ECU or camera pod assembly 10 to a mobile device, such as a smartphone; access to new calibration data via mobile internet access to the vehicle via connecting the storage device of the ECU or camera pod assembly 10; via a data transmission device or interface connected to the storage device of the ECU 26 or camera pod assembly 10; or via wireless transmission of data from the storage device (e.g., RFID, NFC) to the storage device integrated into the ECU 26 or camera pod assembly 10. In another embodiment, updated calibration data is stored in the cloud, and upon starting the vehicle, the ECU 26 and camera pod assembly 10 connect to the cloud and download the appropriate updated calibration data. Any other combination of the methods mentioned is also possible.

[0055] In step 510, the vehicle may be taken to an authorized service provider capable of performing the verification calibration process 301 or repairing and updating the calibration. The authorized calibration repair version may differ from the component verification calibration process 301. The authorized calibration repair version will include steps for pairing the camera modules in the camera pod assembly 10 for image quality and for creating and setting hash value safety systems in at least two camera modules and ECU 26.

[0056] exist Figure 6 In the exemplary case shown, when the replacement camera module 512 replaces the first camera module, the H12 value and calibration data hash value (H4) stored in the ECU 26 and the second camera module 14 will not match the hash value from the replacement camera module 512. Upon detection of a missing or incorrect hash value, this will flag the system during monitoring to display a warning that the camera system 20 has been altered, and recalibration will ensure that the displayed image meets safety rules. The warning will be displayed until authorized recalibration is performed. During authorized recalibration, a new hash value H512 and calibration data hash value (H4) will be created from the replacement camera module 512 (the replacement for the first camera module 12). These new hash values ​​H512 and calibration data hash values ​​(H4) will then be stored in the ECU 26 and the second camera module 14. In this form, the safety portion of the recalibration of the replacement camera module 512 will be as follows: Replace camera module 512 storage The serial number comes from the replacement camera module 512. H14, and At least one calibration data information hash value (H412) and / or Second camera module 14 storage Serial number from second camera module 14 H512, and At least one calibration data information hash value (H412) and / or ECU 26 stores value h3, which can be defined as follows: h3 = hash(hash(serial number replacement camera module 512, at least one calibration data information H412) || hash(serial number second camera module 14, at least one calibration data information H412)) or h3 = (H512, hash value (at least one calibration data information)) || (H14, hash value (at least one calibration data information)).

[0057] In another form, if more than one camera module is replaced, the process creates and stores a new hash value for each replaced camera module and its corresponding calibration data. The recalibration process will also include a section that aligns the camera images to a compliant image.

[0058] exist Figure 7 In the exemplary case shown, the camera pod assembly 10 is externally calibrated and assembled in the vehicle 18 without transmitting calibration data from the camera pod assembly 10 to the ECU 26 via a coaxial cable. The external calibration between the vehicle and the camera pod assembly 10 begins in step 701 by storing the calibration data, along with the relevant serial numbers of the components of the camera pod assembly, in a database. Then, in step 702, pairing is performed between the display and the camera pod assembly 10. Based on this information, an associated database snapshot is created for the corresponding camera pod assembly at the end of the production line (EOL) (step 703). From this point onward, the camera pod assembly and the database snapshot are linked via, for example, a calibration data number to ensure that the pairing between the camera pod assembly and the corresponding database snapshot will not be separated. In step 704, the hardware, including the camera pod assembly 10 and the display unit (not shown), along with the calibration data obtained via the database snapshot, is delivered independently to the OEM. In step 705, it is ensured that the OEM receives a recognizable hardware and data pair due to the calibration data number, ensuring data and hardware pairing even during assembly in the vehicle (step 706). In step 707, the OEM retrieves the calibration data corresponding to the camera pod assembly 10 and provides the corresponding calibration data to the ECU 26. In this form, the calibration data is permanently stored in both the camera pod assembly 10 and the ECU 26. To ensure proper system operation, only information about the camera pod assembly 10 and the calibration data need to be exchanged; the calibration data itself does not need to be transmitted.

[0059] Figure 8An exemplary embodiment of a method for recalibrating ECU 26 is illustrated. After successful pairing of the camera module (e.g., consistent with step 306), a system check (step 801) is performed to ensure that the components of the camera pod assembly 10, the display unit, and the ECU 26 are mated together and properly connected. At the end of the system check in step 801, the presence or absence of calibration data in ECU 26 is verified in step 8010. If calibration data is present in ECU 26, a security check step 802 is then performed to identify data security vulnerabilities. Subsequently, in step 803, the serial number of the camera module and information about the calibration data (e.g., identification number) are checked and compared with that in ECU 26 in step 8030. If the information matches, the camera module can be used (step 804). If the information does not match, in some embodiments, a warning is displayed (such as a warning box on the display unit with the text "Please update your vehicle's calibration data"), and in step 805, ECU 26 must be updated, where the corresponding calibration data is the appropriate information for the camera module. The warning may remain visible until the calibration data is updated. To update ECU 26, in step 8071, calibration data can be transmitted to ECU 26 via various wireless connections (e.g., WLAN, 5G, or NFC standards) or via a wired connection that can be directly or indirectly connected to ECU 26. In some embodiments of this disclosure, ECU 26 may have one or more interfaces to receive calibration data. After the data has been transmitted to ECU 26, the method returns to step 801, and then verifies that appropriate calibration data is now present on both camera pod assembly 10 and ECU 26.

[0060] According to Figure 8 In another alternative to the method, no data is found on ECU 26 during system check step 801. In this case, the method proceeds to step 806, where ECU 26 attempts to obtain informational data (e.g., serial number) and calibration data (e.g., calibration identification number) regarding the camera pod assembly 10. If the camera does have informational data in step 8060, the method proceeds to step 803 as described above and updates ECU 26. If the camera does not have informational data in step 8060, then in step 807, the camera needs to obtain appropriate data, which can be transmitted via various options from step 8070, such as via an additional data transmission cable, a Linbus connection, or any external storage device connected to ECU 26 via a wireless standard or wired connection as described in step 8071. Afterward, the method continues with step 801 and follows the steps described above.

[0061] For purposes of illustration and description, the foregoing description of various preferred embodiments has been presented. This is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed, and it will be apparent that many modifications and variations are possible in accordance with the teachings above. As stated above, exemplary embodiments were chosen and described in order to best explain the principles of the present disclosure and its practical application, thereby enabling others skilled in the art to best utilize the disclosure in various embodiments and with various modifications suitable for a particular purpose. The features disclosed in the foregoing description, drawings, and claims are important individually and in any combination for the implementation of the present disclosure in its various embodiments.

[0062] List of reference symbols

[0063] 10-Camera pod assembly

[0064] 12-First Camera Module

[0065] 14-Second Camera Module

[0066] 16-Mounting Components

[0067] 18-Vehicles

[0068] 20-Camera System

[0069] 22-First Camera Flash Memory

[0070] 24-Second Camera Flash Memory

[0071] 26-Electronic Control Unit (ECU)

[0072] 28-System calibration storage device

[0073] 30-First Lens System

[0074] 31-Second Lens System

[0075] 100-Vehicle Camera System Calibration Method

[0076] 200-Camera Calibration Process

[0077] 300-Camera System Calibration Process

[0078] 301 - Verification and Calibration Process

[0079] 302 - Serial numbers stored and assigned to each camera module

[0080] 304 - Camera module installed in camera system

[0081] 306-Cross-Validation Camera Pairing

[0082] 308-ECU checks stored pairing values

[0083] 310-value matching?

[0084] 312 - Camera module has been properly calibrated

[0085] 400 - Vehicle CMS Calibration Process

[0086] 500 - Component Verification Monitoring Process

[0087] 502 - ECU checks stored pairing values

[0088] 504 - Value Match?

[0089] 506 - Camera module has been properly calibrated

[0090] 508 - Camera calibration warning displayed

[0091] 510 - Authorized Recalibration

[0092] 512 - Replacement Camera Module

[0093] 700 - External Calibration Process

[0094] 701 - Store calibration data

[0095] 702 - Pairing of Camera Pod Assembly and Display

[0096] 703 - Creating a Calibration Database Snapshot

[0097] 704 - Independent Delivery

[0098] 705 - Paired Delivery

[0099] 706 - Interior Assembly

[0100] 707 - Retrieves and provides calibration data

[0101] 800 - Recalibration process without coaxial cable

[0102] 801 System Check

[0103] 802 - Security Check

[0104] 803 - Inspection Information Data

[0105] 804 - A camera can be used.

[0106] 805 - ECU must be updated

[0107] 806 - Get Camera Information

[0108] 807-Camera needs data

[0109] 8010-Decision: Is there data in the ECU?

[0110] 8030 - Decision: Is the serial number or calibration information new?

[0111] 8060-Decision: Does the camera have data?

[0112] 8070 - Data Transfer Options

[0113] 8071 - Data Transfer Options via External Storage Devices

[0114] H12 - First Camera Module Hash Value

[0115] H14 - Second Camera Module Hash Value

[0116] H2-Camera Pod Component Hash Value

[0117] Hash value stored in H3-ECU

[0118] H4 - Calibration data hash value

[0119] H412 - New calibration data hash for replacing the camera module

[0120] H512 - Replace Camera Module Hash Value

Claims

1. A method for setting up a security system (20) for a vehicle (18), comprising the following steps: Create a camera system (20) having at least a first camera module (12) with a first identifier and a second camera module (14) with a second identifier and an electronic control unit (ECU) (26). Create at least one first camera module hash value (H12) based on the first identifier, at least one second camera module hash value (H14) based on the second identifier, and / or at least one camera pod assembly (10) hash value (H2) based on the first and second identifiers. as well as Storing the hash value includes: The hash value (H12) of at least one first camera module is stored in the second camera module (14), and the hash value (H14) of at least one second camera module is stored in the first camera module (12), or The hash value (H2) of the at least one camera pod assembly is stored in the camera pod assembly (10), or Store the hash values ​​(H12) of at least one first camera module and the hash values ​​(H14) of at least one second camera module in pairs, or The hash values ​​of at least one first camera module (H12) and at least one second camera module (H14) are stored in the ECU (26) as the ECU hash value (H3). At least one of the camera modules (12, 14) or camera pod assembly (10) is replaced by the following steps: Replace at least one camera module (12, 14) or camera pod assembly (10). Create a new hash value (H512) to replace the camera module (512) or create a new hash value to replace the camera pod assembly. Create a new calibration data hash value (H412) for the replacement camera module (512) or the replacement camera pod assembly, and The old hash values ​​and old calibration data hash values ​​previously stored in the ECU (26) and / or the camera modules (12, 14) and / or the camera pod assembly (10) are replaced by the new hash values ​​and new calibration data hash values ​​(H512, H412) stored in the replacement camera module (512) or the replacement camera pod assembly (10).

2. The security setting method according to claim 1 further includes the following steps: Encrypt the at least one hash value before storing and / or sending at least one of the hash values ​​(H12, H14, H3, H2); and / or Monitor and / or verify the hash values ​​of at least one first camera module stored in the second camera module (14) and the hash values ​​of at least one second camera module stored in the first camera module (12) by matching them with the ECU hash value (H3) stored in the ECU (26); and / or Calibration data for the calibration of the camera monitoring system (CMS) is transmitted to the camera pod assembly (10) and / or the camera module (12, 14) and / or the ECU (26), wherein the transmission is via a wireless or wired connection.

3. The security setting method according to claim 2, wherein... The monitoring step occurs when the vehicle is started; and / or The calibration data in the transmission step is defined by the calibration datasets assigned to each camera module (12, 14) and / or the camera pod assembly (10), wherein each calibration dataset is assigned a specific calibration data identifier; and / or The transmission step uses a data transmission interface that enables data transmission standards in the form of NFC, WLAN, 5G, USB, LIN bus, storage device reader, Bluetooth, or LAN.

4. The security setting method according to claim 1 further includes the following steps: If the ECU hash value (H3) does not match the hash values ​​(H12, H14) of at least one first and second camera modules or the hash value (H2) of at least one camera pod assembly (10), a warning is displayed; and / or At least one calibration data hash value (H4) is created for at least one calibration data identifier and the at least one calibration data hash value (H4) is stored in the ECU (26) and / or camera module (12, 14) and / or camera pod assembly (10).

5. The security setting method according to claim 4, wherein: A warning is displayed if at least one calibration data hash value (H4) stored in at least one of the camera modules (12, 14) and / or the camera pod assembly (10) does not match at least one calibration data hash value (H4) stored in the ECU (26).

6. The security setting method according to any one of claims 1-4 further includes the following step: Store at least one calibration dataset in at least one storage device; and / or The camera pod assembly (10) and at least one calibration dataset are provided simultaneously or independently, wherein at least one calibration dataset and the camera pod assembly (10) are delivered independently to the OEM.

7. The security setting method according to claim 6, wherein: The storage device can be selected to include cloud, flash drive, hard disk, CD, DVD, printable code, chip and / or mobile device.

8. The security setting method according to claim 1, wherein: At least one of the hash values ​​(H12, H14, H3, H4, H2) includes an identifier and / or a hash value and / or a portion of the identifier and / or a portion of the hash value; and / or The first and second hash values ​​(H12, H14) additionally include information about the calibration dataset; and / or The first and second hash values ​​(H12, H14) include at least two hash values, wherein at least one first hash value determines the hardware module of the camera pod assembly (10) and at least one second hash value determines the calibration dataset.

9. The security setting method according to claim 2, wherein the matching of the ECU hash value (H3) stored in the ECU (26) performed during the verification and / or monitoring steps is also based on at least one calibration data hash value (H4), specifically at least partially stored in the ECU (26) and / or in at least one of the camera modules (12, 14) and / or in the camera pod assembly (10).

10. The security setting method according to any one of claims 2 and 3, wherein: The transmission of the calibration data is in the form of wireless transmission from the camera pod assembly (10) to the ECU (26); and / or The safety setting method further includes checking for calibration data updates, wherein if calibration data updates are available, a warning is displayed, and / or the new calibration dataset is transmitted as an update to at least one of the camera pod assembly (10), the camera modules (12, 14), and / or the ECU (26).

11. The security setting method according to any one of claims 1-4, wherein: The camera system (20) includes more than two camera modules and / or calibration datasets, and creates, stores, monitors and verifies corresponding hash values ​​for the more than two camera modules and / or calibration datasets; and / or At least two hash values ​​are specifically combined into a single hash value or a defined list of hash values; and / or At least one of the hash values ​​includes information about the stored calibration dataset; and / or At least one identifier is a serial number; and / or At least one calibration data hash value (H4) is stored in each camera module, such that the calibration data hash values ​​are interleaved in the camera modules so that each camera module stores a calibration data hash value (H4) related to another camera module.

12. The security setting method according to any one of claims 1-4, wherein the method provides a system calibration method for the camera system (20), comprising: Steps for individual camera module calibration (200), steps for individual camera module calibration (12, 14). Step (300) for camera system calibration, used to calibrate the camera pod assembly (10) to combine individual images from camera modules (12, 14) to produce a unified camera system image, and Step (400) for vehicle camera calibration, used to calibrate parameters when the camera pod assembly (10) has been assembled into the vehicle.

13. The security setting method according to claim 12, wherein... The steps (300) for camera system calibration include: Step (302) is used to store the assigned serial numbers of each camera module. Step (304) for installing the camera modules (12, 14) in the camera system (20). Step (306) for cross-storage and / or cross-validation of camera module pairing. Step (308) is used to check the existence of stored paired values. Step (310) for matching the hash value, and Step (312) is used to assign the camera module as calibrated.

14. The security setting method according to any one of claims 1-4, wherein the method is applicable to a security parameter monitoring camera surveillance system component, comprising: Step (502) is used to check the existence of stored paired hash values. Step (504) for matching the hash value, and Step (506) for assigning the camera modules (12, 14) as calibrated, or step (508) for displaying a camera calibration warning, and / or step (510) for authorizing recalibration, wherein the subsequent return to step (502) for checking the existence of stored paired hash values.

15. The safety setting method according to any one of claims 1-4, wherein the method is adapted to calibrating the camera pod assembly (10) and the ECU (26) using a database, comprising: Step (701) for storing the calibration data, Step (702) for pairing the camera modules (12, 14) of the camera pod assembly (10). Step (703) for creating a calibration database snapshot. Specifically, the step (704) involves independently delivering the paired camera pod assembly (10), the display unit, and the calibration database snapshot. Step (705) for sending information about the pairing of the camera pod assembly (10) and the display unit with the database snapshot, and Step (707) for retrieving calibration data from the database snapshot and providing it to the ECU.

16. The safety setting method according to claim 2 or 3, wherein recalibration without transmitting the calibration data from the camera pod assembly (10) via a coaxial cable comprises: Step (801) for checking the camera system (20) and step (8010) for checking the presence of data on the ECU (26). The steps (802) for checking the security of the camera system (20) when data exists on the ECU (26) or for obtaining information data from the camera modules (12, 14) when no data exists on the ECU (26) but data exists on the camera modules (12, 14) and for providing calibration data to the camera modules (12, 14) specifically via a wired or wireless connection (807, 8070, 8071), wherein the process then returns to the steps (801) for checking the camera system (20) and for checking the presence of data on the ECU (26) (8010). Steps (803, 8030) for checking the serial numbers of the camera modules (12, 14) and / or the camera pod assembly (10), and Step (805) for updating the calibration data of the ECU by providing the calibration data to the ECU in the presence of a new serial number for the camera module (12, 14) and / or the camera pod assembly (10), wherein subsequent steps return to step (801) for checking the camera system (20) and step (8010) for checking the presence of data on the ECU (26), or Step (804) for using the camera module (12, 14) without updating in the absence of a new serial number for the camera module (12, 14) and / or the camera pod assembly (10).

17. A camera pod assembly (10) comprising at least two camera modules (12, 14) to be attached to a vehicle (18) via mounting elements (16) and adapted for use in a safety setting method according to any one of the preceding claims 1-16.

18. The camera pod assembly (10) of claim 17 further includes a storage device, wherein the storage device has a flash drive, a hard disk, a CD, a DVD, printable code, and / or a chip.

19. A camera system (20) comprising at least one camera pod assembly (10) according to claim 17 or 18 and the ECU (26).

20. The camera system (20) of claim 19, wherein the storage device is provided by the cloud and / or a mobile device.

21. A vehicle (18) comprising a camera system according to claim 19 or 20 and at least one display unit.

22. The vehicle (18) according to claim 21, wherein the ECU (26) is provided by the display unit.

Citation Information

Patent Citations

  • Device for generating identifier for equipment state, vehicle, network component, and method

    CN111198867A