Electronic exterior rearview mirror system and automatic calibration method

Abstract

This invention relates to the field of electronic exterior rearview mirrors, and provides an electronic exterior rearview mirror system and an automatic calibration and production line method. The system includes: a camera, a display screen, and a controller; the camera is used to acquire image information outside the vehicle; the display screen is used to display the image information; the image information includes: first image information and second image information; the first image information is external information acquired by the camera, and the second image information is vehicle body information acquired by the camera; the controller is used to confirm the matching degree between the first image information and the first information, and to confirm the matching degree between the second image information and the second information; the first information is feature pattern information acquired under a reference camera; the second information is vehicle body feature information of a reference vehicle acquired by the reference camera and the reference camera in a standard position.

Description

Technical Field

[0001] This invention relates to the field of electronic exterior rearview mirrors, and more particularly to an electronic exterior rearview mirror system and an automatic calibration method. Background Technology

[0002] An electronic rearview mirror (Camera Monitor System, abbreviated as CMS) is a device that uses a camera to acquire external image information, processes the image information, and displays the image information on a display screen.

[0003] Offline calibration refers to the calibration that needs to be performed when a vehicle is off the production line to ensure that the image information displayed on the screen is as close to the actual situation as possible, due to manufacturing errors of the camera image sensor, lens distortion, assembly errors of the camera itself, and errors caused during image processing.

[0004] However, due to the imminent mass production of electronic exterior rearview mirrors in China, each mirror needs to be calibrated individually after production due to camera distortion, assembly errors, and other issues. The current solution is manual calibration by individuals, with no automated calibration solution available, which severely impacts vehicle production schedules and results in low production efficiency. Summary of the Invention

[0005] This invention provides an electronic exterior rearview mirror system and an automatic calibration and off-line method to solve the problem of slow vehicle off-line production caused by manual calibration by individuals, and to improve the generation efficiency of off-line calibration.

[0006] According to a first aspect of the present invention, an electronic exterior rearview mirror system is provided, comprising: a camera, a display screen, and a controller;

[0007] The camera is used to acquire image information outside the vehicle;

[0008] The display screen is used to display the image information;

[0009] The image information includes: first image information and second image information; the first image information is external information acquired by the camera, and the second image information is vehicle body information acquired by the camera;

[0010] The controller is used to confirm the matching degree between the first image information and the first information, and to confirm the matching degree between the second image information and the second information.

[0011] The first information is the feature pattern information obtained under the reference camera; the second information is the body feature information of the reference vehicle obtained by the reference camera and the reference camera in a standard position.

[0012] Optionally, the controller has a first model and a second model preset within it;

[0013] The first model is used to determine the degree of matching between the first image information and the first information;

[0014] The second model is used to confirm the degree of matching between the second image information and the second information.

[0015] Optionally, the controller includes a first model generation unit, the first model generation unit being configured to:

[0016] Select a feature pattern; the feature pattern can trigger the electronic exterior rearview mirror system to enter the calibration mode; the reference camera acquires the features of the feature pattern and obtains first information that conforms to the feature pattern through an image processing algorithm; and extracts the first information to form the first model.

[0017] Optionally, the feature pattern is: a QR code, barcode, circular QR code, or other coded pattern that can trigger the electronic exterior rearview mirror system to enter the calibration mode.

[0018] Optionally, the controller includes a second model generation unit, which is configured to:

[0019] A reference vehicle and a reference camera are selected; the reference camera is placed in a standard position and the body feature information of the reference vehicle is acquired, and second information conforming to the external features is obtained through an image processing algorithm; and the second information is extracted to form the second model.

[0020] Optionally, the second information includes: vehicle first information and vehicle second information;

[0021] The first information about the vehicle is the external features of the reference vehicle obtained when the reference camera is placed in a standard position;

[0022] The second vehicle information is the shape features of the reference vehicle obtained when the reference camera is placed in a standard position.

[0023] According to a second aspect of the present invention, an automobile is provided that includes the electronic exterior rearview mirror system described in the first aspect of the present invention.

[0024] According to a third aspect of the present invention, an automatic calibration method is provided, which involves calibrating the vehicle described in the second aspect of the present invention, comprising:

[0025] S1: Place the car on the production line and start the electronic exterior rearview mirror system;

[0026] S2: The feature pattern is placed within the visible area of ​​the camera. The camera acquires the feature pattern and forms first image information. The first image information is input into the first model of the controller. The first model determines whether the matching degree between the first image information and the first information reaches a first preset value. If yes, the first image information is displayed on the display screen and the calibration mode is enabled, and the process proceeds to S3. If no, the display screen displays an error message to remind that manual intervention is required.

[0027] S3: The camera acquires external features of the vehicle and forms second image information, and inputs the second image information into the second model of the controller. The second model determines whether the matching degree between the second image information and the first information of the vehicle reaches a second preset value. If yes, proceed to S4; if no, the display screen displays an error message to remind that manual intervention is required.

[0028] S4: After adjusting the camera position, acquire the external features of the vehicle and form the second image information. Transmit the second image information to the second model of the controller. The second model determines whether the second image information and the second information of the vehicle reach a third preset value. If yes, it means that the calibration is completed and exit the calibration mode. If no, the display screen displays an error message to remind that manual intervention is required.

[0029] According to a fourth aspect of the present invention, an electronic device is provided, including a processor and a memory; the memory stores a program that can be called by the processor; wherein, when the processor executes the program, it implements the automatic calibration offline method as described in the third aspect of the present invention.

[0030] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, wherein program instructions are stored therein, and when executed by a processor of a computer, the processor performs the automatic calibration offline method described in the third aspect of the present invention.

[0031] The electronic exterior rearview mirror system and automatic calibration off-line method provided by this invention obtains in advance first information representing the feature pattern acquired by a reference camera and second information representing the body feature information of a reference vehicle acquired by the reference camera and the reference camera in a standard position. The first and second information are stored in a controller. The controller confirms the matching degree between the first image information acquired by the camera and the first information, and confirms the matching degree between the second image information acquired by the camera and the second information. By comparing the pre-stored information, the problem of the image displayed on the screen not conforming to the actual situation due to camera distortion or camera position errors is solved, realizing the problem of automatic calibration of the vehicle off-line and improving the production efficiency of off-line calibration. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a block diagram of an electronic exterior rearview mirror system according to an embodiment of the present invention;

[0034] Figure 2 This is a block diagram of the controller structure in one embodiment of the present invention;

[0035] Figure 3 This is a flowchart of generating the first model in one embodiment of the present invention;

[0036] Figure 4 This is a flowchart of generating the second model in one embodiment of the present invention;

[0037] Figure 5 This is a flowchart of a method for automatically calibrating the offline status in one embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0042] Please refer to Figure 1 and Figure 2 This invention provides an electronic exterior rearview mirror system, including: a camera 100, a display screen 200, and a controller 300;

[0043] The camera 100 is used to acquire image information outside the vehicle; specifically, the image information includes: first image information and second image information; the first image information is the external information acquired by the camera 100, and the second image information is the vehicle body information acquired by the camera 100. The acquired image information is displayed through the display screen 200.

[0044] The controller 300 is used to confirm the matching degree between the first image information and the first information, and to confirm the matching degree between the second image information and the second information.

[0045] In one example, the first information is information about a feature pattern acquired by a reference camera; the second information is body feature information of a reference vehicle acquired by the reference camera and the reference camera in a standard position. The "reference" in "reference camera" and "reference vehicle" refers to the fact that the dimensions, parameters, etc., of each part meet the design requirements.

[0046] The second information includes: vehicle first information and vehicle second information.

[0047] The first vehicle information is the external features of the reference vehicle obtained when the reference camera is placed in a standard position, such as the number and corresponding positions of tires, the number and corresponding positions of door handles, etc. in the image obtained by the reference camera in the standard position; the second vehicle information is the shape features of the reference vehicle obtained when the reference camera is placed in a standard position, specifically manifested as the specific shape of the door handles, the specific shape of the car door, etc., obtained by the reference camera in the standard position.

[0048] Specifically, the controller 300 is pre-set with a first model and a second model; the first model is used to confirm the matching degree between the first image information and the first information; if the matching degree between the first image information and the first information reaches a certain value, the calibration mode of the electronic exterior rearview mirror system is activated; otherwise, an error message is displayed on the screen to remind manual intervention.

[0049] The second model is used to confirm the matching degree between the second image information and the second information. If the matching degree between the second image information and the second information reaches a certain value, it indicates that the vehicle equipped with the electronic exterior rearview mirror system has completed its off-line calibration; otherwise, an error message is displayed on the screen to remind manual intervention.

[0050] For details, please refer to Figure 3 The first model is generated by the first model generation unit 301 in the controller 300, and the specific steps are as follows:

[0051] S1: Select a feature pattern; the feature pattern can trigger the electronic exterior rearview mirror system to enter the calibration mode.

[0052] The first image information acquired by the camera 100 is a feature pattern, such as a QR code, barcode, circular QR code, or other coded pattern that can trigger the electronic rearview mirror system to enter the calibration mode after being recognized by the controller.

[0053] Of course, this invention is not limited to entering the calibration mode by recognizing feature patterns. Other methods, such as manual operation of the switch by the operator, to trigger the entry into the calibration mode are also within the scope of protection of this invention.

[0054] S2: The reference camera acquires the features of the feature pattern and obtains first information that conforms to the feature pattern through an image processing algorithm.

[0055] S3 extracts the first information to form the first model.

[0056] Please refer to Figure 4 The second model is generated by the second model generation unit 302 in the controller 300, and the specific steps are as follows:

[0057] S1: Select the reference vehicle and reference camera.

[0058] S2: Place the reference camera in a standard position and acquire the body feature information of the reference vehicle, and obtain second information that conforms to the external features through an image processing algorithm.

[0059] S3: Extract the second information to form the second model.

[0060] The image processing algorithms mentioned above specifically refer to: performing deep learning on images, extracting image features, and finally extracting a set of feature points for that type of image.

[0061] The electronic exterior rearview mirror system provided by this invention pre-sets a first model and a second model in the controller. The first model includes pre-trained first information containing feature patterns that enable the electronic exterior rearview mirror system to enter calibration mode. When the matching degree between the first image information acquired by the camera and the first information reaches a certain value, the electronic exterior rearview system is activated and enters calibration mode. The second model includes second information representing the body feature information of the reference vehicle acquired by the reference camera and the reference vehicle in a standard position. When the matching degree between the second image information acquired by the camera and the second information reaches a certain value, calibration is completed. In other words, the electronic exterior rearview mirror system provided by this invention enables vehicles using the electronic exterior rearview mirror system to automatically activate the offline calibration mode and complete automatic calibration.

[0062] This invention also provides a car, including the electronic exterior rearview mirror system described above. The car provided in this invention can be applied to a production line for automatic calibration of automobiles.

[0063] Please refer to Figure 5 This invention also provides an automatic calibration method for vehicle production line completion, which involves calibrating the aforementioned vehicles, including:

[0064] S1: Place the car on the production line and start the electronic exterior rearview mirror system.

[0065] S2: The feature pattern is placed within the visible area of ​​the camera. The camera acquires the feature pattern and forms first image information. The first image information is input into the first model of the controller. The first model determines whether the matching degree between the first image information and the first information reaches a first preset value. If yes, the first image information is displayed on the display screen and the calibration mode is enabled, and the process proceeds to S3. If no, the display screen displays an error message to remind that manual intervention is required.

[0066] In a specific example, if the first information stored in the first model that can be recognized and triggers entry into the calibration mode is a QR code, after being processed by the image processing algorithm, there are n feature points corresponding to the QR code in the first information; at this time, a first preset value is set in the controller. Here, the first preset value represents the ratio of the minimum number of feature points that are recognized after the first image information is recognized and triggers entry into the calibration mode to the number of n feature points corresponding to the QR code in the first information.

[0067] For example, a QR code containing n feature points is placed within the visible area of ​​a camera on the production line. After the camera acquires the QR code image, it becomes the first image information. The first image information is processed by an image processing algorithm to obtain m feature points corresponding to the first image information. At this time, the first preset value is 0.9. Therefore, if the value of m / n is greater than or equal to 0.9, the QR code can be recognized and the calibration mode can be triggered. If the value of m / n is less than 0.9, there may be dirt on the camera that affects the recognition of the first image information, or there may be some other major malfunctions that prevent the electronic rearview mirror system from entering the automatic calibration mode. In this case, the display screen will show an error message to remind manual intervention.

[0068] S3: The camera acquires external features of the vehicle and forms second image information, and inputs the second image information into the second model of the controller. The second model determines whether the matching degree between the second image information and the first information of the vehicle reaches a second preset value. If yes, proceed to S4; if no, the display screen displays an error message to remind that manual intervention is required.

[0069] S4: After adjusting the camera position, acquire the external features of the vehicle and form the second image information. Transmit the second image information to the second model of the controller. The second model determines whether the second image information and the second information of the vehicle reach a third preset value. If yes, it means that the calibration is completed and exit the calibration mode. If no, the display screen displays an error message to remind that manual intervention is required.

[0070] This invention is not limited to error notification methods; other methods (such as adding in-vehicle audio prompts) are also within its scope. In a specific embodiment, if the second model stores second information of the reference vehicle under a standard position, processed by the controller, the second information is the feature contour information of the car during automatic calibration. Specifically, the second information includes: first vehicle information representing the external features of the reference vehicle acquired by the reference camera under a standard position, such as the number of tires and door handles in the image acquired by the reference camera under a standard position; and second vehicle information representing the shape features of the reference vehicle acquired by the reference camera under a standard position, such as the specific shape of the door handles and the specific shape of the car door acquired by the reference camera under a standard position.

[0071] Specifically, the first vehicle information is as follows: If the reference camera acquires information about the door handles in the reference vehicle from a standard position, such as if the reference camera acquires information about two door handles in the reference vehicle from a standard position, then after image processing algorithm, there will be two sets of feature points, and each set will have N feature points; if the reference camera acquires information about one door handle in the reference vehicle from a standard position, then after image processing algorithm, there will be one set of feature points, and this set will have N feature points. At this time, a second preset value and a third preset value are set in the controller. The second preset value corresponds to the number of feature point sets about the door handles in the reference vehicle acquired by the reference camera from a standard position, and the third preset value corresponds to the number of feature points in one set about the door handles in the reference vehicle acquired by the reference camera from a standard position.

[0072] Correspondingly, the car on the production line acquires second image information. After the second image information is recognized by the image processing algorithm, the feature points corresponding to the door handle in the second image information are obtained. If the second image information obtains a set of feature points after the image processing algorithm, and the reference camera obtains only one set of feature points related to the door handle in the reference vehicle under the standard position, it means that the matching degree between the second image information and the first information of the vehicle reaches the second preset value, and proceeds to the next step; if the second image information obtains a set of feature points after the image processing algorithm, and the reference camera obtains only two set of feature points related to the door handle in the reference vehicle under the standard position, it means that the matching degree between the second image information and the first information of the vehicle does not reach the second preset value, and the display screen will display an error message to remind manual intervention.

[0073] Among them, the matching degree between the second image information and the first vehicle information reaches a second preset value, and entering the next step means that: if the number of feature points in a set of door handle information obtained after the second image information passes through an image processing algorithm is N, and the number of feature points in a set of door handle information in a reference vehicle obtained by a reference camera at a standard position is M, and the third preset value set at this time is Q, if N / M > Q, it represents that the matching degree between the second image information and the second vehicle information reaches the third preset value, indicating that the calibration is completed and the calibration mode is closed; if N / M < Q, it represents that the matching degree between the second image information and the second vehicle information does not reach the third preset value, and the display screen will display an error message to remind manual intervention for viewing.

[0074] Among them, the second information can also be information such as the tires and windows of the reference vehicle obtained by the reference camera at the standard position. Then, the corresponding first vehicle information is the corresponding quantity information; the corresponding second vehicle information is the corresponding specific appearance information.

[0075] The automatic calibration off-line method provided by the present invention obtains in advance the first information representing the information of the feature pattern obtained under the reference camera and the second information representing the body feature information of the reference vehicle obtained by the reference camera and the reference camera at the standard position, and stores the obtained first information and second information in the controller. The controller confirms the matching degree between the first image information obtained by the camera and the first information and the matching degree between the second image information obtained by the camera and the second information according to the first information and the second information; by comparing the pre-stored information, it solves the problem that the image displayed on the display screen does not conform to the actual situation due to camera distortion or error in the camera position, realizes the problem of automatic calibration of the produced vehicle off-line, and improves the generation efficiency of off-line calibration.

[0076] Please refer to Figure 6 , which provides an electronic device 1, including:

[0077] A processor 11; and

[0078] A memory 12 for storing the executable instructions of the processor;

[0079] Among them, the processor 11 is configured to execute the above-mentioned methods by executing the executable instructions.

[0080] The processor 11 can communicate with the memory 12 through a bus 13.

[0081] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above-mentioned methods.

[0082] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An electronic exterior mirror system, characterized by The electronic outside rearview mirror system comprises a camera, a display screen and a controller. The camera is configured to acquire image information outside the vehicle. The display screen is configured to display the image information. The image information comprises first image information and second image information; the first image information is external information acquired by the camera, and the second image information is vehicle body information acquired by the camera. The controller is preconfigured with a first model and a second model; the first model is configured to determine whether the matching degree of the first image information and first information reaches a first preset value; if yes, the calibration mode of the electronic outside rearview mirror system is started; if not, the display screen displays error information to remind manual intervention; the second model is configured to determine the matching degree of the second image information and second information. The first information is information of a characteristic pattern acquired by a reference camera; the characteristic pattern is an encoding pattern capable of triggering the electronic outside rearview mirror system to enter the calibration mode; the second information is vehicle body characteristic information of a reference vehicle acquired by the reference camera and the reference camera in a standard position. The second information comprises vehicle first information and vehicle second information. The vehicle first information is an external characteristic of the reference vehicle acquired by the reference camera in the standard position; the external characteristic comprises the number and corresponding position of tires or the number and corresponding position of door handles in an image acquired by the reference camera in the standard position. The vehicle second information is a shape characteristic of the reference vehicle acquired by the reference camera in the standard position; the shape characteristic comprises the specific shape of a door handle or the specific shape of a door. The controller comprises a first model generation unit configured to:

2. The electronic exterior mirror system according to claim 1, characterized in that select a characteristic pattern; the characteristic pattern is capable of triggering the electronic outside rearview mirror system to enter the calibration mode; the reference camera acquires characteristics of the characteristic pattern and obtains first information conforming to the characteristic pattern through an image processing algorithm; and extract the first information to form the first model. The characteristic pattern is a two-dimensional code, a bar code or a circular two-dimensional code.

3. The electronic exterior mirror system according to claim 2, characterized in that The controller comprises a second model generation unit configured to:

4. The electronic exterior mirror system according to claim 1, characterized in that select a reference vehicle and a reference camera; place the reference camera in a standard position and acquire vehicle body characteristic information of the reference vehicle, and obtain second information conforming to the external characteristic through an image processing algorithm; and extract the second information to form the second model. The electronic outside rearview mirror system of any one of claims 1-4.

5. An automobile characterized by comprising: The calibration of the vehicle of claim 5 comprises:

6. A method of automatically calibrating an offline line, characterized by, S1: placing the vehicle on a production line and starting the electronic outside rearview mirror system; ​ S2: placing the feature pattern in the visual area of the camera, the camera acquires the feature pattern and forms first image information, and inputs the first image information into a first model of the controller, the first model judges whether the matching degree of the first image information and the first information reaches a first preset value, if yes, the first image information is displayed on the display screen and the calibration mode is started, and S3 is entered; if not, the display screen displays error information to remind manual intervention; wherein the feature pattern is an encoding pattern that can trigger the electronic outside rearview mirror system to enter the calibration mode; S3: the camera acquires the external features of the vehicle and forms second image information, and inputs the second image information into a second model of the controller, the second model judges whether the matching degree of the second image information and the first information of the vehicle reaches a second preset value, if yes, S4 is entered; if not, the display screen displays error information to remind manual intervention; S4: after adjusting the position of the camera, the external features of the vehicle are acquired and second image information is formed, and the second image information is transmitted to the second model of the controller, the second model judges whether the second image information and the second information of the vehicle reaches a third preset value, if yes, it represents that the calibration is completed, and the calibration mode is exited; if not, the display screen displays error information to remind manual intervention; Wherein, the first information of the vehicle is that the external features of the reference vehicle are acquired when the reference camera is placed in the standard position, and the external features include: the number and corresponding position of the tires in the image acquired by the reference camera in the standard position, or the number and corresponding position of the door handles; Wherein, the second information of the vehicle is that the shape features of the reference vehicle are acquired when the reference camera is placed in the standard position, and the shape features include: the specific shape of the door handle or the specific shape of the door acquired by the reference camera in the standard position.

7. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized by The processor executes the program to realize the steps of the method of claim 6.

8. A storage medium having stored thereon a program, characterized by The program is executed by the processor to realize the steps of the method of claim 6.

Images