Vehicle-mounted camera calibration method and device, vehicle and computer readable storage medium

Abstract

The invention relates to the field of vehicle calibration, in particular to a vehicle-mounted camera calibration method and device, a vehicle and a computer readable storage medium. The method comprises the following steps: responding to a calibration operation request input by a user for a vehicle-mounted camera; controlling a preset first finite-state machine to perform state transition until the first finite-state machine reaches a final state; the construction process of the first finite-state machine comprises the following steps: determining a first state library according to candidate calibration states of the vehicle-mounted camera; according to a preset calibration algorithm corresponding to the vehicle-mounted camera, defining a first transfer path between the states in the first state library and a first constraint condition of the first transfer path; constructing a first finite-state machine according to the first state library, the first transfer path and the first constraint condition; and returning a calibration result of the vehicle-mounted camera to the user according to the final state. According to the invention, the calibration efficiency and flexibility of the vehicle-mounted camera can be improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a vehicle-mounted camera calibration method, device, vehicle and computer-readable storage medium. Background Art

[0002] With the popularity of autonomous driving, the demand for sensor calibration is increasing. Calibration represents the spatial synchronization information between the sensor and the car body, and is the basis and important underlying technology for autonomous driving.

[0003] There are two ways to calibrate a vehicle's surround view camera. One is static calibration, which uses a single-frame image to determine the external parameters of the surround view camera in a calibration room or a fixed location. The other is dynamic calibration, which uses multiple frames of images to complete the calibration in an environment that does not require advance layout, such as a road.

[0004] Compared with static calibration, vehicle camera calibration does not rely on a specific calibration room, has lower cost, more flexible implementation, and better environmental adaptability. However, the current dynamic calibration design generally follows the pre-set logic from beginning to end, and the control of the vehicle camera calibration algorithm is single-line sequential, which is limited to a single dynamic calibration scenario and simple design requirements, and has high resource usage and low response speed.

[0005] Therefore, a more efficient vehicle camera calibration solution is needed. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a vehicle-mounted camera calibration method, device, vehicle and computer-readable storage medium, which can solve the problems of low efficiency and lack of flexibility in the existing vehicle-mounted camera calibration.

[0007] In a first aspect, an embodiment of the present application provides a vehicle-mounted camera calibration method, the vehicle-mounted camera calibration method comprising: In response to a calibration operation request input by a user for a vehicle-mounted camera; controlling a preset first finite state machine to perform state transfer until the first finite state machine reaches a final state; wherein the construction process of the first finite state machine includes: determining a first state library according to candidate calibration states of the vehicle-mounted camera; defining a first transfer path between each state in the first state library and a first constraint condition of the first transfer path according to a preset calibration algorithm corresponding to the vehicle-mounted camera; constructing the first finite state machine according to the first state library, the first transfer path and the first constraint condition; and returning the calibration result of the vehicle-mounted camera according to the final state.

[0008] By adopting the above scheme, the first finite state machine is constructed by combining the candidate calibration states of the vehicle-mounted camera and the calibration algorithm corresponding to the vehicle-mounted camera. The first finite state machine is used to automatically process different calibration states and realize automatic calibration of the vehicle-mounted camera. At the same time, by arranging the first finite state machine, it can flexibly adapt to different dynamic calibration application scenarios and complex design requirements, realize intelligent management of dynamic calibration of surround view cameras, and improve the calibration efficiency, calibration accuracy, working stability and flexibility of calibration schemes of cameras. In addition, the calibration logic of the vehicle-mounted camera is converted into the jump logic between multiple states of the first finite state machine, which is convenient for the fallback and adjustment of the calibration action, has low resource occupation and high response speed processing capability, and can cope with high-frequency state transitions and event processing.

[0009] In some embodiments, the calibration algorithm includes a plurality of calibration steps; a state in the first state library is used to represent one of uncalibrated, calibrated, successfully calibrated, and failed to calibrate; wherein the state representing the calibrated includes a plurality of sub-states; one of the sub-states represents that a calibration step is in progress; The method further comprises: analyzing the calibration algorithm to obtain the execution order of the multiple calibration steps and the conditions that must be met before the execution of each calibration step; The first transfer path and the first constraint condition are defined according to the execution order and the conditions that need to be satisfied before the execution.

[0010] In some embodiments, the first constraint condition includes an event that triggers the state transition and an action that is performed after the state transition; the method further includes: for a first transition path from any of the sub-states to a state where calibration fails, defining the event that triggers the state transition as a calibration step execution failure corresponding to any of the sub-states; For the first transition path from the first substate to the second substate, the event triggering the state transition is defined as the successful execution of the calibration step corresponding to the first substate, and the action executed after the state transition executes the calibration step corresponding to the second substate; wherein the calibration step corresponding to the first substate is the previous step of the calibration step corresponding to the second substate; With respect to the first transition path between the sub-state corresponding to the last executed calibration step and the state representing successful calibration, the event triggering the state transition is defined as the successful execution of the last executed calibration step.

[0011] In some embodiments, the calibration operation request is sent by the user through a preset calibration interface; the interface state of the calibration interface is updated based on the calibration operation request; The method further includes: using the calibration operation request as an event, triggering a preset second finite state machine to transfer from a current state of being empty to a starting state, and obtaining a first target state to which the second finite state machine transfers; The construction process of the second finite state machine includes: determining a second state library according to the candidate calibration states; defining a second transition path between each state in the second state library and a second constraint condition of the transition path according to a conversion trigger event between the candidate calibration states; constructing the second finite state machine according to the second state library, the second transition path and the second constraint condition; the conversion trigger event includes that the interface state of the calibration interface meets a preset condition; Defining a state mapping relationship between states in the first state library and states in the second state library; Mapping the first target state according to the state mapping relationship to obtain a second target state; The first finite state machine is controlled to perform state transition with the second target state as the current state.

[0012] In some embodiments, the conversion trigger event further includes a state mapping event between the second finite state machine and the first finite state machine; the second state library includes a state representing the state being calibrated; The method also includes: for the current state representing the state being calibrated, defining the second transfer path as a state in a second state library to which the second finite state machine is transferred to a state after the transfer of the first finite state machine is mapped, and defining the second constraint condition as a state transfer event of the first finite state machine.

[0013] In some embodiments, the second constraint condition includes a calibration interface update action performed after the second finite state machine state is transferred; the method further includes: in response to the first finite state machine reaching a final state; Mapping the final state based on the state mapping relationship to obtain a mapped state; The second finite state machine is controlled to perform state transfer with the mapped state as the current state, and the calibration interface update action corresponding to the state transfer is executed to display the calibration result through the calibration interface.

[0014] In some embodiments, the second state library also includes a state representing a calibration failure; The method further includes: for a second transition path from any state not representing calibration failure in the second state library to the calibration failure state, defining the second constraint condition as a non-updated time length of the calibration interface being greater than a preset time length threshold.

[0015] In a second aspect, an embodiment of the present application further provides a vehicle-mounted camera calibration device, the vehicle-mounted camera calibration device comprising: A response module, used to respond to a calibration operation request input by a user for the vehicle-mounted camera; A control module, used to control a preset first finite state machine to perform state transfer until the first finite state machine reaches a final state; wherein the construction process of the first finite state machine includes: determining a first state library according to the candidate calibration state of the vehicle-mounted camera; defining a first transfer path between each state in the first state library and a first constraint condition of the first transfer path according to a preset calibration algorithm corresponding to the vehicle-mounted camera; and constructing the first finite state machine according to the first state library, the first transfer path and the first constraint condition; A return module is used to return the calibration result of the vehicle-mounted camera according to the final state.

[0016] In a third aspect, an embodiment of the present application further provides a vehicle, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the vehicle executes the vehicle camera calibration method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a processor, the processor executes the vehicle-mounted camera calibration method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The present invention is a flowchart of a sub-step of a vehicle camera calibration method provided according to an embodiment of the present application.

[0019] Figure 2 The present invention is a flowchart of a sub-step of a vehicle camera calibration method provided according to an embodiment of the present application.

[0020] Figure 3 The present invention is a flowchart of a sub-step of a vehicle camera calibration method provided according to an embodiment of the present application.

[0021] Figure 4 The present invention is a flowchart of a sub-step of a vehicle camera calibration method provided according to an embodiment of the present application.

[0022] Figure 5 The present invention is a flowchart of a sub-step of a vehicle camera calibration method provided according to an embodiment of the present application.

[0023] Figure 6A schematic diagram of the state transition process of the first finite state machine in the vehicle camera calibration method provided according to an embodiment of the present application is shown.

[0024] Figure 7 A schematic diagram of the state transition process of the second finite state machine in the vehicle camera calibration method provided according to an embodiment of the present application is shown.

[0025] Figure 8 The figure is a schematic diagram of the structure of a vehicle-mounted camera calibration device provided according to an embodiment of the present application.

[0026] Fig. 9 A schematic diagram of the structure of a vehicle calibrated with a vehicle-mounted camera according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the implementation methods of the present application and the features in the implementation methods can be combined with each other without conflict.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The described implementations are only part of the implementations of the present application, rather than all the implementations.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0030] It should be further noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0031] In this application, "at least one" means one or more, and "more" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0032] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0033] In related technologies, with the popularization of autonomous driving, the demand for sensor calibration is increasing. Calibration represents the spatial synchronization information between the sensor and the car body, and is the basis and important underlying technology for autonomous driving. There are two ways to calibrate a vehicle's surround view camera. One is static calibration, which determines the external parameters of the surround view camera through a single frame image in a calibration room or a fixed site; the other is dynamic calibration, which completes the calibration through multiple frames of images in an environment that does not require advance layout, such as roads. Compared with static calibration, vehicle-mounted camera calibration does not rely on a specific calibration room, has lower costs, is more flexible to implement, and has better environmental adaptability.

[0034] Taking the on-board camera as a surround-view camera as an example, the dynamic on-board camera calibration of the surround-view camera includes three parts: one is the dynamic calibration UI (User Interface) operation display such as interface triggering, calibration progress bar, calibration result status display, etc.; the second is the dynamic calibration scene and the dynamic calibration algorithm that depends on the characteristics of the calibration scene; the third is the controller that supports the dynamic on-board camera calibration function logic. In the process of implementing the prior art, the inventor found that the existing dynamic calibration scheme follows the pre-set logic from beginning to end, and the control of the on-board camera calibration algorithm is single-line sequential, and is also limited to a single dynamic calibration scene and simple design requirements, and has high resource occupancy and low response speed.

[0035] From the above, it can be seen that the existing vehicle-mounted camera calibration has the problem of low efficiency.

[0036] In view of this, the embodiments of the present application provide a vehicle-mounted camera calibration method, device, vehicle and computer-readable storage medium, which can improve the efficiency of vehicle-mounted camera calibration.

[0037] See also Figure 1, is a flowchart of the steps of the vehicle camera calibration method provided in an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The vehicle camera calibration method provided in this embodiment is executed based on a preset processor, which is a vehicle with data reading and processing capabilities, such as a computer, a vehicle controller, a microprocessor, etc.

[0038] See also Figure 1 As shown, the vehicle-mounted camera calibration method may include the following steps: Step 101: Responding to a calibration operation request input by a user for a vehicle-mounted camera.

[0039] Among them, the vehicle-mounted camera may include a vehicle-mounted surround view camera, a vehicle-mounted circumferential camera, etc., which is not limited in the embodiments of the present application. The calibration operation request may be sent by a user based on a preset calibration interface. The calibration operation request is used to request a target type calibration operation for the vehicle-mounted camera. The target type may be entering the calibration interface, starting calibration, exiting calibration, pausing calibration, displaying calibration results, etc.

[0040] Step 102: Control a preset first finite state machine to perform state transfer until the first finite state machine reaches a final state; wherein the construction process of the first finite state machine includes: determining a first state library according to the candidate calibration state of the vehicle-mounted camera; defining a first transfer path between each state in the first state library and a first constraint condition of the first transfer path according to a preset calibration algorithm corresponding to the vehicle-mounted camera; and constructing the first finite state machine according to the first state library, the first transfer path and the first constraint condition.

[0041] Among them, the final state refers to a state where there is no transfer path that can make the current state transfer to other states. The candidate calibration state is used to characterize the type of calibration process that the vehicle-mounted camera is in, such as uncalibrated, calibrated, and calibration completed. Among them, the calibration completed state can be divided into calibration success and calibration failure. Optionally, the calibrated state can be further divided according to the object targeted by the current calibration action and the type of calibration action, such as calibrating the front and rear cameras, calibrating the left and right cameras, verifying the front and rear cameras, verifying the left and right cameras, and joint verification of all cameras. The calibration algorithm is used to characterize the method of calibrating the vehicle-mounted camera. The calibration algorithm may include multiple calibration steps executed in a preset order. The execution of each calibration step needs to meet preset conditions, such as the preset data has been obtained, the previous calibration step has been completed, and the execution structure of the previous calibration step meets the preset conditions. Among them, the full camera refers to all cameras in all directions such as front, back, left, and right.

[0042] The state transfer process of the first finite state machine is described as follows: based on the current state (a state in the first state library), it is detected whether the first constraint conditions of each first transfer path corresponding to the current state are met. If there are first constraint conditions that meet the first constraint conditions, the first finite state machine is transferred to the state according to the first transfer path corresponding to the first constraint conditions that meet the first constraint conditions, and the next state of the first finite state machine (another state in the first state library) is obtained. The next state will be used as the current state at the next state transfer. Among them, the first transfer path is used to characterize the transfer path between two first states, and the transfer path is unidirectional, such as transferring from the state being calibrated to the state where the calibration is completed.

[0043] The first constraint condition is used to characterize the conditions that need to be met or the events that need to occur to enter the first transfer path corresponding to the first constraint condition. Optionally, the first constraint condition can also characterize the action executed after the state transition is performed according to the first transfer path corresponding to the first constraint condition, and the executed action can be empty. For example, after transferring from the first state representing uncalibrated to the first state representing calibrated, the action of obtaining the configuration file of the vehicle-mounted camera can be executed, or, after transferring from the first state representing calibrated to the first state representing completed calibration, the action of displaying the calibration result can be executed.

[0044] Specifically, when defining the first transfer path between each state in the first state library and the first constraint condition of the first transfer path according to the preset calibration algorithm corresponding to the vehicle-mounted camera, the calibration algorithm can be divided into multiple calibration steps, and the first transfer path and the first constraint condition corresponding to the first transfer path are determined according to the association relationship between each calibration step and the impact of the execution result of each calibration step on the calibration state. Among them, the association relationship includes the association between the execution order of the calibration steps, the execution result, and the execution trigger condition. The impact of the execution result of the calibration step on the calibration state includes the change of the calibration state if the execution result meets the preset conditions, such as when any one of the front and rear cameras or the left and right cameras fails to be verified, the calibration state is directly transferred to the calibration failure state, and when the joint verification of the front and rear cameras and the left and right cameras is successful, the calibration state is directly transferred to the calibration success state.

[0045] Specifically, the current state of the first finite state machine may be set according to the calibration operation request, so that the first finite state machine enters a state transfer process from the set current state until it transfers to the final state.

[0046] For example, when the calibration operation request represents a user request to enter a pre-calibration state, such as when the user clicks to enter a preset calibration interface but does not request to start calibration, the current state of the first finite state machine is set to uncalibrated. When the calibration operation request represents a user request to start calibration, the current state of the first finite state machine is set to start calibration.

[0047] When the calibration operation request indicates that the user requests to stop calibration, the present state of the first finite state machine is set to calibration completion.

[0048] Specifically, the calibration algorithm includes multiple calibration steps. For example, the multiple calibration steps for the vehicle-mounted camera are arranged in the order of execution as an initialization step, a calibration data acquisition step, a camera parameter calibration step, and a camera parameter verification step; wherein the initialization step includes initializing a dynamic calibration algorithm, initializing a parameter verification algorithm, starting a dynamic vehicle-mounted camera calibration process, and obtaining at least one of the configuration files required for the dynamic calibration algorithm and the parameter verification algorithm. Among them, the dynamic calibration algorithm is used to dynamically calibrate the vehicle-mounted camera, and the parameter verification algorithm is used to verify the eligibility of the calibrated parameters. The configuration file may include the vehicle where the vehicle-mounted camera is located and / or the configuration file related to the calibration verification of the vehicle-mounted camera.

[0049] Optionally, considering that there may be multiple vehicle-mounted cameras, such as front and rear cameras, and left and right cameras, the camera parameter calibration step and the camera parameter verification step may include multiple sub-steps accordingly. For example, the camera parameter calibration step may include a front and rear camera parameter calibration step, and a left and right camera parameter calibration step; the camera parameter verification step may include a front and rear camera parameter verification step, a left and right camera parameter verification step, and a full camera joint verification step.

[0050] The calibration data includes the original image data collected by the vehicle-mounted camera. It is understood that after the initialization is completed, the original image data is processed to obtain image features, and the vehicle-mounted camera parameters are calibrated based on the initialized calibration algorithm according to the image features, and the calibrated parameters are verified based on the initialized parameter verification algorithm.

[0051] A state in the first state library is used to represent one of uncalibrated, calibrated, successfully calibrated, and failed to calibrate. Uncalibrated means that the calibration process has not been entered, successfully calibrated means that the calibration is completed and the calibration parameters have passed the verification, and the calibration parameters are qualified and available, and successfully calibrated means that the calibration is completed and the calibration parameters have not passed the verification, and the calibration parameters are unavailable. Among them, the state representing that the calibration is in progress includes multiple sub-states; one of the sub-states represents that a calibration step is in progress. For example, the state representing that the calibration is in progress may include that the front and rear cameras are being calibrated, the front and rear camera calibration parameters are being verified, the left and right cameras are being calibrated, the left and right camera calibration parameters are being verified, and the whole camera is being jointly verified.

[0052] In some embodiments, Figure 2 As shown, the process of defining the first transfer path and the first constraint condition may include: Step 1021: Analyze the calibration algorithm to obtain the execution order of the multiple calibration steps and the conditions that must be met before the execution of each calibration step.

[0053] Among them, the conditions that need to be met before execution may include the steps that need to be completed before the calibration step is executed, the data that needs to be obtained, the parameter status that the vehicle-mounted camera needs to reach, etc. According to the action execution unit that can be executed independently, the calibration algorithm is divided into multiple calibration steps. The role of each calibration step in the calibration algorithm and the correlation between the steps are analyzed to obtain the execution order between each calibration step and the conditions that need to be met before the execution of each calibration step. For example, the role of the initialization step is to perform preparatory work before the formal parameter calibration, and the role of the parameter calibration step is to calibrate the parameters of the vehicle-mounted camera, and the role of the parameter verification step is to verify the calibrated parameters to determine whether the parameters are available. Therefore, the order of execution should be the initialization step, the parameter calibration step, and the parameter verification step, and the conditions that need to be met before the execution of each calibration step include the successful execution of the previous step of the step, such as successful initialization, completion of parameter calibration, etc.

[0054] Step 1022: Define the first transfer path and the first constraint condition according to the execution order and the conditions that need to be satisfied before the execution.

[0055] Among them, according to the execution order, a first transfer path is set between the sub-states corresponding to the calibration steps executed successively, and the transfer direction of the first transfer path is to transfer the sub-state corresponding to the step executed earlier to the sub-state corresponding to the step executed later. For example, the sub-states may include front and rear camera parameter calibration and front and rear camera parameter verification, and the first transfer path may indicate the transfer from the front and rear camera parameter calibration state to the front and rear camera parameter verification state.

[0056] The first constraint condition corresponding to the first transition path is determined according to the condition that must be satisfied before execution corresponding to the state indicated by the first transition path. Continuing with the above example, for the first transition path from the state of front and rear camera parameter calibration to the state of front and rear camera parameter verification, the first constraint condition corresponding to the first transition path may include that the front and rear camera parameter calibration is completed, that is, there are parameters of the front and rear cameras that can be verified.

[0057] The first constraint condition includes an event that triggers a state transition and an action that is executed after the state transition.

[0058] Optionally, for the first transfer path from uncalibrated to calibrated, the corresponding first constraint condition may be that the dynamic vehicle camera calibration process is scheduled to start, the initialization step is completed, and the like.

[0059] In some embodiments, Figure 3 As shown, step 1022 also includes: Step 1023: For the first transition path from any of the sub-states to the state of calibration failure, define the event that triggers the state transition as the failure to execute the calibration step corresponding to any of the sub-states.

[0060] The criterion for successful calibration of a vehicle-mounted camera is generally that the calibration steps of all the vehicle-mounted cameras are successfully executed. Therefore, the failure to execute the calibration steps corresponding to any sub-state of the calibrating state will result in the failure of the calibration of the vehicle-mounted camera, such as the failure of verification of the front and rear cameras, the failure of verification of the left and right cameras, the failure of the joint verification of all cameras, etc. Therefore, in an embodiment of the present invention, for the first transition path from any of the sub-states to the state of calibration failure, the event that triggers the state transition is defined as the failure to execute the calibration steps corresponding to any of the sub-states.

[0061] Optionally, considering that the calibration process generally does not fail, the execution failure generally refers to the failure of parameter verification, that is, the parameter verification step fails. For example, when any calibration step of the front and rear camera parameter verification, the left and right camera parameter verification, and the full camera parameter joint verification fails, the current state is transferred to the calibration failure state.

[0062] Step 1024: For the first transition path from the first sub-state to the second sub-state, the event that triggers the state transition is defined as the successful execution of the calibration step corresponding to the first sub-state, and the action performed after the state transition executes the calibration step corresponding to the second sub-state; wherein, the calibration step corresponding to the first sub-state is the previous step of the calibration step corresponding to the second sub-state.

[0063] Correspondingly, considering that in the calibration process of the vehicle-mounted camera, the execution condition of each calibration step is generally that the preceding calibration steps are all successfully executed, if the calibration and verification of the front and rear cameras and the left and right cameras should all be passed before the full camera parameter verification step is executed. Therefore, in the embodiment of the present invention, in order to ensure that in the process of the first finite state machine automatically transferring states based on the set current state, the automatic calibration of the vehicle-mounted camera can be completed correctly according to the logic and process specified by the calibration algorithm, for the first transfer path from the first sub-state to the second sub-state, the event that triggers the state transfer is defined as the successful execution of the calibration step corresponding to the first sub-state, and the action executed after the state transfer executes the calibration step corresponding to the second sub-state; wherein, the calibration step corresponding to the first sub-state is the previous step of the calibration step corresponding to the second sub-state.

[0064] Step 1025: For the first transition path between the sub-state corresponding to the last executed calibration step and the state representing successful calibration, define the event triggering the state transition as the successful execution of the last executed calibration step.

[0065] Combined with the above-mentioned step 1023, when all calibration steps are successfully executed, it can be determined that the calibration process of the vehicle-mounted camera is completed and the calibration is successful. Before each calibration step is executed, its previous calibration step should be successfully executed, otherwise the calibration will be directly terminated and a calibration failure result will be obtained. Therefore, in the embodiment of the present invention, for the first transition path between the sub-state corresponding to the last executed calibration step and the state representing the successful calibration, the event that triggers the state transition is defined as the successful execution of the last executed calibration step.

[0066] In some embodiments, the calibration operation request is sent by the user through a preset calibration interface; the interface state of the calibration interface is updated based on the calibration operation request. The preset calibration interface is used for the user to input the calibration operation request, and optionally, the calibration interface can also display calibration process progress information and / or calibration results.

[0067] In order to improve the user experience of vehicle camera calibration, facilitate the user to control the calibration process and understand the real-time status and results of the calibration process, the embodiment of the present invention constructs a second finite state machine according to the logic of the calibration interface and user interaction and interface transformation. Specifically, Figure 4 As shown, in one embodiment of the present invention, step 102 further includes: Step 1101: Taking the calibration operation request as an event, triggering the preset second finite state machine to transfer from the current empty start state, and obtaining the first target state to which the second finite state machine transfers; wherein the construction process of the second finite state machine includes: determining the second state library according to the candidate calibration state; defining the second transfer path between each state in the second state library and the second constraint condition of the transfer path according to the conversion trigger event between the candidate calibration states; constructing the second finite state machine according to the second state library, the second transfer path and the second constraint condition; the conversion trigger event includes that the interface state of the calibration interface satisfies the preset condition.

[0068] The interface state of the calibration interface can be changed based on the operation request input by the user, or it can be the state of the interface itself. For example, the interface state can be that the user inputs a start operation request, or it can be that the interface is currently a progress bar display interface, and the current interface has not been updated for 5 minutes. The preset condition can be that a preset type of user operation request is detected in the calibration interface, or the state of the calibration interface itself meets the preset state parameter condition, such as the current interface has not been updated for more than 5 minutes.

[0069] The second finite state machine is used to respond to the user's calibration interaction operation and display to the user according to the real-time calibration state of the vehicle-mounted camera. In the embodiment of the present invention, the automatic calibration of the vehicle-mounted camera is realized by the first finite state machine. Therefore, in some embodiments, the conversion trigger event also includes a state mapping event between the second finite state machine and the first finite state machine. Among them, the state mapping event refers to mapping according to the current state of the first finite state machine to obtain the current state of the second finite state machine, thereby realizing the state transfer between the second finite state machine and the first finite state machine, so as to automatically update the calibration interface under the control of the second finite state machine.

[0070] Specifically, the second state library includes a state representing the state being calibrated; in some embodiments, the definition process of the second transfer path and the second constraint condition further includes: With respect to the current state, which represents the state being calibrated, the second transfer path is defined as transferring the second finite state machine to the state in the second state library mapped to the post-transfer state of the first finite state machine, and the second constraint condition is defined as a state transfer event of the first finite state machine.

[0071] The process of visualizing the calibration process is implemented and controlled by the second finite state machine. At the same time, a state mapping relationship is established between the second finite state machine and the first finite state machine to realize the state transfer between the second finite state machine and the first finite state machine, thereby triggering the state change of the first finite state machine according to the user's operation request in the calibration interface, and transferring the changed state of the first finite state machine to the second finite state machine, controlling the second finite state machine to enter the state transfer process with the synchronized state as the current state, realizing automatic calibration, and then recursively transferring the state of the second finite state machine to the first finite state machine to realize the update of the calibration interface, and returning the real-time calibration status and results to the user.

[0072] In some embodiments, the second state library also includes a state indicating a calibration failure; and the definition process of the second transfer path and the second constraint condition further includes: For a second transition path from any state not representing calibration failure in the second state library to the state of calibration failure, the second constraint condition is defined as a non-updated time length of the calibration interface being greater than a preset time length threshold.

[0073] Among them, considering that the change of the interface state of the calibration interface can be based on the user request or on the transfer of the state of the first finite state machine, if the first finite state machine does not transfer the state to the second finite state machine within the preset time length, it indicates that an abnormality has occurred in the process of vehicle-mounted camera calibration based on the second finite state machine, resulting in the state of the second finite state machine cannot be updated normally. Since the first finite state machine does not transfer the state to the second finite state machine within the preset time length after the calibration starts, the interface state of the second finite state machine will not be updated, and will remain in a specific interface state, such as staying in the calibration progress bar display state with a calibration progress of 50%. Therefore, in an embodiment of the present invention, when the calibration interface has not been updated for a period of time greater than the preset time length threshold, it is determined that the vehicle-mounted camera calibration has failed, and the second finite state machine is controlled to directly transfer to the final state of calibration failure, so as to facilitate the display of the calibration failure result to the user.

[0074] Step 1102: define a state mapping relationship between states in the first state library and states in the second state library.

[0075] Among them, considering that the states of the first state library correspond to specific calibration steps, the granularity is relatively fine, such as the calibration of the left and right cameras is in progress, the calibration of the left and right cameras is in progress, etc., while the states in the second state library are used to generally show the user whether the calibration results and process are normal, such as whether the calibration has started, whether the calibration is successful, etc. The user may not or does not need to care about the specific fine-grained calibration steps currently in place. Therefore, in an embodiment of the present invention, the states of the first state library are converged and summarized according to whether the calibration process represented by the state is started and normal, and then mapped to the states in the second state library. For example, the idle state and the uncalibrated state in the first state library are mapped to the uncalibrated state in the second state library. For another example, the left and right camera calibration, front and rear camera calibration, left and right camera verification, front and rear camera verification and the uncalibrated state in the first state library are mapped to the calibrated state in the second state library. The mapping of the states representing the calibration results between the first state library and the second state library is relatively simple. The state representing the calibration failure in the first state library is mapped to the state representing the calibration failure in the second state library, and the state representing the calibration success in the first state library is mapped to the state representing the calibration success in the second state library.

[0076] Step 1103: Map the first target state according to the state mapping relationship to obtain a second target state.

[0077] The state to which the first target state is mapped according to the state mapping relationship is queried, and the mapped state is determined as the second target state.

[0078] Step 1104: Control the first finite state machine to perform state transition with the second target state as the current state.

[0079] The second target state is set as the current state of the first finite state machine, so that the first finite state machine enters a state transfer process starting from the current state.

[0080] Step 103: Return the calibration result of the vehicle-mounted camera according to the final state.

[0081] Specifically, the calibration result can be determined according to the mapping relationship between the final state and the calibration result. Specifically, when the final state is a state that represents a successful calibration, the calibration result is determined to be a successful calibration. Correspondingly, when the final state represents a state other than a successful calibration, the calibration result is determined to be a failed calibration. Optionally, for a state that is not successfully calibrated, when the final state represents uncalibrated, the calibration result is determined to be uncalibrated, and when the final state represents a failed calibration, the calibration result is determined to be a failed calibration.

[0082] Optionally, the calibration result may be returned to the user, and the return method may be displayed through a preset calibration interface.

[0083] In some embodiments, based on the automatic calibration achieved by the first finite state machine, in order to automatically control the state change of the calibration interface according to the calibration state of the vehicle-mounted camera, in one embodiment of the present invention, the second constraint condition includes a calibration interface update action executed after the state of the second finite state machine is transferred; Figure 5 As shown, step 103 also includes: Step 1031: In response to the first finite state machine reaching a final state, the final state is mapped based on the state mapping relationship to obtain a mapped state.

[0084] When it is detected that the first finite state machine reaches the final state, the final state is mapped according to the state mapping relationship defined in the aforementioned step 1102 to obtain the state to which the second finite state machine needs to be synchronized. For example, when the final state of the first finite state machine is calibration failure, it is determined according to the state mapping relationship that the current state of the second finite state machine should also be updated to calibration failure. For another example, when the final state of the first finite state machine is the sub-state of front and rear camera calibration during the calibration of the vehicle-mounted camera, it can be seen from the state mapping relationship that the sub-state is mapped to the state of being calibrated.

[0085] Step 1032: Control the second finite state machine to perform state transfer with the mapped state as the current state, and execute the corresponding calibration interface update action after the state transfer, so as to display the calibration result through the calibration interface.

[0086] The mapped state is passed to the second finite state machine as the current state of the second finite state machine, so that the second finite state machine starts state transition from the mapped state and executes the calibration interface update action corresponding to the state transition. For example, when the mapped state is successfully calibrated, the calibration interface is controlled to switch to an interface state that displays the calibration result of the successful calibration. For another example, when the mapped state is calibrating, the calibration interface is controlled to switch to an interface state that displays a calibration progress bar.

[0087] The embodiment of the present invention separates the logic of the interaction between the calibration interface and the user from the underlying vehicle calibration logic, implements the logic of the interaction between the calibration interface and the user through a first finite state machine, and implements vehicle calibration through a second finite state machine, so as to adapt to different dynamic calibration application scenarios and complex design requirements by arranging the first finite state machine and the second finite state machine. It has low resource occupancy and high response speed processing capability, can cope with high-frequency state transitions and event processing, and can realize intelligent management of the dynamic calibration of the surround view camera, thereby improving the calibration efficiency, calibration accuracy and working stability of the camera.

[0088] Combination Figure 6 as well as Figure 7 The vehicle camera calibration process in one embodiment of the present invention is described as follows: in, Figure 6 A schematic diagram showing the process of state transfer of the first finite state machine is shown, Figure 7 A schematic diagram showing the process of state transfer of the second finite state machine is shown.

[0089] First combine Figure 6 The state transfer process of the first finite state machine is explained. The first finite state machine is tightly coupled with the vehicle camera calibration algorithm and can be mapped to the state of the second finite state machine. The first finite state machine defines a state set, a condition set, and an action set. The current state and the secondary state are both derived from the state set. By arranging the current state, condition, action, and secondary state, it can adapt to different dynamic calibration application scenarios and design requirements. The specific definitions are as follows: State set 2SS: {0-idle, 1-uncalibrated, 2-started, 3-front and rear camera calibration, 4-front and rear surround camera parameter verification, 5-left and right camera calibration, 6-left and right surround camera parameter verification, 7-all camera joint verification, 8-calibration failed, 9-calibration successful}.

[0090] Event set 2CS: {0-dynamic calibration algorithm and parameter evaluation algorithm have not been initialized, 1-dynamic vehicle camera calibration process is scheduled to start, 2-vehicle and camera related configuration files and other information have been obtained, 3-multi-frame image fisheye data has been prepared, 4-multi-frame image fisheye data meets the input of the calibration algorithm, 5-front and rear surround view fisheye camera extrinsic parameters have been calibrated, 6-front and rear surround view fisheye camera extrinsic parameter calibration verification is completed, 7-left and right surround view fisheye camera extrinsic parameters have been calibrated, 8-front, rear, left and right four surround view cameras calibration is completed, 9-front, rear, left and right four surround view cameras extrinsic parameter joint verification is completed}.

[0091] Action set 2AS: {0-initialize the calibration algorithm and parameter evaluation algorithm, start the dynamic vehicle camera calibration process, 1-obtain the vehicle body and camera related configuration file information required for the dynamic calibration algorithm processing and parameter evaluation algorithm verification, 2-trigger the dynamic vehicle camera calibration fisheye image data acquisition module to collect multiple frames of fisheye image calibration data, 3-successfully collect multiple frames of fisheye image data for calibration, 4-the calibration algorithm calculates the external parameters of the front and rear surround view cameras, 5-the verification algorithm verifies that the external parameters of the front and rear surround view cameras are qualified, 6-the calibration algorithm calculates the external parameters of the left and right surround view cameras, 7-verifies that the external parameters of the left and right surround view cameras are qualified, 8-jointly verifies that the external parameters of the front, rear, left and right surround view cameras are qualified, 9-cancel the collection of data for calibration, 10-verifies that the external parameters of the front and rear surround view cameras are unqualified, 11-verifies that the external parameters of the left and right surround view cameras are unqualified, 12-jointly verifies that the external parameters of the front, rear, left and right surround view cameras are unqualified, 13-exit the collection of vehicle camera calibration data}.

[0092] Under the condition of 2CS0, the link secondary state is 2SS0 "idle" and action 2AS0 is executed. Under the condition of 2CS1, the link secondary state of the current state 2SS0 is 2SS1 "uncalibrated" and action 2AS1 is executed. Under the condition of 2CS2, the secondary state of the current state 2SS1 is 2SS2 "started" and action 2AS2 is executed. Under the condition of 2CS3, the secondary state of the current state 2SS2 is 2SS8 "calibration failed" and action 2AS9 is executed. Under the condition of 2CS3, the secondary state of the current state 2SS2 is 2SS3 "front and back calibration" and action 2AS3 is executed. Under the condition of 2CS4, the secondary state of the current state 2SS3 is 2SS4 "front and back verification" and action 2AS4 is executed. Under the condition of 2CS5, the secondary state of the current state 2SS4 is 2SS8 "calibration failed" and action 2AS10 is executed. Under the condition of 2CS5, the secondary state of the current state 2SS4 is 2SS5 "left and right calibration". And execute action 2AS5, the current state 2SS5 is 2SS6 "left and right check" under the condition of 2CS6 and execute action 2AS6, the current state 2SS6 is 2SS7 "full check" under the condition of 2CS7 and execute action 2AS7, the current state 2SS6 is 2SS8 "calibration failure" under the condition of 2CS7 and execute action 2AS11, the current state 2SS7 is 2SS9 "calibration success" under the condition of 2CS8 and execute action 2AS8, the current state 2SS7 is 2SS8 "calibration failure" under the condition of 2CS8 and execute action 2AS12, the current state 2SS8 is 2SS1 "uncalibrated" under the condition of 2CS9 and execute action 2AS13, the current state 2SS8 is 2SS1 "uncalibrated" under the condition of 2CS9 and execute action 2AS13, the table shows as shown in Table 1 below: Table 1 The first finite state machine transfer path table Recombination Figure 7The state transition process of the second finite state machine is described. First, for the second finite state machine, the state set, condition set, and action set are defined as follows: State set 1SS: {0-uncalibrated, 1-calibrated, 2-calibration failed, 3-calibration successful}.

[0093] Event set 1CS: {0-calibration interface is not in the calibration function operation interface, 1-calibration interface is in the calibration function operation interface, 2-calibration interface is in the calibration result display interface, 3-calibration interface is in the calibration process progress bar display interface, 4-calibration interface shows that the single calibration duration has exceeded the maximum calibration threshold setting time, 5-calibration interface shows that the single calibration duration does not exceed the maximum calibration threshold setting time, 6-user clicks to enter the calibration function operation interface, 7-user clicks to start calibration, 8-user clicks to return to the calibration function operation interface, 9-the first finite state machine transmits a state representing that the calibration parameters are invalid, 10-the first finite state machine transmits a state representing that the calibration parameters are valid}.

[0094] Action set 1AS: {0-enter the calibration function operation interface, 1-transfer the state to the first finite state machine and enter the calibration progress bar display interface, 2-return to the calibration function operation interface, 3-enter the calibration result display interface}.

[0095] The interface display types of the calibration interface include the calibration function operation interface, the calibration result display interface and the calibration process progress bar display interface.

[0096] The actions and sub-states are: when conditions 1CS0 and 1CS6 are met, the sub-state is 1SS0 "uncalibrated" and action 1AS0 is executed; when conditions 1CS1 and 1CS7 are met, the sub-state of current state 1SS0 is 1SS1 "calibrating" and action 1AS1 is executed; when conditions 1CS2 and 1CS8 are met at the same time, the sub-state of current state 1SS2 and 1SS3 is 1SS0 "uncalibrated" and action 1AS2 is executed; when conditions 1CS3, 1CS5 and 1CS10 are met, the sub-state of current state 1SS1 is 1SS3 "calibration successful" and action 1AS4 is executed; when conditions 1CS3, 1CS4 and 1CS9 are met at the same time, the sub-state of current state 1SS1 is 1SS2 "calibration failed" and action 1AS4 is executed. The table is shown in Table 2 below: Table 2 Second finite state machine transfer path table Combination Figure 6 and Figure 7 The vehicle camera calibration process in the embodiment of the present invention is described as follows: First, define the current state, events, actions and next states of the second finite state machine, and define the current state, events, actions and next states of the first finite state machine; establish a mapping relationship between the states of the second finite state machine and the first finite state machine, the second finite state machine is tightly coupled with the UI, and the first finite state machine is tightly coupled with the dynamic calibration algorithm.

[0097] Specifically, the mapping relationship between the states of the second finite state machine and the first finite state machine is defined as follows: states 2SS0 and 2SS1 are mapped to 1SS0 of the first state machine; states 2SS2, 2SS3, 2SS4, 2SS5, 2SS6, and 2SS7 correspond to the first state machine 2SS1; state 2SS8 can be mapped to 1SS2 of the first state machine; state 2SS9 can be mapped to 1SS3 of the first state machine.

[0098] Secondly, enter the calibration function operation interface, synchronously refresh and initialize the corresponding states of the first state machine and the second finite state machine; after triggering the online calibration, read the four-way original fisheye images of the front, back, left and right that are time-synchronized, denoise and dedistort the images, extract semantic geometric features from the dedistorted images, and the calibration function logic of the vehicle-mounted camera jumps to the calibration progress bar display interface under the control of the first state machine.

[0099] Then, the state of the second finite state machine is refreshed. First, data for the vehicle camera calibration algorithm is successfully collected for several consecutive frames. The vehicle camera calibration function logic calculates the initial value of a single extrinsic parameter based on valid geometric features under the control of the second finite state machine, and further optimizes and verifies the extrinsic parameter based on the corner points and straight line features extracted by the geometric features. The camera extrinsic parameters are fully verified by combining the common view geometric features of the cross-fusion area of ​​the front, back, left, and right surround cameras.

[0100] Finally, the second finite state machine synchronizes the calibration result of the vehicle camera calibration algorithm to the first state machine. The first state machine determines whether the vehicle camera calibration is completed according to preset conditions, jumps to the vehicle camera calibration result status display interface, and completes the vehicle camera calibration.

[0101] Please refer to Figure 8 , is a schematic diagram of the structure of the calibration device 201 for the vehicle-mounted camera provided in the embodiment of the present application. Figure 8 As shown, the calibration device 201 of the vehicle-mounted camera includes: A response module 2001 is used to respond to a calibration operation request input by a user for a vehicle-mounted camera; The control module 2002 is used to control a preset first finite state machine to perform state transfer until the first finite state machine reaches a final state; wherein the construction process of the first finite state machine includes: determining a first state library according to the candidate calibration state of the vehicle-mounted camera; defining a first transfer path between each state in the first state library and a first constraint condition of the first transfer path according to a preset calibration algorithm corresponding to the vehicle-mounted camera; and constructing the first finite state machine according to the first state library, the first transfer path and the first constraint condition; The return module 2003 is used to return the calibration result of the vehicle-mounted camera according to the final state.

[0102] Please refer to Fig. 9 , is a schematic diagram of the hardware structure of the vehicle 401 provided in the embodiment of the present application. Fig. 9 As shown, the vehicle 401 may include a processor 4001 and a memory 4002. The memory 4002 is used to store one or more computer programs 4003. The one or more computer programs 4003 are configured to be executed by the processor 4001. The one or more computer programs 4003 include instructions, and the above instructions can be used to implement the above-mentioned vehicle-mounted camera calibration method in the vehicle 401.

[0103] It is understandable that the structure shown in this embodiment does not constitute a specific limitation on the vehicle 401. In other embodiments, the vehicle 401 may include more or fewer components than shown, or combine some components, or separate some components, or arrange the components differently.

[0104] The processor 4001 may include one or more processing units, for example, the processor 4001 may include an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0105] The processor 4001 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 4001 is a cache memory. The memory may store instructions or data that the processor 4001 has just used or circulated. If the processor 4001 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 4001, and thus improves the efficiency of the system.

[0106] In some embodiments, the processor 4001 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0107] In some embodiments, processor 4001 is used to execute acceleration schemes such as single instruction multiple data (SIMD) and very long instruction word (VLIW).

[0108] In some embodiments, memory 4002 may include high-speed random access memory and may also include non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0109] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the instructions are executed on a processor, the processor executes the above-mentioned related method steps to implement the vehicle-mounted camera calibration method in the above-mentioned embodiment.

[0110] Among them, the vehicle, device, and computer-readable storage medium provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0111] In practical applications, the above functions can be distributed to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0112] In several embodiments provided in the present application, the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are schematic. For example, the division of the modules or units is a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0113] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0114] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0116] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A vehicle-mounted camera calibration method, characterized in that: The method comprises: In response to a calibration operation request input by a user for a vehicle-mounted camera, a preset first finite state machine is controlled to perform state transfer until the first finite state machine reaches a final state; wherein the construction process of the first finite state machine includes: determining a first state library according to candidate calibration states of the vehicle-mounted camera; defining a first transfer path between each state in the first state library and a first constraint condition of the first transfer path according to a preset calibration algorithm corresponding to the vehicle-mounted camera; constructing the first finite state machine according to the first state library, the first transfer path and the first constraint condition; The calibration result of the vehicle-mounted camera is returned according to the final state.

2. The method according to claim 1, characterized in that The calibration algorithm includes a plurality of calibration steps; a state in the first state library is used to represent one of uncalibrated, calibrated, successfully calibrated, and failed to calibrate; wherein the state representing the calibrated includes a plurality of sub-states; one of the sub-states represents that a calibration step is in progress; The process of defining the first transfer path and the first constraint condition includes: Analyze the calibration algorithm to obtain the execution order of the multiple calibration steps and the conditions that must be met before the execution of each calibration step; The first transfer path and the first constraint condition are defined according to the execution order and the conditions that need to be satisfied before the execution.

3. The method according to claim 2, characterized in that The first constraint condition includes an event that triggers a state transition and an action that is performed after the state transition; The process of defining the first transfer path and the first constraint condition further includes: For the first transition path from any of the sub-states to the state of calibration failure, the event triggering the state transition is defined as the failure to execute the calibration step corresponding to any of the sub-states; For a first transition path from a first substate to a second substate, the event triggering the state transition is defined as the successful execution of a calibration step corresponding to the first substate, and the action performed after the state transition is defined as the execution of a calibration step corresponding to the second substate; wherein the calibration step corresponding to the first substate is the previous step of the calibration step corresponding to the second substate; With respect to the first transition path between the sub-state corresponding to the last executed calibration step and the state representing successful calibration, the event triggering the state transition is defined as the successful execution of the last executed calibration step.

4. The method according to claim 1, characterized in that: The calibration operation request is sent by the user through a preset calibration interface; the interface state of the calibration interface is updated based on the calibration operation request; The process of controlling the preset first finite state machine to perform state transfer also includes: Taking the calibration operation request as an event, triggering the preset second finite state machine to transfer from the current empty start state, and obtaining the first target state to which the second finite state machine transfers; wherein the construction process of the second finite state machine includes: determining the second state library according to the candidate calibration state; defining the second transfer path between each state in the second state library and the second constraint condition of the transfer path according to the conversion trigger event between the candidate calibration states; constructing the second finite state machine according to the second state library, the second transfer path and the second constraint condition; the conversion trigger event includes that the interface state of the calibration interface meets the preset condition; Defining a state mapping relationship between states in the first state library and states in the second state library; Mapping the first target state according to the state mapping relationship to obtain a second target state; The first finite state machine is controlled to perform state transition with the second target state as the current state.

5. The method according to claim 4, characterized in that The conversion triggering event also includes a state mapping event between the second finite state machine and the first finite state machine; The second state library includes a state representing the state being calibrated; The process of defining the second transfer path and the second constraint condition further includes: With respect to the current state, which represents the state being calibrated, the second transfer path is defined as transferring the second finite state machine to the state in the second state library mapped to the post-transfer state of the first finite state machine, and the second constraint condition is defined as a state transfer event of the first finite state machine.

6. The method according to claim 4, characterized in that The second constraint condition includes a calibration interface update action executed after the second finite state machine state is transferred; and the calibration result of the vehicle-mounted camera is returned according to the final state, including: In response to the first finite state machine reaching a final state, Mapping the final state based on the state mapping relationship to obtain a mapped state; The second finite state machine is controlled to perform state transfer with the mapped state as the current state, and the calibration interface update action corresponding to the state transfer is executed to display the calibration result through the calibration interface.

7. The method according to claim 4, characterized in that The second state library also includes a state representing a calibration failure; The process of defining the second transfer path and the second constraint condition further includes: For a second transition path from any state not representing calibration failure in the second state library to the state of calibration failure, the second constraint condition is defined as a non-updated time length of the calibration interface being greater than a preset time length threshold.

8. A vehicle-mounted camera calibration device, characterized in that: The vehicle-mounted camera calibration device comprises: A response module, used to respond to a calibration operation request input by a user for the vehicle-mounted camera; A control module, used to control a preset first finite state machine to perform state transfer until the first finite state machine reaches a final state; wherein the construction process of the first finite state machine includes: determining a first state library according to the candidate calibration state of the vehicle-mounted camera; defining a first transfer path between each state in the first state library and a first constraint condition of the first transfer path according to a preset calibration algorithm corresponding to the vehicle-mounted camera; and constructing the first finite state machine according to the first state library, the first transfer path and the first constraint condition; A return module is used to return the calibration result of the vehicle-mounted camera according to the final state.

9. A vehicle, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the vehicle executes the vehicle camera calibration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a processor, the processor executes the vehicle-mounted camera calibration method according to any one of claims 1 to 7.

Images