Scene calibration method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a scene calibration method and device, electronic equipment and a storage medium. According to the invention, by obtaining the scene calibration request containing the target scene and the scene demand, the calibration demand is accurately positioned. Then, the corresponding target execution file is obtained according to the request and runs, and the processing thread corresponding to the scene requirement is created, so that the mode that a set of general programs cope with all scenes is avoided, and resources can be allocated for different scenes. And then, calling a processing thread to execute the associated task to obtain a calibration result, and generating a scene calibration result, so that the whole-process personalized adaptation from task execution to result output is realized. Compared with an existing scheme, the method has the advantages that calibration tasks can be completed more comprehensively and efficiently in the face of various calibration scenes, calibration accuracy and efficiency are improved, and differentiated requirements for camera calibration in different scenes are met.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a scene calibration method, device, electronic device and storage medium. Background Art

[0002] With the widespread adoption of autonomous driving technology, sensor calibration, a key underlying technology for determining spatial synchronization between sensors and the vehicle body, has become increasingly important, and the demand for related calibration has also exploded. Currently, vehicle camera calibration primarily utilizes two methods: static calibration and dynamic calibration. Static calibration determines the extrinsic parameters of a surround-view camera from a single image frame in a calibration room or fixed location. Dynamic calibration utilizes multiple image frames to complete the calibration task in environments such as roads that do not require pre-planned calibration. Both methods are implemented using calibration program files or dynamic libraries.

[0003] However, existing solutions either do not involve the calibration program file calling method, or focus on the project packaging end which has low correlation with the calibration calling scenario, or only support dynamic library multi-threaded calling which is difficult to meet complex scheduling requirements, or the calling process is inconsistent with the actual calibration, or focus on the compilation stage and cannot solve the execution and calling problems. All of them cannot fully and efficiently adapt to various calibration scenarios. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a scene calibration method, device, electronic device, and storage medium to solve the problem that existing solutions cannot fully and efficiently adapt to multiple calibration scenarios.

[0005] In a first aspect, an embodiment of the present invention provides a scene calibration method, the method comprising:

[0006] Obtaining a scene calibration request, wherein the scene calibration request includes a target scene and scene requirements of the target scene;

[0007] In response to the scene calibration request, obtaining a target execution file of a camera calibration program corresponding to the target scene, and creating a processing thread corresponding to the scene requirement by running the target execution file;

[0008] The processing thread is called to execute the processing task associated with the scene requirement to obtain a calibration result, and a scene calibration result of the target scene is generated based on the calibration result obtained by each of the processing threads.

[0009] Furthermore, the step of creating a processing thread corresponding to the scenario requirement by running the target execution file includes:

[0010] Invoke a subprocess based on the target scenario;

[0011] Loading the target execution file into the sub-process, and using the sub-process including the target execution file as a calibration process;

[0012] The calibration process is called to run the target execution file, and when the target execution file is running, a corresponding processing thread is created according to the scenario requirements.

[0013] Furthermore, creating a corresponding processing thread according to the scenario requirements includes:

[0014] Obtaining the processing tasks within the scenario requirements and the task attributes of the processing tasks;

[0015] Determine the number of threads and the order in which threads are created using the task attributes;

[0016] Processing threads are created according to the thread creation order and the thread quantity.

[0017] Furthermore, generating a scene calibration result of the target scene based on the calibration result obtained by each processing thread includes:

[0018] Collaboratively analyze the calibration results of each processing thread to obtain the calibration results of adjacent cameras;

[0019] The calibration results of adjacent cameras are used to jointly optimize the extrinsic parameters of adjacent cameras to obtain the optimized extrinsic parameters;

[0020] The calibration results of each processing thread are corrected using the optimized external parameters, and fused based on the corrected calibration results to obtain the scene calibration result.

[0021] Furthermore, the method of jointly optimizing the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters includes:

[0022] Calling an optimization thread to jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters, wherein the optimization thread is created based on the scene requirements; or

[0023] The calibration process is called to jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain the optimized extrinsic parameters.

[0024] Furthermore, the method further comprises:

[0025] Obtain the calibration progress of each processing thread when executing the processing task according to the preset time period;

[0026] Analyzing the calibration progress of the processing thread in each preset time period to obtain a progress change of the processing thread;

[0027] The processing thread whose progress change is less than a preset threshold is regarded as a processing thread to be coordinated;

[0028] The task content of the processing task executed by the thread to be coordinated is obtained, and resource information related to the task content is allocated to the thread to be coordinated.

[0029] Furthermore, after analyzing the marked progress of the processing thread in each preset time period to obtain the progress change of the processing thread, the method further includes:

[0030] The processing thread whose progress change deviates from the preset range is regarded as an exception processing thread;

[0031] Acquiring multimodal data of the current environment and analyzing the degree of influence of the multimodal data on the calibration progress of the exception handling thread;

[0032] A resource allocation operation is performed to the exception handling thread based on the impact level.

[0033] In a second aspect, an embodiment of the present invention provides a scene calibration device, the device comprising:

[0034] An acquisition module, configured to acquire a scene calibration request, wherein the scene calibration request includes a target scene and scene requirements of the target scene;

[0035] A response module, configured to respond to the scene calibration request, obtain a target execution file of a camera calibration program corresponding to the target scene, and create a processing thread corresponding to the scene requirement by running the target execution file;

[0036] The calling module is used to call the processing thread to execute the processing task associated with the scene requirement to obtain a calibration result, and generate a scene calibration result of the target scene based on the calibration result obtained by each processing thread.

[0037] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0038] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0039] The present invention accurately locates the calibration requirements by obtaining a scene calibration request containing the target scene and scene requirements. Then, according to the request, the corresponding target execution file is obtained and run, and a processing thread corresponding to the scene requirement is created, avoiding the mode of a general program to deal with all scenes, and being able to allocate resources for different scenes. Then, the processing thread is called to execute the associated task to obtain the calibration result, and then the scene calibration result is generated, realizing personalized adaptation of the entire process from task execution to result output. Compared with the existing solution, this on-demand customization method can complete the calibration task more comprehensively and efficiently when facing a variety of calibration scenes, improve the calibration accuracy and efficiency, and meet the differentiated requirements for camera calibration in different scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 is a flowchart of a scene calibration method according to some embodiments of the present invention;

[0042] Figure 2 is a schematic diagram of creating a processing thread according to some embodiments of the present invention;

[0043] Figure 3 is a schematic diagram of a scene calibration process according to some embodiments of the present invention;

[0044] Figure 4 is a flowchart of another scene calibration method according to some embodiments of the present invention;

[0045] Figure 5 is a structural block diagram of a scene calibration method and apparatus according to an embodiment of the present invention;

[0046] Figure 6 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0048] According to an embodiment of the present invention, a scene calibration method, apparatus, electronic device and storage medium are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0049] In this embodiment, a scene calibration method is provided. Figure 1 is a flow chart of a scene calibration method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0050] Step S101: Obtain a scene calibration request, wherein the scene calibration request includes a target scene and scene requirements of the target scene.

[0051] In the embodiment of the present application, the host process waits for the scene calibration request in the following two ways:

[0052] If the host process has a UI interface, the user can actively initiate a scene calibration request by performing operations on the UI interface, such as clicking a specific button, entering relevant instructions, etc.

[0053] If the host process acts as the receiving end of the diagnostic network, it will continue to monitor the external network and wait for scene calibration requests sent from other devices (such as diagnostic equipment, vehicle control unit, etc.).

[0054] After receiving the request, the host process parses the request data and uses preset parsing rules to accurately extract the target scene information from the request, clarifying the specific scene types such as parking static, parking dynamic, driving static or driving dynamic; at the same time, further parsing obtains the scene requirements corresponding to the target scene, including but not limited to the camera type that needs to be calibrated (fisheye camera or pinhole camera), the calibration environment (EOL production line, 4S store, road, calibration room, base road test track, etc.), the expected calibration accuracy, the type of features to be detected, and other key information.

[0055] Step S102 , responding to the scene calibration request, obtaining a target execution file of a camera calibration program corresponding to the target scene, and creating a processing thread corresponding to the scene requirement by running the target execution file.

[0056] In an embodiment of the present application, based on the type of target scene in the request, it is determined whether it is a driving scene or a parking scene. If it is a driving scene, it is further determined whether it is static (such as static target calibration in the calibration room) or dynamic (such as base road test track calibration, 4S shop after-sales urban road calibration). If it is a parking scene, its static (EOL production line calibration, 4S calibration) or dynamic (road calibration) attributes are also clarified; according to the judgment result, the corresponding target execution file is accurately obtained from the four types of pre-stored camera calibration program target files, for example, the driving camera static calibration program execution file is obtained for the driving static scene, and the parking camera dynamic calibration program execution file is obtained for the parking dynamic scene.

[0057] In the embodiment of the present application, running the target execution file to create a processing thread corresponding to the scenario requirement includes the following steps A1-A3:

[0058] Step A1: Call the subprocess based on the target scenario.

[0059] Specifically, upon receiving a scene calibration request containing a target scene, the host process, acting as the core controller, determines whether it is a driving-related scene (e.g., driving static, driving dynamic) or a parking-related scene (e.g., parking static, parking dynamic) based on the target scene's specific information. The host process then calls the fork function to create a child process. Initially, this child process inherits some of the host process's properties and resources, but has a separate process identifier, distinguishing it from the host process.

[0060] Step A2: Load the target execution file into the sub-process, and use the sub-process including the target execution file as the calibration process.

[0061] Specifically, after successfully calling the child process, the host process will accurately select the corresponding target execution file from the four types of pre-stored camera calibration program target files (driving camera static calibration program execution file, driving camera dynamic calibration program execution file, parking camera static calibration program execution file and parking camera dynamic calibration program execution file) according to the target scene.

[0062] The target executable file is then loaded into the child process using the execv function. During the loading process, the user-level context is overwritten, switching the child process's execution environment to the environment defined by the target executable file. At this point, the child process containing the target executable file becomes the calibration process, responsible for subsequent camera calibration operations and possessing independent execution logic and resource usage.

[0063] Step A3: Call the calibration process to run the target execution file, and when the target execution file is running, create a corresponding processing thread according to the scenario requirements.

[0064] Specifically, after the calibration process is created, the target execution file will be immediately executed. During the execution of the target execution file, the program will analyze the scenario requirements in detail and then create a processing thread.

[0065] Among them, creating corresponding processing threads according to scenario requirements includes the following steps: obtaining processing tasks within the scenario requirements and task attributes of the processing tasks; using task attributes to determine the number of threads and the order in which threads are created; and creating processing threads according to the thread creation order and the number of threads.

[0066] First, after receiving the scene calibration request, the host process conducts an in-depth analysis of the scene requirements contained therein, and accurately extracts specific processing tasks, such as corner point detection in the forward, backward, left, and right camera ROI areas, mapping of WCS world coordinate points, and single-channel PnP solution of initial values ​​of external parameters. At the same time, it obtains the task attributes of each processing task, such as task priority, computational complexity, data dependencies, and execution time estimation.

[0067] Next, a comprehensive analysis is performed based on the task attributes. If the tasks have high computational complexity and no strong data dependencies, a larger number of threads are determined for parallel processing based on computing resources and performance requirements. If there is a strict sequence of tasks, the thread creation order is determined according to data dependencies and priorities, and threads that execute predecessor tasks are created first.

[0068] Finally, the calibration program execution file calls the thread creation interface provided by the operating system according to the determined thread creation order and number of threads, and creates processing threads in sequence. Each thread is assigned a specific processing task, and is executed independently to meet scenario requirements based on sharing some parent process resources (code segment, data segment, stack, etc.).

[0069] As an example, Figure 2 As shown in the figure, when the camera calibration program is running, the host process triggers the start of the calibration algorithm process. The calibration algorithm process creates four processing threads based on the internal task division strategy: forward, backward, left, and right processing threads. The forward processing thread focuses on the forward camera ROI area, performs corner detection, completes the mapping of the WCS world coordinate points, and solves the initial values ​​of the extrinsic parameters through a single-path PnP solution; the backward, left, and right processing threads also perform the same operations on their respective cameras. At the same time, the calibration algorithm process also creates four optimization threads. The left front optimization thread is responsible for jointly optimizing the extrinsic parameters of the left front adjacent camera. The left rear, right front, and right rear optimization threads also jointly optimize the extrinsic parameters of the corresponding adjacent camera combinations. These threads work together to complete the camera calibration task.

[0070] Step S103 : calling a processing thread to execute a processing task associated with the scenario requirement to obtain a calibration result, and generating a scene calibration result of the target scene based on the calibration result obtained by each processing thread.

[0071] In the embodiment of the present application, after creating processing threads, these threads are called to execute processing tasks associated with the scene requirements. Each processing thread processes the camera data for which it is responsible according to pre-defined logic, such as performing corner point detection, coordinate mapping, and calculating initial values ​​for external parameters. During processing, the thread records the processing progress and intermediate results in real time. When all processing threads complete their respective tasks, they output their respective calibration results.

[0072] In the embodiment of the present application, generating a scene calibration result of the target scene based on the calibration result obtained by each processing thread includes the following steps B1-B3:

[0073] Step B1: collaboratively analyze the calibration results of each processing thread to obtain the calibration results of adjacent cameras.

[0074] Specifically, the calibration results from all processing threads are first categorized and organized based on the camera installation locations and layout information, clarifying the proximity relationships between cameras. Next, by comparing and matching the calibration data of adjacent cameras at the same feature points or areas, correlations and differences between them are identified. For example, the imaging angle, position, and size of the same target object from adjacent cameras are analyzed, and this data is used to perform consistency checks and error assessments. Finally, the calibration results of adjacent cameras are extracted from the calibration results of each processing thread.

[0075] Step B2: jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters.

[0076] Specifically, the external parameters of the adjacent cameras are jointly optimized using the calibration results of the adjacent cameras to obtain the optimized external parameters, including: calling an optimization thread to jointly optimize the external parameters of the adjacent cameras using the calibration results of the adjacent cameras to obtain the optimized external parameters, wherein the optimization thread is created according to the scene requirements.

[0077] Optimization threads are created based on pre-received scenario requirements. These requirements specify information such as optimization accuracy requirements, optimization time limits, and the camera combinations to be optimized. Based on this information, the number of optimization threads and the task allocation for each thread are determined. For example, if the scenario requirements involve optimizing multiple adjacent camera groups, multiple optimization threads are created, each responsible for optimizing the extrinsic parameters of a group of adjacent cameras.

[0078] After creating the optimization thread, the calibration results of the adjacent cameras are passed to the corresponding optimization thread. The optimization thread will select an appropriate optimization algorithm, such as one based on minimizing the reprojection error, and use the calibration results of the adjacent cameras as input to construct the optimization objective function.

[0079] Then, by continuously iteratively adjusting the extrinsic parameters (such as rotation matrices and translation vectors) of adjacent cameras, the objective function's value is gradually reduced. During this iterative process, the optimization thread monitors the optimization progress and the convergence of the objective function in real time. When the objective function converges to the preset accuracy or reaches the maximum number of iterations, the optimization thread stops iterating and outputs the optimized extrinsic parameters, completing the joint optimization process for adjacent camera extrinsic parameters.

[0080] Specifically, the extrinsic parameters of the adjacent cameras are jointly optimized using the calibration results of the adjacent cameras to obtain the optimized extrinsic parameters, including: calling a calibration process to jointly optimize the extrinsic parameters of the adjacent cameras using the calibration results of the adjacent cameras to obtain the optimized extrinsic parameters.

[0081] The calibration process first loads a pre-set joint optimization algorithm. Then, the calibration process inputs the calibration result data of adjacent cameras into the optimization algorithm model, including information such as the different perspective coordinates of each camera for the same target point and imaging feature points.

[0082] Next, the calibration process iteratively adjusts the extrinsic parameters of adjacent cameras, such as rotation matrices and translation vectors, to optimize the output of the mathematical model. During each iteration, the error value for the current parameter settings is calculated. The calibration process stops when the error value meets the preset convergence condition or reaches the maximum number of iterations. The resulting extrinsic parameters are then considered the optimized extrinsic parameters, completing the joint optimization of the extrinsic parameters of adjacent cameras.

[0083] In step B3, the calibration results of each processing thread are corrected using the optimized external parameters, and the corrected calibration results are fused to obtain a scene calibration result.

[0084] Specifically, based on the external parameter relationship between adjacent cameras, coordinate transformation and error compensation are performed on the calibration results of each processing thread to eliminate calibration errors caused by factors such as camera installation errors and viewing angle differences. The corrected calibration results are more accurate and consistent.

[0085] Next, the corrected calibration results are fused. During the fusion process, each calibration result is assigned a weight based on factors such as the task importance and data reliability of each processing thread. The calibration results from each processing thread are then combined using a weighted average or other fusion algorithm to produce a complete scene calibration result. This scene calibration result accurately reflects the camera calibration status for the entire scene.

[0086] The present invention accurately locates the calibration requirements by obtaining a scene calibration request containing the target scene and scene requirements. Then, according to the request, the corresponding target execution file is obtained and run, and a processing thread corresponding to the scene requirement is created, avoiding the mode of a general program to deal with all scenes, and being able to allocate resources for different scenes. Then, the processing thread is called to execute the associated task to obtain the calibration result, and then the scene calibration result is generated, realizing personalized adaptation of the entire process from task execution to result output. Compared with the existing solution, this on-demand customization method can complete the calibration task more comprehensively and efficiently when facing a variety of calibration scenes, improve the calibration accuracy and efficiency, and meet the differentiated requirements for camera calibration in different scenes.

[0087] As an example, Figure 3 The figure shows an example of multi-process and multi-threaded calls based on the static calibration algorithm executable file for a parking surround view camera. The carlinx_avm_main process with a user interface (UI) serves as the host process. Its UI activates the static calibration function for the parking surround view camera, spawning the calibration trigger thread, calibCameraThread or camera306DThread, which in turn launches the static calibration algorithm process, carlinx_avm_calib. It also creates the progressThread calibration progress acquisition thread to obtain calibration progress. When executing the calibration algorithm, the carlinx_avm_calib process creates four processing threads: forward, backward, left, and right. These threads handle corner detection in the corresponding camera ROI, world coordinate point mapping, and single-path pass-n-pass (PNP) solution for initial extrinsic parameters. It also creates four optimization threads: front left, rear left, front right, and rear right. These threads handle the joint optimization of extrinsic parameters for adjacent cameras. The figure also shows how the carlinx_avm main program uses standard threads to link to related threads and processes, such as the AVM diagnostic thread and the TCP server.

[0088] It's important to note that the carlinx_avm host process, as the core, uses a standard thread mechanism to establish a connection with the AVM diagnostic thread, enabling real-time system status monitoring and diagnosing potential issues, ensuring stable program operation. Simultaneously, the host process drives the TCP server via a standard thread, which in turn connects to the AVMTCP client via a standard thread, establishing a data transmission and interaction channel. The AVMTCP client then connects to the calibration camera thread via a standard thread, which in turn connects to the progress thread to transmit calibration progress.

[0089] Figure 4 is a flow chart of a scene calibration method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0090] Step S201 : obtaining the calibration progress of each processing thread when executing the processing task according to a preset time period.

[0091] In this embodiment of the present application, a fixed time period can be pre-set, such as every 5 minutes. During the camera calibration process, the host process periodically obtains the calibration progress of each processing thread as it executes its tasks, based on this preset time period. By interacting with the processing thread, the host process reads information such as the task completion percentage and the amount of processed data recorded within the thread to determine the current calibration progress.

[0092] For example, for the forward processing thread, the monitoring module can obtain the completion percentage of the diagonal point detection task at the current time point, as well as the completion status of the WCS world coordinate point mapping, etc., and summarize and organize this information to obtain the calibration progress of the thread.

[0093] Step S202 : analyzing the marked progress of the processing thread in each preset time period to obtain the progress variation of the processing thread.

[0094] In the embodiment of the present application, first, the calibrated progress value of the processing thread in each time period is recorded, and then the progress change of the processing thread in the time period is obtained by calculating the difference between the calibrated progress values ​​of two adjacent time periods.

[0095] For example, if the forward processing thread's progress is 20% at the end of the first 5-minute period and 25% at the end of the second 5-minute period, the progress change for that thread between the two periods is 25% - 20% = 5%. This calculation and analysis is performed for all processing threads in each preset time period to ultimately determine the progress change for each processing thread.

[0096] Step S203: The processing thread whose progress change is less than a preset threshold is regarded as a thread to be coordinated.

[0097] In the embodiment of the present application, a threshold is pre-set to measure whether the progress change of the processing thread is normal. After the progress change of all processing threads is calculated, the progress change of each processing thread is compared with the preset threshold. If the progress change of a processing thread is less than the preset threshold, it means that the calibration progress of the thread is slow, and there may be problems such as insufficient resources or excessive task complexity. In this case, the processing thread will be marked as a processing thread to be coordinated.

[0098] For example, the preset threshold is 8%, and the progress change of the forward processing thread is 5%, which is less than the preset threshold. Then, the forward processing thread will be identified as a thread to be coordinated.

[0099] Step S204: obtaining the task content of the task to be processed by the thread to be coordinated, and allocating resource information related to the task content to the thread to be coordinated.

[0100] In the embodiment of the present application, once a thread to be coordinated is identified, the details of the task being performed by that thread are immediately obtained. This includes information such as the type of task (e.g., corner detection, WCS world coordinate point mapping, etc.), the complexity of the task, the required computing resources, and the amount of data. Based on the task content, the type and amount of resources required to complete the task are analyzed, such as the need for more CPU computing power, larger memory space, or more image data.

[0101] Then, appropriate resources are selected from the available resource pool and allocated to the threads to be coordinated. For example, if the thread to be coordinated is making slow progress in the corner detection task due to insufficient CPU computing power, more CPU cores will be allocated to it to speed up the task execution.

[0102] As an example, assume that in the camera calibration program, the preset time period is 10 minutes. When the program is running, every 10 minutes, the calibration progress of each processing thread, such as forward, backward, left, and right, is taken. For example, the forward processing thread completes 30% of the corner detection task. Then analyze the progress of two adjacent 10-minute cycles. If the previous cycle completes 25%, the progress change is 5%. This 5% is then compared with the preset threshold (such as 8%). Since 5% is less than 8%, the forward processing thread becomes the thread to be coordinated. Finally, it is obtained that the task of the forward processing thread is corner detection. Due to the large amount of data processing, the progress is slow, so more memory resources and computing resources are allocated to it to help the task advance efficiently.

[0103] By acquiring the calibration progress of processing threads according to a preset time period, this embodiment of the application can provide real-time information on the execution status of each thread's tasks. By analyzing progress changes, it can accurately locate threads with slow progress. Threads with progress changes less than a preset threshold are designated as threads to be coordinated, clearly identifying those requiring intervention. Furthermore, the task content is acquired and related resource information is allocated, effectively addressing the issue of low thread execution efficiency. Overall, this effectively ensures the balanced and efficient execution of processing thread tasks, promptly identifies and addresses potential issues during execution, and improves the stability and overall efficiency of the camera calibration program.

[0104] In an embodiment of the present application, after analyzing the calibration progress of the processing thread in each preset time period and obtaining the progress change of the processing thread, the method also includes: treating the processing thread whose progress change deviates from the preset range as an exception processing thread; obtaining multimodal data of the current environment, and analyzing the degree of influence of the multimodal data on the calibration progress of the exception processing thread; and performing resource allocation operations on the exception processing thread based on the degree of influence.

[0105] Specifically, first, a preset range of progress change is obtained, for example, the progress change should be between 5% and 15%. After obtaining the progress change of each processing thread within a preset time period (e.g., by a method similar to the above method of obtaining the calibrated progress and calculating the change by period), the progress change of each processing thread is compared with the preset range.

[0106] If the progress change of a processing thread is less than 5% or greater than 15%, the processing thread is determined to be an abnormal processing thread. For example, if the progress change of the forward processing thread in a time period is only 3%, which is lower than the lower limit, it will be marked as an abnormal processing thread.

[0107] Secondly, multimodal data about the current environment is collected in real time. This data includes, but is not limited to, hardware resource data (such as CPU usage, memory utilization, and GPU load) and environmental parameter data (such as temperature and humidity, which may affect operational stability if the device is in a special environment). This multimodal data is then processed using a data analysis algorithm. By establishing a correlation model, the relationship between each type of data and the calibration progress of the exception handling thread is analyzed.

[0108] For example, through historical data and algorithm analysis, it was found that when the CPU usage rate is higher than 80% for a long time, the calibration progress of the processing thread will slow down significantly, thereby determining that the CPU usage rate has a greater impact on the calibration progress of the exception handling thread; if the network delay is too high, resulting in untimely transmission of image data, it will also affect the calibration progress, thereby evaluating the impact of network delay, etc.

[0109] Then, based on the analysis of the impact of multimodal data, a corresponding resource allocation strategy is formulated. If it is found that high CPU usage is significantly impacting the calibration progress of an exception handling thread (such as the forward processing thread), an attempt will be made to allocate CPU resources from other idle processes or threads to the exception handling thread. For example, CPU cores originally allocated to some low-priority background tasks will be temporarily allocated to the forward processing thread.

[0110] If the analysis shows that insufficient memory is the main influencing factor, some useless cache data will be cleared, memory space will be released and allocated to the exception handling thread; if the network delay problem is prominent, the data transmission channel bandwidth of the exception handling thread will be prioritized, and the transmission priority of other non-critical data will be reduced, etc., so as to improve the operating status of the exception handling thread and improve its calibration progress through reasonable resource allocation operations.

[0111] The embodiment of the present application can accurately identify abnormal processing threads. By marking threads whose progress changes deviate from the preset range as abnormal, threads with execution efficiency problems can be quickly located to avoid the problem from escalating. Secondly, multimodal data is obtained and its impact is analyzed, and the effects of various factors such as hardware resources, environmental parameters, and network conditions on the thread calibration progress are fully considered, which can deeply analyze the root cause of the problem. Finally, resources are allocated based on the degree of impact, which can specifically solve the resource bottlenecks faced by abnormal threads, realize the reasonable dynamic allocation of resources, improve the thread operation efficiency, ensure the overall stability and efficient operation of the camera calibration program, and reduce the delay or failure of calibration tasks caused by thread abnormalities.

[0112] This embodiment also provides a thread calling device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0113] This embodiment provides a scene calibration device, such as Figure 5 As shown, including:

[0114] An acquisition module 501 is configured to acquire a scene calibration request, wherein the scene calibration request includes a target scene and scene requirements of the target scene;

[0115] A response module 502 is configured to respond to a scene calibration request, obtain a target execution file of a camera calibration program corresponding to a target scene, and create a processing thread corresponding to the scene requirement by running the target execution file;

[0116] The calling module 503 is used to call the processing thread to execute the processing task associated with the scene requirement to obtain the calibration result, and generate the scene calibration result of the target scene based on the calibration result obtained by each processing thread.

[0117] In this embodiment of the present application, the response module 502 is specifically configured to call a subprocess based on a target scenario, load a target executable file into the subprocess, and use the subprocess containing the target executable file as a calibration process. The calibration process is called to run the target executable file, and when the target executable file runs, a corresponding processing thread is created according to the scenario requirements.

[0118] In an embodiment of the present application, the response module 502 is specifically used to obtain the processing tasks and task attributes of the processing tasks within the scenario requirements; use the task attributes to determine the number of threads and the thread creation order; and create processing threads according to the thread creation order and the number of threads.

[0119] In an embodiment of the present application, the calling module 503 is specifically used to collaboratively analyze the calibration results of each processing thread to obtain the calibration results of adjacent cameras; jointly optimize the external parameters of the adjacent cameras using the calibration results of the adjacent cameras to obtain optimized external parameters; correct the calibration results of each processing thread using the optimized external parameters, and fuse them based on the corrected calibration results to obtain the scene calibration results.

[0120] In an embodiment of the present application, module 503 is called to specifically call an optimization thread to jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters, wherein the optimization thread is created based on scene requirements; or, a calibration process is called to jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters.

[0121] In an embodiment of the present application, the device also includes: an analysis module for obtaining the calibrated progress of each processing thread when executing a processing task according to a preset time period; analyzing the calibrated progress of the processing thread in each preset time period to obtain the progress change of the processing thread; treating the processing thread whose progress change is less than a preset threshold as a processing thread to be coordinated; obtaining the task content of the processing task executed by the processing thread to be coordinated, and allocating resource information related to the task content to the processing thread to be coordinated.

[0122] In an embodiment of the present application, the analysis module is used to treat the processing thread whose progress change deviates from a preset range as an exception processing thread; obtain multimodal data of the current environment, and analyze the degree of influence of the multimodal data on the calibration progress of the exception processing thread; and perform resource allocation operations on the exception processing thread based on the degree of influence.

[0123] See also Figure 6 , Figure 6 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 6As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0124] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0125] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0126] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created based on the use of an electronic device presented by a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0127] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0128] The electronic device further includes a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0129] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0130] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A scene calibration method, characterized in that: The method comprises: Obtaining a scene calibration request, wherein the scene calibration request includes a target scene and scene requirements of the target scene; In response to the scene calibration request, obtaining a target execution file of a camera calibration program corresponding to the target scene, and creating a processing thread corresponding to the scene requirement by running the target execution file; The processing thread is called to execute the processing task associated with the scene requirement to obtain a calibration result, and a scene calibration result of the target scene is generated based on the calibration result obtained by each of the processing threads.

2. The method according to claim 1, characterized in that The step of creating a processing thread corresponding to the scenario requirement by running the target execution file includes: Invoke a subprocess based on the target scenario; Loading the target execution file into the sub-process, and using the sub-process including the target execution file as a calibration process; The calibration process is called to run the target execution file, and when the target execution file is running, a corresponding processing thread is created according to the scenario requirements.

3. The method according to claim 2, characterized in that The step of creating a corresponding processing thread according to the scenario requirements includes: Obtaining the processing tasks within the scenario requirements and the task attributes of the processing tasks; Determine the number of threads and the order in which threads are created using the task attributes; Processing threads are created according to the thread creation order and the thread quantity.

4. The method according to claim 1, wherein Generating the scene calibration result of the target scene based on the calibration result obtained by each processing thread includes: Collaboratively analyze the calibration results of each processing thread to obtain the calibration results of adjacent cameras; The calibration results of adjacent cameras are used to jointly optimize the extrinsic parameters of adjacent cameras to obtain the optimized extrinsic parameters; The calibration results of each processing thread are corrected using the optimized external parameters, and fused based on the corrected calibration results to obtain the scene calibration result.

5. The method according to claim 4, characterized in that The method of jointly optimizing the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters includes: Calling an optimization thread to jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain optimized extrinsic parameters, wherein the optimization thread is created based on the scene requirements; or The calibration process is called to jointly optimize the extrinsic parameters of adjacent cameras using the calibration results of adjacent cameras to obtain the optimized extrinsic parameters.

6. The method according to claim 1, wherein The method further comprises: Obtain the calibration progress of each processing thread when executing the processing task according to the preset time period; Analyzing the calibration progress of the processing thread in each preset time period to obtain a progress change of the processing thread; The processing thread whose progress change is less than a preset threshold is regarded as a processing thread to be coordinated; The task content of the processing task executed by the thread to be coordinated is obtained, and resource information related to the task content is allocated to the thread to be coordinated.

7. The method according to claim 6, characterized in that After analyzing the calibrated progress of the processing thread in each preset time period to obtain the progress change of the processing thread, the method further includes: The processing thread whose progress change deviates from the preset range is regarded as an exception processing thread; Acquiring multimodal data of the current environment and analyzing the degree of influence of the multimodal data on the calibration progress of the exception handling thread; A resource allocation operation is performed to the exception handling thread based on the impact level.

8. A scene calibration device, characterized in that: The device comprises: An acquisition module, configured to acquire a scene calibration request, wherein the scene calibration request includes a target scene and scene requirements of the target scene; A response module, configured to respond to the scene calibration request, obtain a target execution file of a camera calibration program corresponding to the target scene, and create a processing thread corresponding to the scene requirement by running the target execution file; The calling module is used to call the processing thread to execute the processing task associated with the scene requirement to obtain a calibration result, and generate a scene calibration result of the target scene based on the calibration result obtained by each processing thread.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images