Calibration method, device and storage medium of camera

Abstract

The application provides a camera calibration method, device and storage medium, the method comprising: calling the camera to collect multiple frames of original image data of the car when receiving a calibration instruction triggered by the camera installed in the elevator car; measuring the depth value of the pixel points in the original image data according to the geometric parameters of the car and the basic parameters of the camera to obtain actual depth image data; fusing the multiple frames of actual depth image data into target depth image data; constructing theoretical depth image data according to the geometric parameters of the car and the basic parameters of the camera; constructing the transformation relationship between the target depth image data and the theoretical depth image data; and generating the internal and external parameters of the camera according to the transformation relationship. The embodiment can effectively reduce the influence of light and the like, effectively improve the efficiency of camera calibration, and is suitable for large-scale deployment in elevators.

Description

Technical Field

[0001] This invention belongs to the technical field of elevators, and in particular relates to a camera calibration method, device and storage medium. Background Technology

[0002] Cameras are typically installed in elevator cars to collect image data from parts of the car and waiting area. The image data is then processed to provide intelligent services such as security and elevator door control.

[0003] When installing cameras in elevator cars, technicians typically use tools such as checkerboards and calibration boards to manually calibrate the cameras.

[0004] Manual calibration has a high technical threshold, and the calibration process is greatly affected by factors such as lighting, making it difficult to mark in the elevator car and waiting hall environment, resulting in low calibration efficiency when elevators are deployed on a large scale. Summary of the Invention

[0005] In view of this, the present invention provides a camera calibration method, device and storage medium to improve the efficiency of marking elevator cameras.

[0006] A first aspect of the present invention provides a camera calibration method, comprising:

[0007] Upon receiving a calibration command triggered by a camera installed inside the elevator car, the camera is invoked to acquire multiple frames of raw image data of the elevator car.

[0008] The depth values ​​of the pixels in the original image data are measured based on the geometric parameters of the car and the basic parameters of the camera to obtain the actual depth image data;

[0009] The actual depth image data from multiple frames is fused into target depth image data;

[0010] Theoretical depth image data is constructed based on the geometric parameters of the car and the basic parameters of the camera;

[0011] Construct the transformation relationship between the target depth image data and the theoretical depth image data;

[0012] The intrinsic and extrinsic parameters of the camera are generated based on the transformation relationship.

[0013] A second aspect of the present invention provides a camera calibration apparatus, comprising:

[0014] The raw image data acquisition module is used to call the camera to acquire multiple frames of raw image data of the elevator car when a calibration command is received that triggers the camera installed in the elevator car;

[0015] The actual depth image data generation module is used to measure the depth values ​​of pixels in the original image data based on the geometric parameters of the car and the basic parameters of the camera to obtain actual depth image data.

[0016] The target depth image data fusion module is used to fuse multiple frames of actual depth image data into target depth image data;

[0017] The theoretical depth image data construction module is used to construct theoretical depth image data based on the geometric parameters of the car and the basic parameters of the camera;

[0018] A transformation relationship construction module is used to construct the transformation relationship between the target depth image data and the theoretical depth image data;

[0019] The parameter generation module is used to generate the intrinsic and extrinsic parameters of the camera based on the transformation relationship.

[0020] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements a camera calibration method as described in the first aspect above.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the camera calibration method as described in the first aspect above.

[0022] A fifth aspect of the present invention provides a computer program product that, when run on a computer, causes the computer to perform the camera calibration method as described in the first aspect above.

[0023] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0024] In this embodiment, upon receiving a calibration command triggered on a camera installed inside an elevator car, the camera is invoked to acquire multiple frames of raw image data of the car. Depth values ​​of pixels in the raw image data are measured based on the car's geometric parameters and the camera's fundamental parameters to obtain actual depth image data. These multiple frames of actual depth image data are then fused into target depth image data. Theoretical depth image data is constructed based on the car's geometric parameters and the camera's fundamental parameters. A transformation relationship between the target depth image data and the theoretical depth image data is established. The camera's intrinsic and extrinsic parameters are generated based on this transformation relationship. This embodiment performs self-calibration of the camera within an elevator scenario, fusing multiple frames of actual depth image data into target depth image data. This effectively reduces the impact of lighting conditions, and prior knowledge of elevators in similar or identical scenarios can be generalized, significantly improving camera calibration efficiency and making it suitable for large-scale elevator deployment. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of a camera calibration method provided in an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of a camera calibration device provided in an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will recognize that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of the present application.

[0030] The technical solution of the present invention will be illustrated below through specific embodiments.

[0031] Reference Figure 1 The diagram illustrates a camera calibration method according to an embodiment of the present invention, which may specifically include the following steps:

[0032] Step 101: Upon receiving a calibration command triggered by the camera installed in the elevator car, the camera is invoked to collect multiple frames of raw image data of the car.

[0033] Different types of buildings, especially high-rise buildings, have different transportation needs for people, pets, and goods. Therefore, different types of elevators can be deployed in buildings according to different transportation needs, such as passenger elevators, freight elevators, sightseeing elevators, etc. This embodiment does not impose any restrictions on this.

[0034] The structure of elevators also varies among different types of elevators.

[0035] In one example, the components of a certain type of elevator include an elevator control system, car, traction machine, control cabinet, speed governor, door operator, car frame, car door, counterweight guide rail, car guide rail, guide rail support, traveling cable, counterweight device, compensating chain (cable), landing door, guide device for compensating chain (cable), buffer, etc.

[0036] In some types of elevators, the traction machine, control cabinet, speed governor, traveling cable, etc., can be omitted.

[0037] These devices can be divided into different sets according to their functions, thus forming various subsystems that support the operation of the elevator. The elevator control system is connected to the multiple systems of the elevator via wired means such as serial port or serial clock line (SCL). The elevator control system monitors each system and controls the operation of each subsystem, so that the car moves in the hoistway and reaches each floor of the building.

[0038] In one example, the elevator control system includes a door system, a frequency converter system, a call system, and a traction system. The door system controls the elevator doors. The car is equipped with a car door, and the elevator has hall doors on each floor. The elevator doors include the car door and the hall doors on each floor. The car door and the hall door are of the same type and open and close simultaneously. The frequency converter system controls the frequency converter. The call system controls the logic of internal call (calling the elevator from inside the car) and external call (calling the elevator from the hall). The traction system controls the car to move vertically (vertically upward or vertically downward) in the hoistway.

[0039] Elevator managers or owners can choose whether to install edge computing nodes on the elevator based on factors such as the elevator's load status. If they choose not to install edge computing nodes, the elevator control system will maintain its original control logic and will not affect the normal operation of the elevator. If they choose to install edge computing nodes, they can select appropriate computing devices as the elevator's edge computing nodes according to their needs. The edge computing nodes will be combined with the original elevator control system to form a new elevator control system and redefine the elevator's control logic.

[0040] Generally, edge computing nodes are computing devices with strong computing capabilities, such as computers, servers, or embedded devices. In addition, depending on the different intelligent services, edge computing nodes can be equipped with graphics processing units (GPUs) or embedded neural network processors (NPUs).

[0041] Edge computing nodes refer to new business platforms built at the network edge near elevators, providing storage, computing, and network resources. This allows some critical business applications to be offloaded to the edge of the access network, reducing bandwidth and latency losses caused by network transmission and multi-level forwarding. Located between the user and the cloud (server), edge computing nodes are closer to the user (data source) than traditional cloud computing. They are characterized by miniaturization, distribution, and closer proximity to the user. Massive amounts of data (such as audio data) no longer need to be uploaded to the cloud for processing; data processing can be achieved at the network edge, reducing request response time, reducing network bandwidth, and ensuring data security and privacy.

[0042] In addition, edge computing nodes can realize algorithm functions and model inference, communicate with the original elevator control system, and provide the original elevator control system with artificial intelligence (AI) and complex computing capabilities; edge computing can also communicate with the cloud to realize algorithm functions and model updates, and relay the original elevator control system's function calls, etc.

[0043] In this embodiment, calibration commands can be triggered on the camera installed in the elevator car in multiple scenarios and at multiple times to iteratively update the camera's intrinsic and extrinsic parameters, so that the camera's intrinsic and extrinsic parameters gradually converge to the optimal solution.

[0044] That is, this embodiment supports online adaptive optimization of the camera. The camera's internal and external parameters will be automatically corrected as the usage environment changes, thus achieving self-learning.

[0045] For example, calibration instructions are generated in at least one of the following scenarios:

[0046] Inside the car where a new elevator is initially installed;

[0047] The installation configuration parameters were modified inside the elevator car;

[0048] The set update time has been reached.

[0049] Furthermore, the goal of multi-scene optimization is that regardless of which elevator car the camera is installed in, the geometric parameters (spatial dimensions) of the elevator car and the installation parameters (azimuth angle) of the camera are known. Multi-scene optimization is based on the same process of recalibration, which eliminates the need for manual annotation for different scenes and effectively avoids the deviations that may be caused by manual annotation.

[0050] The goal of multi-moment optimization is to prevent data deviations caused by changes in camera position after manual calibration when the camera has been reinstalled or external factors have caused a deviation from its initial installation position. This can be achieved by periodically or by triggering recalibration in conjunction with certain logic.

[0051] In this embodiment, the calibration command triggered by the camera installed in the elevator car can be listened to whether the elevator is stationary or running. When the calibration command triggered by the camera installed in the elevator car is received, the camera is invoked to collect multiple frames of raw image data of the elevator car.

[0052] Step 102: Measure the depth values ​​of the pixels in the original image data based on the car's geometric parameters and the camera's basic parameters to obtain the actual depth image data.

[0053] In this embodiment, based on a monocular camera model, the depth values ​​of pixels in each frame of the original image data can be measured according to the geometric parameters of the car and the basic parameters of the camera to obtain dimensionless actual depth image data.

[0054] Step 103: Fuse multiple frames of actual depth image data into target depth image data.

[0055] In general, the space inside an elevator car is relatively small, and the changes in lighting caused by events such as door opening and closing and passenger movement significantly affect camera imaging. There are errors such as scale shift and nonlinear shift (i.e., the difference between the same position in the original image data is not completely linear) between multiple frames of actual depth image data after inference from multiple frames of original image data. The reliability of single-frame actual depth image data for camera calibration is low. Therefore, multiple frames of actual depth image data can be fused into target depth image data through inter-frame fusion (such as weighting, filtering, geometric consistency, etc.) to eliminate single-frame noise and inference jitter, obtain a stable scene depth representation, and ensure the convergence of subsequent camera intrinsic and extrinsic parameter optimization.

[0056] In one embodiment of the present invention, step 103 may include the following steps:

[0057] Step 1031: Detect the pixels representing the human body in the actual depth image data and use them as target points.

[0058] In this embodiment, the pixels in the actual depth image data contain both color components (such as RGB (red, green, and blue) values) and depth values. Pixels representing the human body can be detected in the actual depth image data using methods such as object detection and semantic segmentation, and used as target points.

[0059] Step 1032: Divide the actual depth image data into multiple raw depth blocks.

[0060] In this embodiment, the same method can be used to divide multiple frames of actual depth image data into multiple raw depth blocks, each of which has a number.

[0061] For example, the actual depth image data is divided into multiple original depth blocks of the same size, with each original depth block being 16×16.

[0062] Step 1033: Filter out all original depth blocks where all pixels are non-target points to obtain candidate depth blocks.

[0063] For each frame of actual depth image data, traverse each original depth region and select the original depth blocks from all original depth blocks where all pixels are non-target points, and record them as candidate depth blocks.

[0064] At this point, the candidate depth block is the area not obscured by the passenger; conversely, the original depth block, where some or all pixels are the target points, is the area obscured by the passenger.

[0065] Step 1034: Merge multiple candidate depth blocks under the same number into a target depth block.

[0066] In this embodiment, multiple candidate depth blocks with the same number in multiple frames of actual depth image data can be fused into a target depth block using linear or nonlinear methods.

[0067] Furthermore, the number of multiple candidate depth blocks under the same number can be counted and compared with a preset threshold k.

[0068] If the number is greater than or equal to the preset threshold k, then multiple candidate depth blocks under the same number will be merged into the target depth block.

[0069] If the number is less than the preset threshold k, then continue to collect raw image data until the number of multiple candidate depth blocks under the same number is greater than or equal to the preset threshold k.

[0070] In one fusion method, the proportion of target points in the original image data is statistically analyzed, and the shortest distance between the candidate depth region and the target point is calculated.

[0071] The target point ratio is the ratio between the number of target points and the total number of pixels, representing the proportion of the original image data that is obscured by passengers.

[0072] Generally, the shortest distance between a representative pixel (such as the center) in the candidate depth region and the target point can be calculated as the shortest distance between the candidate depth region and the target point.

[0073] The proportion is mapped to the upper limit of the weight; where the upper limit of the weight is negatively correlated with the proportion of the target point, that is, the higher the proportion of the target point, the more obvious the impact of changes in lighting, and the lower the upper limit of the weight, and vice versa.

[0074] For example, the difference between the percentage of 1 and the target point can be calculated as the upper limit value.

[0075] For example, if the target point accounts for 50%, the upper limit is 0.5; if the target point accounts for 60%, the upper limit is 0.4, and so on.

[0076] The upper limit value and the preset lower limit value (such as 0) form the fluctuation range.

[0077] The candidate depth region is assigned a weight within the fluctuation range based on the shortest distance between the candidate depth region and the target point. The weight is positively correlated with the shortest distance; that is, the larger the shortest distance between the candidate depth region and the target point, the greater the weight of the candidate depth region, and vice versa.

[0078] For example, quantiles can be set within the fluctuation range according to the number of candidate depth regions, the candidate depth regions can be sorted, and the values ​​corresponding to the quantiles can be assigned as weights to the corresponding sorted candidate depth regions.

[0079] For example, in a frame of actual depth image data, there are 6 candidate depth regions, which are sorted from largest to smallest according to the shortest distance (i.e., from farthest to nearth relative to the target point) and denoted as region A, region B, region C, region D, region E, and region F, with a fluctuation range of [0, 0.5]. Then, the weight of region A is 0.5, the weight of region B is 0.4, the weight of region C is 0.3, the weight of region D is 0.2, the weight of region E is 0.1, and the weight of region F is 0.

[0080] For multiple candidate depth blocks under the same number, the product between the candidate depth block (especially the depth value of the pixel) and the weight is added to obtain the target depth block.

[0081] Step 1035: Stitch together the target depth blocks under multiple numbers into target depth image data.

[0082] In this embodiment, multiple target depth blocks under different numbers can be normalized, and the normalized target depth blocks under different numbers can be stitched together according to the positions indicated by the numbers to obtain target depth image data.

[0083] Step 104: Construct theoretical depth image data based on the car's geometric parameters and the camera's basic parameters.

[0084] In this embodiment, theoretical depth image data, i.e., spatial theoretical depth image data, can be generated based on the camera's basic parameters (such as installation angle, three-dimensional spatial orientation of installation, field of view (FoV), optical center, etc.) and prior scene knowledge (i.e., the geometric parameters of the elevator car (such as the length, width, and height of the elevator car)) combined with the pinhole model, and can serve as a reference benchmark for subsequent optimization.

[0085] Step 105: Construct the transformation relationship between the target depth image data and the theoretical depth image data.

[0086] In three-dimensional space, a transformation relationship is constructed between theoretical depth image data and target depth image data. This transformation relationship includes affine transformation matrix, projection transformation matrix, etc., which maps the virtual theoretical depth to the actual depth, forming a mathematical bridge for inferring the intrinsic and extrinsic parameters of the camera.

[0087] In one embodiment of the present invention, step 105 may include the following steps:

[0088] Step 1051: Fit target boundary data into the target depth image data.

[0089] In this embodiment, the intersection lines (i.e. geometric boundaries) between the main scenes (such as the bottom of the car, the side walls of the car, the top of the car, etc.) can be extracted from the target depth image data. The intersection lines (i.e. geometric boundaries) are simplified using algorithms such as RANSAC (Random Sample Consensus) and multi-plane fitting to obtain the target boundary data.

[0090] The fitting process transforms dense target depth image data into geometric constraint expressions that can be used for parameter optimization. In other words, it abstracts pixel-level data into the geometric structure of the scene, thereby transforming irregular depth data into controllable geometric references and creating conditions for aligning theoretical depth image data.

[0091] In one fitting method, multiple origins can be set in the target depth image data, and multiple sets of rays in different directions can be emitted from the origins.

[0092] The changes in depth values ​​are observed along the pixels (i.e., trajectory points) on the ray, and the depth values ​​of the pixels along the ray are filtered to obtain key points; among them, key points are pixels whose depth values ​​change abruptly, and the key points are used to fit the target boundary data.

[0093] Step 1052: Query the theoretical boundary data in the theoretical depth image data.

[0094] In this embodiment, the intersection lines (i.e. geometric boundaries) between the main scenes (such as the bottom of the car, the side walls of the car, the top of the car, etc.) can be queried from the theoretical depth image data to obtain the theoretical boundary data.

[0095] Step 1053: Adjust the theoretical boundary data with reference to the target boundary data.

[0096] In this embodiment, the theoretical boundary data is transformed in three-dimensional space using the target boundary data as a reference, and the spatial geometric features of the theoretical boundary data are fine-tuned.

[0097] In the specific implementation, a k-dimensional tree (KD tree) is constructed using the target boundary data and the theoretical boundary data. The residual vector r between the pixels on the target boundary data and the pixels on the theoretical boundary data is calculated in the k-dimensional tree, where r = pa, p is the pixel at the same relative position on the target boundary data, and a is the pixel at the same relative position on the theoretical boundary data. Multiple residual vectors can be obtained by comparing the target boundary data and the theoretical boundary data.

[0098] The theoretical boundary data is adjusted with the goal of minimizing the residual vector. Minimizing the residual vector means that the adjustment objective of the theoretical depth image can be represented as joint optimization. The goal of subsequent camera intrinsic and extrinsic parameter back-calculation is to minimize the residual vector.

[0099] Step 1054: When the target boundary data is aligned with the theoretical boundary data, construct the transformation relationship between the target depth image data and the theoretical depth image data.

[0100] When aligning the target boundary data with the theoretical boundary data, a transformation relationship can be constructed between the target depth image data and the theoretical depth image data.

[0101] Step 106: Generate the camera's intrinsic and extrinsic parameters based on the transformation relationship.

[0102] In this embodiment, algorithms such as ICP (Iterated Closest Points) can be used to calculate the camera's extrinsic parameters (i.e., rotation matrix, translation matrix, etc.) based on the transformation relationship between the target depth image data and the theoretical depth image data. Based on the parameters of the pinhole model and combined with the transformation relationship between the target depth image data and the theoretical depth image data, algorithms such as least squares can be used to calculate the camera's intrinsic parameters (i.e., horizontal focal length, vertical focal length, x-coordinate of the principal point, y-coordinate of the principal point, etc.).

[0103] In this embodiment, upon receiving a calibration command triggered on a camera installed inside an elevator car, the camera is invoked to acquire multiple frames of raw image data of the car. Depth values ​​of pixels in the raw image data are measured based on the car's geometric parameters and the camera's fundamental parameters to obtain actual depth image data. These multiple frames of actual depth image data are then fused into target depth image data. Theoretical depth image data is constructed based on the car's geometric parameters and the camera's fundamental parameters. A transformation relationship between the target depth image data and the theoretical depth image data is established. The camera's intrinsic and extrinsic parameters are generated based on this transformation relationship. This embodiment performs self-calibration of the camera within an elevator scenario, fusing multiple frames of actual depth image data into target depth image data. This effectively reduces the impact of lighting conditions, and prior knowledge of elevators in similar or identical scenarios can be generalized, significantly improving camera calibration efficiency and making it suitable for large-scale elevator deployment.

[0104] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0105] Reference Figure 2 The diagram illustrates a camera calibration device according to an embodiment of the present invention, which may specifically include the following modules:

[0106] The raw image data acquisition module 201 is used to call the camera to acquire multiple frames of raw image data of the elevator car when a calibration command is received that triggers the camera installed in the elevator car;

[0107] The actual depth image data generation module 202 is used to measure the depth value of the pixels in the original image data based on the geometric parameters of the car and the basic parameters of the camera to obtain actual depth image data;

[0108] The target depth image data fusion module 203 is used to fuse multiple frames of actual depth image data into target depth image data.

[0109] Theoretical depth image data construction module 204 is used to construct theoretical depth image data based on the geometric parameters of the car and the basic parameters of the camera;

[0110] Transformation relationship construction module 205 is used to construct the transformation relationship between the target depth image data and the theoretical depth image data;

[0111] The parameter generation module 206 is used to generate the intrinsic and extrinsic parameters of the camera based on the transformation relationship.

[0112] In one embodiment of the present invention, the target depth image data fusion module 203 includes:

[0113] The target point detection module is used to detect pixels representing the human body in the actual depth image data as target points;

[0114] The raw depth block segmentation module is used to segment the depth image data into multiple raw depth blocks; each raw depth block is numbered.

[0115] The candidate depth block filtering module is used to filter out the original depth blocks in which all pixels are non-target points to obtain candidate depth blocks;

[0116] The target depth block fusion module is used to fuse multiple candidate depth blocks under the same number into a target depth block;

[0117] The target depth image data stitching module is used to stitch together multiple target depth blocks under the specified number into target depth image data.

[0118] In one embodiment of the present invention, the target depth block fusion module includes:

[0119] The quantity statistics module is used to count the number of multiple candidate depth blocks under the same number;

[0120] The confidence fusion module is used to fuse multiple candidate depth blocks under the same number into a target depth block if the number is greater than or equal to a preset threshold.

[0121] In one embodiment of the present invention, the target depth block fusion module includes:

[0122] The image statistics module is used to count the proportion of the target point in the actual depth image data and to calculate the shortest distance between the candidate depth region and the target point;

[0123] An upper limit mapping module is used to map the percentage to an upper limit value; the upper limit value is negatively correlated with the percentage.

[0124] A fluctuation range composition module is used to compose the fluctuation range by combining the upper limit value and the preset lower limit value;

[0125] A weight selection module is used to select weights for the candidate depth regions within the fluctuation range according to the shortest distance; the weights are positively correlated with the shortest distance.

[0126] The weighted fusion module is used to add the product between the candidate depth blocks and the weights for multiple candidate depth blocks under the same number to obtain the target depth block.

[0127] In one embodiment of the present invention, the transformation relationship construction module 205 includes:

[0128] A target boundary data fitting module is used to fit target boundary data into the target depth image data;

[0129] Theoretical boundary data query module, used to query theoretical boundary data in the theoretical depth image data;

[0130] Theoretical boundary data adjustment module, used to adjust the theoretical boundary data with reference to the target boundary data;

[0131] The transformation relationship construction module is used to construct the transformation relationship between the target depth image data and the theoretical depth image data when the target boundary data is adjusted to be aligned with the theoretical boundary data.

[0132] In one embodiment of the present invention, the target boundary data fitting module includes:

[0133] The origin setting module is used to set multiple origins in the target depth image data;

[0134] A ray emitting module is used to emit multiple sets of rays in different directions from the origin.

[0135] A filtering module is used to filter the depth values ​​of pixels along the ray to obtain key points; the key points are pixels whose depth values ​​change abruptly.

[0136] The key point fitting module is used to fit the target boundary data using the key points.

[0137] In one embodiment of the present invention, the theoretical boundary data adjustment module includes:

[0138] The k-dimensional tree construction module is used to construct a k-dimensional tree using the target boundary data and the theoretical boundary data.

[0139] The residual vector calculation module is used to calculate the residual vector between the pixels on the target boundary data and the pixels on the theoretical boundary data in the k-dimensional tree;

[0140] The optimization and adjustment module is used to adjust the theoretical boundary data with the goal of minimizing the residual vector.

[0141] In one embodiment of the present invention, the calibration command is generated in at least one of the following scenarios:

[0142] Inside the car where a new elevator is initially installed;

[0143] The installation configuration parameters were modified inside the elevator car;

[0144] The set update time has been reached.

[0145] The present invention provides a camera calibration device, which can be used to implement the steps in the aforementioned camera calibration method embodiments.

[0146] It should be noted that the module division in the various camera calibration devices provided in the above embodiments is illustrative and only represents a logical functional division. In actual implementation, other division methods may also be used. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0147] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of the embodiments of the present invention can be embodied in the form of a computer program product, which is stored in a computer storage medium and includes several instructions to cause an electronic device or processor to execute all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned computer storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] Furthermore, the camera calibration device and camera calibration method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0149] Reference Figure 3 The diagram illustrates an electronic device according to an embodiment of the present invention. Figure 3 As shown, the electronic device in this embodiment of the invention includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above-described camera calibration method embodiment. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described camera calibration device embodiment.

[0150] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which can be used to describe the execution process of the computer program in the electronic device.

[0151] The electronic device may be a desktop computer, a cloud server, or other computing device. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 3 This is merely one example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0152] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0153] The memory can be an internal storage unit of the electronic device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0154] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the camera calibration method as described in the foregoing embodiments.

[0155] This invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the camera calibration method as described in the foregoing embodiments.

[0156] This invention also discloses a computer program product that, when run on a computer, causes the computer to execute the camera calibration method described in the foregoing embodiments.

[0157] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A camera calibration method, characterized in that, include: Upon receiving a calibration command triggered by a camera installed inside the elevator car, the camera is invoked to acquire multiple frames of raw image data of the elevator car. The depth values ​​of the pixels in the original image data are measured based on the geometric parameters of the car and the basic parameters of the camera to obtain the actual depth image data; The actual depth image data from multiple frames is fused into target depth image data; Theoretical depth image data is constructed based on the geometric parameters of the car and the basic parameters of the camera; Construct the transformation relationship between the target depth image data and the theoretical depth image data; The intrinsic and extrinsic parameters of the camera are generated based on the transformation relationship; The step of fusing multiple frames of depth image data into target depth image data includes: Pixels representing the human body are detected in the actual depth image data and used as target points; The depth image data is divided into multiple raw depth blocks; each raw depth block is numbered. The original depth blocks in which all pixels are non-target points are filtered out to obtain candidate depth blocks; Multiple candidate depth blocks under the same number are merged into a target depth block; The target depth blocks under the multiple numbered sections are stitched together to form target depth image data; The process of constructing the transformation relationship between the target depth image data and the theoretical depth image data includes: Fit target boundary data into the target depth image data; Query the theoretical boundary data in the theoretical depth image data; The theoretical boundary data are adjusted based on the target boundary data; When the target boundary data is adjusted to be aligned with the theoretical boundary data, a transformation relationship between the target depth image data and the theoretical depth image data is constructed.

2. The method according to claim 1, characterized in that, The step of fusing multiple candidate depth blocks under the same number into a target depth block includes: Count the number of multiple candidate depth blocks under the same number; If the number is greater than or equal to a preset threshold, then multiple candidate depth blocks under the same number will be merged into a target depth block.

3. The method according to claim 1, characterized in that, The step of fusing multiple candidate depth blocks under the same number into a target depth block includes: The proportion of the target point in the actual depth image data is statistically analyzed, and the shortest distance between the candidate depth region and the target point is calculated. The percentage is mapped to an upper limit value; the upper limit value is negatively correlated with the percentage. The upper limit value and the preset lower limit value form a fluctuation range; Weights are selected for the candidate depth regions within the fluctuation range according to the shortest distance; the weights are positively correlated with the shortest distance. For multiple candidate depth blocks under the same number, the product between the candidate depth block and the weight is added to obtain the target depth block.

4. The method according to claim 1, characterized in that, Fitting target boundary data into the target depth image data includes: Multiple origins are set in the target depth image data; Multiple sets of rays in different directions are emitted from the origin; The key points are obtained by filtering the depth values ​​of the pixels along the ray; the key points are the pixels whose depth values ​​change abruptly. The target boundary data is fitted using the key points.

5. The method according to claim 4, characterized in that, The adjustment of the theoretical boundary data with reference to the target boundary data includes: Construct a k-dimensional tree using the target boundary data and the theoretical boundary data; Calculate the residual vector between the pixels on the target boundary data and the pixels on the theoretical boundary data in the k-dimensional tree; The theoretical boundary data are adjusted with the goal of minimizing the residual vector.

6. The method according to any one of claims 1-5, characterized in that, The calibration command is generated in at least one of the following scenarios: Inside the car where a new elevator is initially installed; The installation configuration parameters were modified inside the elevator car; The set update time has been reached.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the camera calibration method as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the camera calibration method as described in any one of claims 1-6.

Images